Backtracking&

Branch and Bound

2

Backtracking• Suppose you have to make a series of decisions, among various choices, where– You don’t have enough information to know

what to choose– Each decision leads to a new set of choices– Some sequence of choices (possibly more than

one) may be a solution to your problem• Backtracking is a methodical way of trying

out various sequences of decisions, until you find one that “works”

3

Backtracking Algorithm

• Based on depth-first recursive search• Approach

1. Tests whether solution has been found2. If found solution, return it3. Else for each choice that can be made

a) Make that choiceb) Recurc) If recursion returns a solution, return it

4. If no choices remain, return failure• Some times called “search tree”

Backtracking

• Performs a depth-first traversal of a tree• Continues until it reaches a node that is non-

viable or non-promising• Prunes the sub tree rooted at this node and

continues the depth-first traversal of the tree

5

Backtracking Algorithm – Example

• Find path through maze– Start at beginning of maze– If at exit, return true– Else for each step from current location• Recursively find path• Return with first successful step• Return false if all steps fail

Backtracking• Backtracking is a technique used to solve problems with a

large search space, by systematically trying and eliminating possibilities.

• A standard example of backtracking would be going through a maze. – At some point in a maze, you might have two options of which

direction to go:

JunctionPortion A

Porti

on B

Backtracking

Junction

Portion B

Porti

on A

One strategy would be to try going through Portion A of the maze. If you get stuck before you

find your way out, then you "backtrack" to the junction.

At this point in time you

know that Portion A will NOT lead you out of the maze, so you then start searching

in Portion B

Backtracking• Clearly, at a single junction

you could have even more than 2 choices.

• The backtracking strategy says to try each choice, one after the other, – if you ever get stuck,

"backtrack" to the junction and try the next choice.

• If you try all choices and never found a way out, then there IS no solution to the maze.

Junction

BC

A

Backtracking – Eight Queens Problem

• Find an arrangement of 8 queens on a single chess board such that no two queens are attacking one another.

• In chess, queens can move all the way down any row, column or diagonal (so long as no pieces are in the way).

– Due to the first two restrictions, it's clear that each row and column of the board will have exactly one queen.

• The backtracking strategy is as follows:1) Place a queen on the first available

square in row 1.

2) Move onto the next row, placing a queen on the first available square there (that doesn't conflict with the previously placed queens).

3) Continue in this fashion until either:

a) you have solved the problem, or b) you get stuck. – When you get stuck, remove the queens

that got you there, until you get to a row where there is another valid square to try.

Backtracking – Eight Queens Problem

Animated Example:http://www.hbmeyer.de/backtrack/achtdamen/eight.htm#up

QQ

QQ

Q Q

Continue…

Backtracking – Eight Queens Problem• When we carry out backtracking, an easy

way to visualize what is going on is a tree that shows all the different possibilities that have been tried.

• On the board we will show a visual representation of solving the 4 Queens problem (placing 4 queens on a 4x4 board where no two attack one another).

Backtracking – Eight Queens Problem

• The neat thing about coding up backtracking, is that it can be done recursively, without having to do all the bookkeeping at once.– Instead, the stack or recursive calls does most of

the bookkeeping – (ie, keeping track of which queens we've placed,

and which combinations we've tried so far, etc.)

void solveItRec(int perm[], int location, struct onesquare usedList[]) {

if (location == SIZE) { printSol(perm); } for (int i=0; i<SIZE; i++) {

if (usedList[i] == false) {

if (!conflict(perm, location, i)) { perm[location] = i; usedList[i] = true; solveItRec(perm, location+1, usedList); usedList[i] = false; } } } }

perm[] - stores a valid permutation of queens from index 0 to location-1. location – the column we are placing the next queenusedList[] – keeps track of the rows in which the queens have already been placed.

Found a solution to the problem, so print it!

Loop through possible rows to place this queen.

Only try this row if it hasn’t been used

Check if this position conflicts with any previous queens on the diagonal

1) mark the queen in this row2) mark the row as used3) solve the next column location

recursively4) un-mark the row as used, so we

can get ALL possible valid solutions.

Backtracking – 8 queens problem - Analysis

• Another possible brute-force algorithm is generate the permutations of the numbers 1 through 8 (of which there are 8! = 40,320), – and uses the elements of each permutation as indices to place a queen on each row. – Then it rejects those boards with diagonal attacking positions.

• The backtracking algorithm, is a slight improvement on the permutation method, – constructs the search tree by considering one row of the board at a time,

eliminating most non-solution board positions at a very early stage in their construction.

– Because it rejects row and diagonal attacks even on incomplete boards, it examines only 15,720 possible queen placements.

• A further improvement which examines only 5,508 possible queen placements is to combine the permutation based method with the early pruning method: – The permutations are generated depth-first, and the search space is pruned if the

partial permutation produces a diagonal attack

Sudoku and Backtracking• Another common puzzle that can be solved by backtracking is

a Sudoku puzzle.

• The basic idea behind the solution is as follows:1) Scan the board to look for an empty square that could take on the

fewest possible values based on the simple game constraints.2) If you find a square that can only be one possible value, fill it in

with that one value and continue the algorithm.3) If no such square exists, place one of the possible numbers for

that square in the number and repeat the process.4) If you ever get stuck, erase the last number placed and see if there

are other possible choices for that slot and try those next.

Mazes and Backtracking• A final example of something that can be solved using

backtracking is a maze. – From your start point, you will iterate through each possible

starting move. – From there, you recursively move forward. – If you ever get stuck, the recursion takes you back to where

you were, and you try the next possible move.

• In dealing with a maze, to make sure you don't try too many possibilities, – one should mark which locations in the maze have been visited

already so that no location in the maze gets visited twice. – (If a place has already been visited, there is no point in trying to

reach the end of the maze from there again.

Example: N-Queens Problem

• Given an N x N sized chess board• Objective: Place N queens on the

board so that no queens are in danger

• One option would be to generate a tree of every possible board layout

• This would be an expensive way to find a solution

Backtracking

• Backtracking prunes entire sub trees if their root node is not a viable solution

• The algorithm will “backtrack” up the tree to search for other possible solutions

Efficiency of Backtracking

• This given a significant advantage over an exhaustive search of the tree for the average problem

• Worst case: Algorithm tries every path, traversing the entire search space as in an exhaustive search

Branch and Bound

• Where backtracking uses a depth-first search with pruning, the branch and bound algorithm uses a breadth-first search with pruning

• Branch and bound uses a queue as an auxiliary data structure

The Branch and Bound Algorithm

• Starting by considering the root node and applying a lower-bounding and upper-bounding procedure to it

• If the bounds match, then an optimal solution has been found and the algorithm is finished

• If they do not match, then algorithm runs on the child nodes

Example: The Traveling Salesman Problem

Branch and bound can be used to solve the TSP using a priority queue as an auxiliary data structure

An example is the problem with a directed graph given by this adjacency matrix:

Traveling Salesman Problem• The problem starts at vertex 1• The initial bound for the minimum tour is the

sum of the minimum outgoing edges from each vertex

Vertex 1           min (14, 4, 10, 20)       =  4

Vertex 2           min (14, 7, 8, 7)          =  7

Vertex 3           min (4, 5, 7, 16)          =  4

Vertex 4           min (11, 7, 9, 2)          =  2

Vertex 5           min (18, 7, 17, 4)       =  4

Bound                                                 = 21

Traveling Salesman Problem• Next, the bound for the node for the partial

tour from 1 to 2 is calculated using the formula:Bound = Length from 1 to 2 + sum of min outgoing

edges for vertices 2 to 5 = 14 + (7 + 4 + 2 + 4) = 31

Traveling Salesman Problem

• The node is added to the priority queue• The node with the lowest bound is then

removed• This calculation for the bound for the node of

the partial tours is repeated on this node• The process ends when the priority queue is

empty

Traveling Salesman Problem

• The final results of this example are in this tree:

• The accompanying number for each node is the order it was removed in

Efficiency of Branch and Bound

• In many types of problems, branch and bound is faster than branching, due to the use of a breadth-first search instead of a depth-first search

• The worst case scenario is the same, as it will still visit every node in the tree