scientific programming [24pt]backtrackingdisi.unitn.it/~montreso/sp/slides/b07-backtrack.pdfsi...
TRANSCRIPT
Scientific Programming
Backtracking
Alberto Montresor
Università di Trento
2018/12/15
This work is licensed under a Creative CommonsAttribution-ShareAlike 4.0 International License.
references
Table of contents
1 Introduction2 Enumeration
SubsetsPermutationsGames
Introduction
Introduction
Problem classes (decisional, search, optimization)
Definition bases on the concept of admissible solution: a solutionthat satisfies a given set of criteria
Typical problems
Build one or all admissible solutionCounting the admissible solutionsFind the admissible solution "largest", "smallest", in general"optimal"
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 1 / 37
Introduction
Typical problems
Enumeration
List algorithmically all possible solutions (search space)
Example: list all the permutations of a set
Build at least a solution
We use the algorithm for enumeration, stopping at the firstsolution found
Example: identify a sequence of steps in the Fifteen game
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 2 / 37
Introduction
Typical problems
Count the solutions
In some cases, it is possibleto count in analytical way
Example: counting thenumber of subsets of kelements taken by a set of nelements
n!
k! (n− k)!
In other cases, we build thesolutions and we count them
Example: number of subsetsof a integer set S whose sumis equal to a prime number
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 3 / 37
Introduction
Typical problems
Find optimal solutions
We enumerate all possible solutions and evaluate them through acost function
Only if other techniques are not possible:Dynamic programmingGreedy
Example: Hamiltonian circuit (Traveling salesman)
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 4 / 37
Introduction
Build all solutions
To build all the solutions, we use a "brute-force" approach
Sometimes, it is the only possible way
The power of modern computer makes possible to deal withproblems medium-small problems
10! = 3.63 · 106 (permutation of 10 elements)220 = 1.05 · 106 (subsets of 20 elements)
Sometimes, the space of all possible solutions does not need to beanalyzed entirely
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 5 / 37
Introduction
Backtracking
Approach
"Prova a fare qualcosa, e se non va bene, disfalo e provaqualcos’altro"
"Ritenta, e sarai più fortunato"
How it works?
A systematic methods to iterate all possible instances of a researchspaceIt is an algorithm technique that, as the others, need to be"customized" for the specific problem to be solved.
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 6 / 37
Introduction
General organization
General organization
A solution is represented by a list SThe content of element S[i] is taken from a set of choices C thatdepends on the problem
Examples
C generic set, possible solutions permutations of CC generic set, possible solutions subsets of CC game moves, possible solutions a sequence of movesC edges of a graph, possible solutions paths
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 7 / 37
Introduction
Partial solutions
At each step, we start from a partial solution S where k ≥ 0choices have been already taken
If S[0 : k] is an admissible solution, we "process" itE.g., we can print itWe can then decide to stop here or keep going by listing/printing allsolutions
If S[0 : k] is not a complete solution:If possible, we extended solution S[0 : k] with one of the possiblechoices to get a solution S[0 : k + 1]Otherwise, we "cancel" the element S[k] (backtrack) and we goback to to solution S[0 : k − 1]
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 8 / 37
Introduction
Decision trees
Decision tree ≡ Search spaceRoot ≡ Empty solutionInternal nodes ≡ Partial solutionsLeaves ≡ Admissible solutions
S[0]
S[3]
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 9 / 37
Introduction
Decision trees
Decision tree ≡ Search spaceRoot ≡ Empty solutionInternal nodes ≡ Partial solutionsLeaves ≡ Admissible solutions
S[0]
S[1]
S[3]
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 9 / 37
Introduction
Decision trees
Decision tree ≡ Search spaceRoot ≡ Empty solutionInternal nodes ≡ Partial solutionsLeaves ≡ Admissible solutions
S[0]
S[1]
S[2]
S[3]
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 9 / 37
Introduction
Decision trees
Decision tree ≡ Search spaceRoot ≡ Empty solutionInternal nodes ≡ Partial solutionsLeaves ≡ Admissible solutions
S[0]
S[1]
S[2]
S[3]
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 9 / 37
Introduction
Decision trees
Decision tree ≡ Search spaceRoot ≡ Empty solutionInternal nodes ≡ Partial solutionsLeaves ≡ Admissible solutions
S[0]
S[1]
S[2]
S[3]
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 9 / 37
Introduction
Decision trees
Decision tree ≡ Search spaceRoot ≡ Empty solutionInternal nodes ≡ Partial solutionsLeaves ≡ Admissible solutions
S[0]
S[1]
S[2]
S[3] S[3]
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 9 / 37
Introduction
Decision trees
Decision tree ≡ Search spaceRoot ≡ Empty solutionInternal nodes ≡ Partial solutionsLeaves ≡ Admissible solutions
S[0]
S[1]
S[2]
S[3]
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 9 / 37
Introduction
Decision trees
Decision tree ≡ Search spaceRoot ≡ Empty solutionInternal nodes ≡ Partial solutionsLeaves ≡ Admissible solutions
S[0]
S[1]
S[3]
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 9 / 37
Introduction
Decision trees
Decision tree ≡ Search spaceRoot ≡ Empty solutionInternal nodes ≡ Partial solutionsLeaves ≡ Admissible solutions
S[0]
S[1]
S[3]
S[2]
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 9 / 37
Introduction
Decision trees
Decision tree ≡ Search spaceRoot ≡ Empty solutionInternal nodes ≡ Partial solutionsLeaves ≡ Admissible solutions
S[0]
S[1]
S[3]
S[2]
S[3]
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 9 / 37
Introduction
Pruning
"Branches" of the trees that do not bring to admissible solutionscan be "pruned"The evaluation is done in the partial solutions corresponding tointernal nodes
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 10 / 37
Introduction
Pruning
"Branches" of the trees that do not bring to admissible solutionscan be "pruned"The evaluation is done in the partial solutions corresponding tointernal nodes
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 10 / 37
Table of contents
1 Introduction2 Enumeration
SubsetsPermutationsGames
Enumeration
Enumeration
boolean enumeration(object[ ] S, int n, int i, . . . )Set C = choices(S, n, i, . . .) % Compute C based on S[0 : i− 1]foreach c ∈ C do
S[i] = cif isAdmissible(S, n, i) then
if processSolution(S, n, i, . . .) thenreturn True
if enumeration(S, n, i+ 1, . . .) thenreturn True
return False
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 11 / 37
Enumeration
Enumeration
S: list containing the partial solutions
i: current index
. . .: additional information
C: the set of possible candidates to extend the current solution
isAdmissible(): returns True if S[0 : i] is an admissible solution
processSolution(): returnsTrue to stop the execution at the first admissible solutionFalse to explore the entire tree
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 12 / 37
Enumeration Subsets
Example 1
List all subsets of {0, . . . , n− 1}
def subsets(S,n,i):C = [1, 0] if i<n else []for c in C:
S[i] = cif i==n-1:
if (processSolution(S)):return True
else:subsets(S,n,i+1)
return False
def processSolution(S):for i in range(len(S)):
if (S[i]==1):print(i, "", end="")
print("")return False
n = 4S = [0]*nsubsets(S,4,0)
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 13 / 37
Enumeration Subsets
Example 1 (Clean-up version)
List all possible subsets of {0, . . . , n− 1}
def subsets(S,n,i):for c in [1, 0]:
S[i] = cif i == n-1:
processSolution(S)else:
subsets(S,n,i+1)
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 14 / 37
Enumeration Subsets
Example 1
There is no pruning. All the possible space is explored.But this is required by the definition of the problem
Computational complexity O(n2n)
In which order sets are printed?
Is it possible to think to an iterative version, ad-hoc for thisproblem?(non-backtracking)
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 15 / 37
Enumeration Subsets
Example 1
subsets(int n)for j = 0 to 2n − 1 do
print ’{’for i = 0 to n− 1 do
if (j and 2i) 6= 0 thenprint i
print ’}’
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 16 / 37
Enumeration Subsets
Example 2
List all possible subsets of size k of a set {0, . . . , n− 1}
Iterative versionWhat is the cost?
Solution based on backtrackingCan we prune?
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 17 / 37
Enumeration Subsets
Example 2 - Attempt 1
def subsets(S, k, n,i):C = [1,0]for c in C:
S[i] = cif i == n-1:
if admissible(S, k):processSolution(S)
else:subsets(S, k, n, i+1):
def admissible(S, k):return sum(S) == k
def processSolution(S):for i in range(len(S)):
if (S[i]==1):print(i, "", end="")
print("")return False
n = 4k = 2S = [0]*nsubsets(S, 2, 4, 0)
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 18 / 37
Enumeration Subsets
Example 2 - Attempt 2
def subsets(S, k, n, i, count):C = [1, 0]for c in C:
S[i] = ccount = count + cif i == n-1:
if count == k:processSolution(S)
else:subsets(S,k, n, i+1, count)
count = count - c
def admissible(S, k):tot = sum(S)return tot == k
def processSolution(S):for i in range(len(S)):
if (S[i]==1):print(i, "", end="")
print("")return False
n = 4k = 2S = [0]*nsubsets(S, 2, 4, 0, 0)
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 19 / 37
Enumeration Subsets
Example 2 - Attempt 3
def subsets(S, k, n, i, count):if count < k and count + (n-i) >= k:
C = [1,0]else
C = []for c in C:
S[i] = ccount = count + cif count == k:
processSolution(S)else:
subsets(S, k, n, i+1, count)count = count - cS[i] = 0
def processSolution(S):for i in range(len(S)):
if (S[i]==1):print(i, "", end="")
print("")return False
n = 4k = 2S = [0]*nsubsets(S, 2, 4, 0, 0)
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 20 / 37
Enumeration Subsets
Example 2: advantages
© Alberto Montresor 20
Esempio 2: vantaggi
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 21 / 37
Enumeration Subsets
Example 2 - Summary
Cosa abbiamo imparato?
By "customizing" the generic algorithm, we can get a moreefficient solution
It is efficient forsmall values of k (close to 1)large values of k (close to n)
No great saving for n/2
It is difficult to obtain the same efficiency with an iterativealgorithm
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 22 / 37
Enumeration Permutations
Example 3
Print all permutations of a set A
def permutations(S, i, original):for c in original:
S[i] = coriginal.remove(c)if (len(original)==0):
print(S)else:
permutations(S, i+1, original)original.add(c)
original = { 1,2,3,4 }S = [0]*len(original)permutations(S, 0, original)Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 23 / 37
Enumeration Games
Eight queens puzzle
ProblemThe eight queens puzzle is the problem of placing eight chess queens onan 8× 8 chessboard so that no two queens threaten each other
History:Introduced by Max Bezzel(1848)Gauss found 72 of the 92solutions
Let’s start from the stupidestapproach, and let’s refine thesolution step by step
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 24 / 37
Enumeration Games
Eight queens puzzle
Idea: There are n2 possible positions where a queen may be located
S[0 : n2] binary array S[i] = True⇒ "queen in i"
isAdmissible() i == n2
choices(S, n, i) {True,False}
pruning if the new queen threaten one of theexisting queens, returns ∅
# Solutions for n = 8 264 ≈ 1.84 · 1019
Comments
Maybe we have a representation problem?Binary matrix, really sparse
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 25 / 37
Enumeration Games
Eight queens puzzle
Idea: For each queen, there are n2 possible positions
S[0 : n] coordinates in{0 . . . n2 − 1}
S[i] coordinate of queen i
isAdmissible() i == n
choices(S, n, i) {0 . . . n2 − 1}
pruning returns the subset of legal moves
# Solutions for n = 8 (n2)n = 648 = 248 ≈ 2.81 · 1014
Comments
Better, but the space is still huge . . .Problem: how to distinguish between a solution "1-7-. . . " and asolution "7-1-. . . " ?
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 26 / 37
Enumeration Games
Eight queens puzzle
Idea: do not place queens in positions that occur "before" the onesalready taken
S[0 : n] coordinates in{0 . . . n2 − 1}
S[i] coordinate of queen i
isAdmissible() i == n
choices(S, n, i) {0 . . . n2 − 1}
pruning returns legal moves, S[i] > S[i− 1]
# Solutions for n = 8 (n2)n/n! = 248/40320 ≈ 6.98 · 109
Comments
Very good, but there is still space for improvementAlberto Montresor (UniTN) SP - Backtracking 2018/12/15 27 / 37
Enumeration Games
Eight queens puzzle
Idea: each row of the board must contain exactly one queen
S[0 : n] coordinates in{0 . . . n− 1}
S[i] column of queen i, where row = i
isAdmissible() i == n
choices(S, n, i) {0 . . . n− 1}
pruning returns only legal columns
# Solutions for n = 8 nn = 88 ≈ 1.67 · 107
Comments
Almost there
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 28 / 37
Enumeration Games
Eight queens puzzle
Idea: every column must contain exactly one queen
S[0 : n] coordinates in{0 . . . n− 1}
permutations of {1 . . . n}
isAdmissible() i == n
choices(S, n, i) {0 . . . n− 1}
pruning removes diagonals
# Solutions for n = 8 n! = 8! = 40320
Comments
Solutions actually visited = 15720
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 29 / 37
Enumeration Games
Eight queens puzzle
def queens(S, i, columns):for c in columns:
S[i] = ccolumns.remove(c)if (diagonalsOK(S,i)):
if len(columns)==0:printBoard(S)
else:queens(S,i+1,columns)
columns.add(c)
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 30 / 37
Enumeration Games
Knight’s tour
ProblemA Knight’s tour is a sequence of moves of a knight on a chessboard suchthat the knight visits every square only once.
© Alberto Montresor 30
Giro di cavallo
✦ Problema: ✦ Si consideri ancora una scacchiera n·n; lo scopo è trovare un “giro di cavallo”,
ovvero un percorso di mosse valide del cavallo in modo che ogni casella venga visitata al più una volta
✦ Soluzione: ✦ Tramite backtrack
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 31 / 37
Enumeration Games
Knight’s tour
Solution
Board n× n whose cells contains:0 if the cell has not been visitedi if the cell has been visited at step i
# Solutions: 64! ≈ 1089
But: at each step, there are at most 8 possible cells, so themaximum number of visits is 864 ≈ 1057
In reality, much less thank to pruning
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 32 / 37
Enumeration Games
Knight’s tour
def knight(S, i, x, y):n = len(S)C = moves(S, x, y)if (i == n*n):
for k in range(n):print(S[k])
return Trueelse:
S[x][y] = ifor c in C:
if knight(S, i+1, x+dx[c], y+dy[c]):return True
S[x][y] = 0return False
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 33 / 37
Enumeration Games
Knight’s tour
dx = [-1, +1, +2, +2, +1, -1, -2, -2]dy = [-2, -2, -1, +1, +2, +2, +1, -1]
def moves(S, x, y):res = set()for i in range(8):
nx = x + dx[i]ny = y + dy[i]if (0 <= nx < 8) and (0 <= ny < 8) and S[nx][ny] ==0 :
res.add(i)return res
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 34 / 37
Enumeration Games
Sudoku - "Suuji wa dokushin ni kagiru"
2 5 9 7 6
2 4
1 5 3 9
8 9 4 5 2 6
1 2 4
2 5 6 7 3
8 3 2 1
9 7
3 7 8 9 2
2 5 3 8 9 1 4 7 6
8 9 7 2 6 4 3 1 5
6 4 1 5 7 3 9 2 8
7 8 9 4 3 5 2 6 1
1 3 6 7 2 9 8 5 4
4 2 5 6 1 8 7 3 9
9 6 8 3 5 2 1 4 7
5 1 2 9 4 7 6 8 3
3 7 4 1 8 6 5 9 2
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 35 / 37
Enumeration Games
Sudoku
boolean sudoku(int[ ][ ] S, int i)
int x = i mod 9int y = bi/9cSet C = Set()if i ≤ 80 then
if S[x, y] 6= 0 thenC.insert(S[x, y])
elsefor c = 1 to 9 do
if check(S, x, y, c) thenC.insert(c)
int old = S[x, y]foreach c ∈ C do
S[x, y] = cif i = 80 then
processSolution(S, n)return True
if sudoku(S, i+ 1) thenreturn True
S[x, y] = oldreturn False
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 36 / 37
Enumeration Games
Sudoku
boolean check(int[ ][ ] S, int x, int y, int c)for j = 0 to 8 do
if S[x, j] = c thenreturn False % Column check
if S[j, y] = c thenreturn False % Row check
int bx = bx/3cint by = by/3cfor ix = 0 to 2 do
for int iy = 0 to 2 do % Subtable checkif S[bx · 3 + ix, by · 3 + iy] = c then
return False
return True
Alberto Montresor (UniTN) SP - Backtracking 2018/12/15 37 / 37