lecture 1: constraint networks overview
DESCRIPTION
TRANSCRIPT
![Page 1: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/1.jpg)
Constraint NetworksOverview
![Page 2: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/2.jpg)
Suggested reading
Russell and Norvig. Artificial Intelligence: Modern Approach. Chapter 5.
![Page 3: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/3.jpg)
3
Good source of advanced information
Rina Dechter,
Constraint Processing,Morgan Kaufmann
![Page 4: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/4.jpg)
4
Outline
CSP: Definition, and simple modeling examples Representing constraints Basic search strategy Improving search:
– Consistency algorithms– Look-ahead methods– Look-back methods
![Page 5: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/5.jpg)
5
Outline
CSP: Definition, and simple modeling examples Representing constraints Basic search strategy Improving search:
– Consistency algorithms– Look-ahead methods– Look-back methods
![Page 6: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/6.jpg)
A Bred greenred blackgreen redgreen blackblack greenblack red
Constraint Satisfaction
Example: map coloring Variables - countries (A,B,C,etc.) Values - colors (e.g., red, green, black) Constraints:
A≠B, A≠D, D≠E , etc .
C
A
B
D
E
F
G
![Page 7: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/7.jpg)
7
Constraint Network; Definition
A constraint network is: R=(X,D,C)X variables
D domain
C constraints
R expresses allowed tuples over scopes
A solution is an assignment to all variables that satisfies all constraints (join of all relations).
Tasks: consistency?, one or all solutions, counting, optimization
X={X1 , .. . ,X n}
D={D1 ,. . . , Dn}, Di={v1 , . .. vk }
C={C1 ,. .. C t },,, C i= S i , Ri
![Page 8: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/8.jpg)
Example The 4-queen problem
Q
Q QQ
Place 4 Queens on a chess board of 4x4 such that no two queens reside in the same row, column or diagonal.
Standard CSP formulation of the problem:• Variables: each row is a variable.
X1
X 4
X 3
X 2
1 2 3 4
• Domains: Di={1,2,3,4}.
• Constraints: There are = 6 constraints involved:42( )
R12={1,31,4 2,4 3,1 4,1 4,2 }R13={1,2 1,4 2,12,3 3,2 3,4 4,1 4,3}R14={1,2 1,3 2,12,3 2,4 3,1 3,2 3,4 4,2 4,3 }R23={1,31,4 2,4 3,1 4,1 4,2 }R24={1,2 1,4 2,1 2,3 3,2 3,4 4,1 4,3 }R34={1,3 1,4 2,4 3,1 4,1 4,2 }
• Constraint Graph :X1
X 2 X 4
X 3
![Page 9: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/9.jpg)
![Page 10: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/10.jpg)
![Page 11: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/11.jpg)
16
Outline
CSP: Definition, and simple modeling examples Representing constraints Basic search strategy Improving search:
– Consistency algorithms– Look-ahead methods– Look-back methods
![Page 12: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/12.jpg)
Spring 2009 17
Constraint’s representations
Relation: allowed tuples
Algebraic expression:
Propositional formula:
Constraint graph
X Y 2≤ 10 , X ≠Y
a∨b ¬c
X Y Z1 3 22 1 3
![Page 13: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/13.jpg)
Spring 2009 19
Figure 1.8: Example of set operations intersection, union, and difference applied to relations.
![Page 14: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/14.jpg)
Spring 2009 21
Constraint Graphs: Primal, Dual and Hypergraphs
A (primal) constraint graph: a node per variable arcs connect constrained variables. A dual constraint graph: a node per constraint’s scope, an
arc connect nodes sharing variables =hypergraph
![Page 15: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/15.jpg)
![Page 16: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/16.jpg)
![Page 17: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/17.jpg)
![Page 18: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/18.jpg)
26
Outline
CSP: Definition, and simple modeling examples Representing constraints Basic search strategy Improving search:
– Consistency algorithms– Look-ahead methods– Look-back methods
![Page 19: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/19.jpg)
![Page 20: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/20.jpg)
28
Backtracking search
![Page 21: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/21.jpg)
Search space
![Page 22: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/22.jpg)
Backtracking
![Page 23: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/23.jpg)
![Page 24: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/24.jpg)
![Page 25: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/25.jpg)
![Page 26: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/26.jpg)
![Page 27: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/27.jpg)
The search space
A tree of all partial solutions A partial solution: (a1,…,aj) satisfying all
relevant constraints The size of the underlying search space depends
on: Variable ordering Level of consistency possessed by the problem
![Page 28: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/28.jpg)
Search space and the effect of ordering
![Page 29: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/29.jpg)
Backtracking
Complexity of extending a partial solution: Complexity of consistent O(e log
t), t bounds tuples, e constraints Complexity of selectvalue O(e k
log t)
![Page 30: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/30.jpg)
A coloring problem example
![Page 31: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/31.jpg)
Backtracking search for a solution
![Page 32: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/32.jpg)
Backtracking Search for a Solution
![Page 33: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/33.jpg)
Backtracking Search for All Solutions
![Page 34: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/34.jpg)
46
Outline
CSP: Definition, and simple modeling examples Representing constraints Basic search strategy Improving search:
– Consistency algorithms– Look-ahead methods– Look-back methods
![Page 35: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/35.jpg)
• Before search: (reducing the search space)– Arc-consistency, path-consistency, i-consistency– Variable ordering (fixed)
• During search:– Look-ahead schemes:
• Value ordering/pruning (choose a least restricting value), • Variable ordering (Choose the most constraining variable)
– Look-back schemes:• Backjumping• Constraint recording• Dependency-directed backtracking
Improving Backtracking O(exp(n))
![Page 36: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/36.jpg)
48
Consistency methods
Constraint propagation – inferring new constraints
Can get such an explicit network that the search will find the solution without dead-ends.
Approximation of inference: Arc, path and i-consistency
Methods that transform the original network into a tighter and tighter representations
![Page 37: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/37.jpg)
49
Arc-consistency
32,1,
32,1, 32,1,
1 X, Y, Z, T 3X YY = ZT ZX T
X Y
T Z
32,1,
=
- infer constraints based on pairs of variables
Insures that every legal value in the domain of a single variable hasa legal match In the domain of any other selected variable
![Page 38: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/38.jpg)
50
1 X, Y, Z, T 3X YY = ZT ZX T
X Y
T Z
=
1 3
2 3
Arc-consistency
![Page 39: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/39.jpg)
Arc-consistency algorithm
domain of x domain of y
Arc is arc-consistent if for any value of there exist a matching value of
Algorithm Revise makes an arc consistent
Begin
1. For each a in Di if there is no value b in Dj that matches a then delete a from the Dj.
End.
Revise is , k is the number of value in each domain.
)( ji ,XX iX iX
)( ji ,XX
)O(k 2
![Page 40: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/40.jpg)
Algorithm AC-3
• Begin– 1. Q <--- put all arcs in the queue in both directions– 2. While Q is not empty do,– 3. Select and delete an arc from the queue Q
• 4. Revise• 5. If Revise cause a change then add to the queue all arcs
that touch Xi (namely (Xi,Xm) and (Xl,Xi)).– 6. end-while
• End• Complexity:
– Processing an arc requires O(k^2) steps– The number of times each arc can be processed is 2·k– Total complexity is
)( ji ,XX
)( ji ,XX
)O(ek 3
![Page 41: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/41.jpg)
53
Sudoku –Constraint Satisfaction
Each row, column and major block must be alldifferent
“Well posed” if it has unique solution: 27 constraints
2 34 62
•Variables: empty slots
•Domains = {1,2,3,4,5,6,7,8,9}
•Constraints: 27 all-different
•Constraint •Propagation
•Inference
![Page 42: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/42.jpg)
55
Path-consistency
![Page 43: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/43.jpg)
57
I-consistency
![Page 44: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/44.jpg)
The Effect of Consistency Level
• After arc-consistency z=5 and l=5 are removed
• After path-consistency– R’_zx– R’_zy– R’_zl– R’_xy– R’_xl– R’_yl
Tighter networks yield smaller search spaces
![Page 45: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/45.jpg)
59
Outline
CSP: Definition, and simple modeling examples Representing constraints Basic search strategy Improving search:
– Consistency algorithms– Look-ahead methods– Look-back methods
![Page 46: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/46.jpg)
![Page 47: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/47.jpg)
![Page 48: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/48.jpg)
![Page 49: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/49.jpg)
![Page 50: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/50.jpg)
![Page 51: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/51.jpg)
![Page 52: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/52.jpg)
![Page 53: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/53.jpg)
![Page 54: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/54.jpg)
![Page 55: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/55.jpg)
![Page 56: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/56.jpg)
![Page 57: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/57.jpg)
![Page 58: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/58.jpg)
![Page 59: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/59.jpg)
79
Outline
CSP: Definition, and simple modeling examples Representing constraints Basic search strategy Improving search:
– Consistency algorithms– Look-ahead methods– Look-back methods
![Page 60: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/60.jpg)
Look-back: Backjumping / Learning
• Backjumping: – In deadends, go back to the most recent culprit.
• Learning: – constraint-recording, no-good recording.– good-recording
![Page 61: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/61.jpg)
Backjumping
• (X1=r,x2=b,x3=b,x4=b,x5=g,x6=r,x7={r,b})• (r,b,b,b,g,r) conflict set of x7• (r,-,b,b,g,-) c.s. of x7• (r,-,b,-,-,-,-) minimal conflict-set• Leaf deadend: (r,b,b,b,g,r)• Every conflict-set is a no-good
![Page 62: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/62.jpg)
![Page 63: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/63.jpg)
![Page 64: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/64.jpg)
![Page 65: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/65.jpg)
![Page 66: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/66.jpg)
![Page 67: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/67.jpg)
The cycle-cutset method
• An instantiation can be viewed as blocking cycles in the graph
• Given an instantiation to a set of variables that cut all cycles (a cycle-cutset) the rest of the problem can be solved in linear time by a tree algorithm.
• Complexity (n number of variables, k the domain size and C the cycle-cutset size):
)( 2knkO C
![Page 68: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/68.jpg)
Tree Decomposition
![Page 69: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/69.jpg)
![Page 70: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/70.jpg)
![Page 71: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/71.jpg)
![Page 72: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/72.jpg)
GSAT – local search for SAT(Selman, Levesque and Mitchell, 1992)
1. For i=1 to MaxTries2. Select a random assignment A3. For j=1 to MaxFlips4. if A satisfies all constraint, return A5. else flip a variable to maximize the score 6. (number of satisfied constraints; if no variable 7. assignment increases the score, flip at random)8. end9. end
Greatly improves hill-climbing by adding restarts and sideway moves
![Page 73: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/73.jpg)
WalkSAT (Selman, Kautz and Cohen, 1994)
With probability p random walk – flip a variable in some unsatisfied constraintWith probability 1-p perform a hill-climbing step
Adds random walk to GSAT:
Randomized hill-climbing often solves large and hard satisfiable problems
![Page 74: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/74.jpg)
More Stochastic Search: Simulated Annealing, reweighting• Simulated annealing:
– A method for overcoming local minimas– Allows bad moves with some probability:
• With some probability related to a temperature parameter T the next move is picked randomly.
– Theoretically, with a slow enough cooling schedule, this algorithm will find the optimal solution. But so will searching randomly.
• Breakout method (Morris, 1990): adjust the weights of the violated constraints
![Page 75: Lecture 1: Constraint networks overview](https://reader036.vdocument.in/reader036/viewer/2022082700/5490b25cb4795915548b4582/html5/thumbnails/75.jpg)