class 3: constraints; search rina dechterfrossi/rina3.pdf · rina dechter class2 padova . road map...
TRANSCRIPT
-
Reasoning with Deterministic and Probabilistic graphical models
Class 3: Constraints; Search
Rina Dechter
class2 Padova
-
Road Map
n Graphical models n Constraint networks Model n Inference
n Variable elimination: n Tree-clustering n Constraint propagation
n Search n Probabilistic Networks
class2 Padova
-
Road Map: Search in Constraint Networks
n Improving search by bounded-inference in
branching ahead n Improving search by looking-back n The alternative AND/OR search space
class2 Padova
-
Road Map: Search in Constraint Networks
n Improving search by bounded-inference in
branching ahead n Improving search by looking-back n The alternative AND/OR search space
class2 Padova
-
Backtracking Search for a Solution
class2 Padova
-
class2 Padova
Backtracking Search for a Solution
-
class2 Padova
Backtracking Search for All Solutions
-
The Search Space Before Arc-Consistency
X!
..... ..... .....!
Y!
T!
X Y
T Z
<
=
<
∧
X=1 2 3
Y=2 3 3
T=2 3
Z
D={1,2,3}
class2 Padova
-
The Search Space After Arc-Consistency
X!
..... ..... .....!
Y!
T!
X=1
Y=2
T=2
Z
X Y <
=
<
∧ 1 3
2 3 T Z
class2 Padova
-
The Effect of Variable Ordering
z divides x, y and t
class2 Padova
-
The Effect of Consistency Level
n After arc-consistency z=5 and l=5 are removed
n After path-consistency n R’_zx n R’_zy n R’_zl n R’_xy n R’_xl n R’_yl
Tighter networks yield smaller search spaces
class2 Padova
-
Improving Backtracking O(exp(n))
n Before search: (reducing the search space) n Arc-consistency, path-consistency, i-consistency n Variable ordering (fixed)
n During search: n Look-ahead schemes:
n value ordering/pruning (choose a least restricting value), n variable ordering (Choose the most constraining
variable) n Look-back schemes:
n Backjumping n Constraint recording n Dependency-directed backtracking
class2 Padova
-
Looking-Ahead: Constraint Propagation in Search
n Apply some level of constraint propagation at each node, n Forward-checking (FC) n Arc-consistency (MAC)
n Then: n Value pruning or ordering : prune values that lead
to deadend n Variable ordering: choose a variable that leaves
least options open
class2 Padova
-
Forward-Checking for Value Ordering
)( 2ekO
)()(3
2
ekOekOFC overhead:
MAC overhead:
class2 Padova
-
Forward-Checking for Value Ordering
)( 2ekO
)()(3
2
ekOekOFW overhead:
MAC overhead:
class2 Padova
-
Forward-Checking, Variable Ordering
)( 2ekO
)()(3
2
ekOekOFW overhead:
MAC overhead:
class2 Padova
-
Forward-Checking, Variable Ordering
)( 2ekO
)()(3
2
ekOekOFW overhead:
MAC overhead:
After X1 = red choose X3 and not X2
class2 Padova
-
Forward-Checking, Variable Ordering
)( 2ekO
)()(3
2
ekOekOFW overhead:
MAC overhead:
After X1 = red choose X3 and not X2
class2 Padova
-
Forward-Checking, Variable Ordering
)( 2ekO
)()(3
2
ekOekOFW overhead:
MAC overhead:
After X1 = red choose X3 and not X2
class2 Padova
-
Arc-consistency for Value Ordering
)( 2ekO
)()(3
2
ekOekOFW overhead:
MAC overhead:
class2 Padova
-
Arc-Consistency for Value Ordering
)( 2ekO
)()(3
2
ekOekOFW overhead:
MAC overhead:
Arc-consistency prunes x1=red Prunes the whole tree Not searched
By MAC
class2 Padova
-
Constraint Programming
n Constraint solving embedded in programming languages
n Allows flexible modeling with algorithms n Logic programs + forward checking n Eclipse, ILog, OPL n Using only look-ahead schemes
class2 Padova
-
Branching-Ahead for SAT: DLL example: (~AVB)(~CVA)(AVBVD)(C)
Only enclosed area will be explored with unit-propagation
Backtracking look-ahead with Unit propagation= Generalized arc-consistency
(Davis, Logeman and Laveland, 1962)
class2 Padova
-
Road Map: Search in Constraint Networks
n Improving search by bounded-inference in
branching ahead n Improving search by looking-back n The alternative AND/OR search space
class2 Padova
-
Look-back: Backjumping / Learning
n Backjumping: n In deadends, go back to
the most recent culprit.
n Learning: n constraint-recording, no-
good recording. n good-recording
class2 Padova
-
Look-Back: Backjumping
n (X1=r,x2=b,x3=b,x4=b,x5=g,x6=r,x7={r,b}) n (r,b,b,b,g,r) conflict set of x7 n (r,-,b,b,g,-) c.s. of x7 n (r,-,b,-,-,-,-) minimal conflict-set n Leaf deadend: (r,b,b,b,g,r) n Every conflict-set is a no-good
class2 Padova
-
Jumps at dead-ends (Gascnnig-style 1977)
class2 Padova
-
Jumps at Dead-Ends (Gascnnig 1977)
class2 Padova
-
Complexity of Backjumping
Simple: always jump back to parent in pseudo tree Complexity for csp: exp(w*log n) From exp(n) to exp(w*logn) while linear space
Graph-based and conflict-based backjumpint
class2 Padova
-
Look-back: No-good Learning
n (x1=2,x2=2,x3=1,x4=2) is a dead-end
n Conflicts to record: n (x1=2,x2=2,x3=1,x4=2) 4-ary n (x3=1,x4=2) binary n (x4=2) unary
Learning means recording conflict sets used as constraints to prune future search space.
class2 Padova
-
No-good Learning Example
class2 Padova
-
Complexity of Nogood-Learning for consistency
n The complexity of learning along d is time and space exponential in w*(d):
n The number of dead-ends is bounded by n Number of constraint tests per dead-end are
Space complexity is Time complexity is No-good Learning reduces time to O(exp(w*)) but requires O(exp(w*)) space.
)()(
)*(2
)*(
dw
dw
kenOnkO
⋅
)( )*(dwnkO)(eO
class2 Padova
-
Moving to new queries
n Consistency and one solution. n Counting n enumerating
class5 huji
-
Bucket-elimination for counting
class2 Padova
-
:
:
:
:
:
E
RC
RRA
RRD
RRB
CE
AEAC
ADDE
BDAB
A
C B
D E
≠
}3,2,1{
}2,1{
}2,1{}2,1{
}2,1{
≠ ≠
≠ ≠
≠
Bucket-elimination for counting
∑ ∗= B DBRBARDAR ),(),(),("↓$,% "↓',$
’"′↓(' R_E
-
#CSP - Tree DFS Traversal
0 1
0
0 1
0 1
0
C
D
F
E
B
A 0
A
E
C
B
F
D
A B C RABC 0 0 0 1 0 0 1 1 0 1 0 0 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
A B E RABE 0 0 0 1 0 0 1 0 0 1 0 1 0 1 1 1 1 0 0 0 1 0 1 1 1 1 0 1 1 1 1 0
A E F RAEF 0 0 0 0 0 0 1 1 0 1 0 1 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
B C D RBCD 0 0 0 1 0 0 1 1 0 1 0 1 0 1 1 0 1 0 0 1 1 0 1 0 1 1 0 1 1 1 1 1
0 1
1 0
1
0 1
0 1
0 1
1 0
1 1
2
0
0 1
0 1
0 1
1 0
1 0 1
1
3
Value of node = number of solutions below it
5
0 1 0 1 0 1
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1
1 0 1
1
0 1 0 1 1 1 1 1 1 0 1 0 1 0 1 1
1 2 1 2
0
0 1
0 1
0 2
0
0
0
2
2
2
1
0 3 3
6
6
9
1 1 1
1 2
3
14
class2 Padova
-
#CSP - OR Search Tree
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1
C
D
F
E
B
A 0 1
A
E
C
B
F
D
A B C RABC 0 0 0 1 0 0 1 1 0 1 0 0 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
B C D RBCD 0 0 0 1 0 0 1 1 0 1 0 1 0 1 1 0 1 0 0 1 1 0 1 0 1 1 0 1 1 1 1 1
A B E RABE 0 0 0 1 0 0 1 0 0 1 0 1 0 1 1 1 1 0 0 0 1 0 1 1 1 1 0 1 1 1 1 0
A E F RAEF 0 0 0 0 0 0 1 1 0 1 0 1 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
14 solutions
1 2 3 4 6 7 9 10 11 12 13 14 5 8
class2 Padova
-
Summary: Search Principles
n Constraint propagation prunes search space n Constraint propagation yields good advise for
how to branch and where to go n Backjumping and no-good learning helps
prune search space and revise problem. n Good learning cache results but helps only
counting, enumeration… as we see shortly.
class2 Padova
-
Thank You
n “Constraint Processing”, Morgan Kaufmann, 2003
n Probabilistic networks: Transferring these ideas to Probabilistic network, helping unifying the principles.
n
Ijcai 2011 39
-
Road Map: Search in Graphical Models
n Improving search by bounded-inference in
branching ahead n Improving search by looking-back n The alternative AND/OR search space
class2 Padova
-
OR Search Space A
D
B C
E
F
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1
E
C
F
D
B
A 0 1
Ordering: A B E C D F
class2 Padova
-
AND/OR Search Space
A OR
0 AND 1
B OR B
0 AND 1 0 1
E OR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
A
D
B C
E
F
A
D
B
C E
F
Primal graph DFS tree
A
D
B C
E
F
A
D
B C
E
F
class2 Padova
-
AND/OR vs. OR
E 0 1 0 1 0 1 0 1
0 C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 B 1 0 1
A 0 1
A OR
0 AND 1
B OR B
0 AND 1 0 1
E OR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
E 0 1 0 1 0 1 0 1
0 C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 B 1 0 1
A 0 1
E 0 1 0 1 0 1 0 1
0 C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 B 1 0 1
A 0 1
AND/OR
OR
A
D
B C
E
F A
D
B
C E
F
AND/OR size: exp(4), OR size exp(6)
class2 Padova
-
AND/OR vs. OR
E 0 1 0 1 0 1 0 1
0 C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 B 1 0 1
A 0 1
A OR
0 AND 1
B OR B
0 AND 1 0 1
E OR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
E 0 1 0 1 0 1 0 1
0 C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 B 1 0 1
A 0 1
E 0 1 0 1 0 1 0 1
0 C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 B 1 0 1
A 0 1
AND/OR
OR
A
D
B C
E
F A
D
B
C E
F
No-goods (A=1,B=1) (B=0,C=0)
class2 Padova
-
AND/OR vs. OR
F
AND/OR
A
D
B C
E
F A
D
B
C E
F
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 B 1 0
A 0 1
A OR
0 AND 1
B OR B
0 AND 1 0
E OR C E C E C
OR D F D F D F D F AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 1 0 1 0 1 0 1 1 0 1
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 B 1 0
A 0 1
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 B 1 0
A 0 1
OR
(A=1,B=1) (B=0,C=0)
class2 Padova
-
AND/OR vs. OR
F
AND/OR
A
D
B C
E
F A
D
B
C E
F
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 B 1 0
A 0 1
A OR
0 AND 1
B OR B
0 AND 1 0
E OR C E C E C
OR D F D F D F D F AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 1 0 1 0 1 0 1 1 0 1
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 B 1 0
A 0 1
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 B 1 0
A 0 1
OR
(A=1,B=1) (B=0,C=0)
Space: linear Time: O(exp(m)) O(w* log n)
Linear space, Time: O(exp(n))
class2 Padova
-
AND/OR vs. OR Spaces
width depth OR space AND/OR space
Time (sec.) Nodes Time (sec.) AND nodes OR nodes
5 10 3.15 2,097,150 0.03 10,494 5,247
4 9 3.13 2,097,150 0.01 5,102 2,551
5 10 3.12 2,097,150 0.03 8,926 4,463
4 10 3.12 2,097,150 0.02 7,806 3,903
5 13 3.11 2,097,150 0.10 36,510 18,255
Random graphs with 20 nodes, 20 edges and 2 values per node
class2 Padova
-
#CSP – AND/OR Search Tree
A
E
C
B
F
D
A
D
B
E C
F
A B C RABC 0 0 0 1 0 0 1 1 0 1 0 0 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
A B E RABE 0 0 0 1 0 0 1 0 0 1 0 1 0 1 1 1 1 0 0 0 1 0 1 1 1 1 0 1 1 1 1 0
A E F RAEF 0 0 0 0 0 0 1 1 0 1 0 1 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
B C D RBCD 0 0 0 1 0 0 1 1 0 1 0 1 0 1 1 0 1 0 0 1 1 0 1 0 1 1 0 1 1 1 1 1
A OR
0 AND
B OR
0 AND
OR E
OR F F
AND 0 1 0 1
AND 0 1
C
D D
0 1 0 1
0 1
1
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
1
B
0
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
1
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
class2 Padova
-
#CSP – AND/OR Tree DFS
A
E
C
B
F
D
A
D
B
E C
F
A B C RABC 0 0 0 1 0 0 1 1 0 1 0 0 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
A B E RABE 0 0 0 1 0 0 1 0 0 1 0 1 0 1 1 1 1 0 0 0 1 0 1 1 1 1 0 1 1 1 1 0
A E F RAEF 0 0 0 0 0 0 1 1 0 1 0 1 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
B C D RBCD 0 0 0 1 0 0 1 1 0 1 0 1 0 1 1 0 1 0 0 1 1 0 1 0 1 1 0 1 1 1 1 1
A OR
0 AND
B OR
0 AND
OR E
OR F
AND 0 1
AND 0 1
C
D D
0 1 0 1
0 1
1 1 1 0 0 1
2 1 1
2 1 1 0
3 1
3
9
9
1
E
F F
0 1 0 1
0 1
C
D
0 1
0 1
1
B
0
E
F
0 1
0 1
C
D D
0 1 0 1
0 1
1
E
F
0 1
0 1
C
D
0 1
0 1
1 1 0 1 1 1
2 1 2
2 1 2 0
2 3
6
1 1 1 0 1 0 1 0 1 1
2 1 1 1 2
3 1
2 1 1 0 0 1 0 2
1 2
3 2
5
5
14
OR node: Marginalization operator (summation)
AND node: Combination operator (product)
class2 Padova
-
Pseudo-Trees (Freuder 85, Bayardo 95, Bodlaender and Gilbert, 91)
(a) Graph
4 6 1
3 2 7 5
(b) DFS tree depth=3
(c) pseudo- tree depth=2
(d) Chain depth=6
4 6
1
3
2 7
5 2 7
1
4
3 5
6
4
6
1
3
2
7
5
m
-
a
1
2
b c
3
a c
4
OR
AND
OR
AND
OR
AND
AND
OR
c b
4
b
3
a b
4
c b
4
c
7
b c
5
a c
6
c b
6
b
5
a b
6
c b
6
c
b
2
a c
3
b c
4
c a
4
a
3
a b
4
c
4
c a
7
a c
5
b c
6
c a
6
a
5
a b
6
c
6
c a
c
2
a b
3
b c
4
a
4
b a
3
a c
4
b
4
b a
7
a b
5
b c
6
a
6
b a
5
a c
6
b
6
b a
a
1
3
a
4
b c
OR
AND
OR
AND
OR
AND
2
b c
b
4
c
2
c
c
4
b
2
b
5
a
7
b c
6
b c
b
7
c
6
c
c
7
b
6
b
b
3
b
4
a c
2
a c
a
4
c
2
c
c
4
a
2
a
c
3
c
4
a b
2
a b
a
4
b
2
b
b
4
a
2
a
5
b
7
a c
6
a c
a
7
c
6
c
c
7
a
6
a
5
c
7
a b
6
a b
a
7
b
6
b
b
7
a
6
a
class2 Padova
-
AND/OR search tree for graphical models
n The AND/OR search tree of R relative to a tree, T, has: n Alternating levels of: OR nodes (variables) and AND nodes (values)
n Successor function: n The successors of OR nodes X are all its consistent values along its path n The successors of AND are all X child variables in T
n A solution is a consistent subtree n Task: compute the value of the root node
0
A
B
0
E
OR
AND
OR
AND
OR
AND
C
0
OR
AND
D
0 1
F
0 1
1
D
0 1
F
0 1
0 1
1
E C
0
D
0 1
F
0 1
1
D
0 1
F
0 1
0 1
1
B
0
E C
0
D
0 1
F
0 1
1
D
0 1
F
0 1
0 1
1
E C
0
D
0 1
F
0 1
1
D
0 1
F
0 1
0 1
A
D
B C
E
F
A
D
B
C E
F
class2 Padova
-
From Search A/O Trees to Search A/O Graphs
n Any two nodes that root identical subtrees/subgraphs can be merged
n Minimal AND/OR search graph: closure under merge of the AND/OR search tree
n Inconsistent sub-trees can be pruned too. n Some portions can be collapsed or reduced.
class2 Padova
-
From search trees to search graphs
n Any two nodes that root identical subtrees (subgraphs) can be merged
class2 Padova
-
From search trees to search graphs
n Any two nodes that root identical subtrees (subgraphs) can be merged
class2 Padova
-
AND/OR Tree
A
D
B C
E
F
A
D
B
C E
F
G H
J
K
G
H
J
K A OR
0 AND 1
B OR B
0 AND 1 0 1
E OR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
OR
OR
AND
AND
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
class2 Padova
-
An AND/OR Graph: Caching Goods
A
D
B C
E
F
A
D
B
C E
F
G H
J
K
G
H
J
K A OR
0 AND 1
B OR B
0 AND 1 0 1
E OR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
OR
OR
AND
AND
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
class2 Padova
-
Context-based Caching
n Caching is possible when context is the same
n context = parent-separator set in induced pseudo-graph = current variable + parents connected to subtree below
A
D
B C
E
F
A
D
B
C E
F
G H
J
K
G
H
J
K
context(B) = {A, B}
context(c) = {A,B,C}
context(D) = {D}
context(F) = {F}
class2 Padova
-
Complexity of AND/OR Graph
n Theorem: Traversing the AND/OR search graph is time and space exponential in the induced width/tree-width.
n If applied to the OR graph complexity is time and space exponential in the path-width.
class2 Padova
-
#CSP – AND/OR Tree DFS
A
E
C
B
F
D
A
D
B
E C
F
A B C RABC 0 0 0 1 0 0 1 1 0 1 0 0 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
A B E RABE 0 0 0 1 0 0 1 0 0 1 0 1 0 1 1 1 1 0 0 0 1 0 1 1 1 1 0 1 1 1 1 0
A E F RAEF 0 0 0 0 0 0 1 1 0 1 0 1 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
B C D RBCD 0 0 0 1 0 0 1 1 0 1 0 1 0 1 1 0 1 0 0 1 1 0 1 0 1 1 0 1 1 1 1 1
A OR
0 AND
B OR
0 AND
OR E
OR F
AND 0 1
AND 0 1
C
D D
0 1 0 1
0 1
1 1 1 0 0 1
2 1 1
2 1 1 0
3 1
3
9
9
1
E
F F
0 1 0 1
0 1
C
D
0 1
0 1
1
B
0
E
F
0 1
0 1
C
D D
0 1 0 1
0 1
1
E
F
0 1
0 1
C
D
0 1
0 1
1 1 0 1 1 1
2 1 2
2 1 2 0
2 3
6
1 1 1 0 1 0 1 0 1 1
2 1 1 1 2
3 1
2 1 1 0 0 1 0 2
1 2
3 2
5
5
14
class2 Padova
-
#CSP – AND/OR Search Graph (Caching Goods)
A
E
C
B
F
D
A
D
B
E C
F
A B C RABC 0 0 0 1 0 0 1 1 0 1 0 0 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
A B E RABE 0 0 0 1 0 0 1 0 0 1 0 1 0 1 1 1 1 0 0 0 1 0 1 1 1 1 0 1 1 1 1 0
A E F RAEF 0 0 0 0 0 0 1 1 0 1 0 1 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
B C D RBCD 0 0 0 1 0 0 1 1 0 1 0 1 0 1 1 0 1 0 0 1 1 0 1 0 1 1 0 1 1 1 1 1
A OR
0 AND
B OR
0 AND
OR E
OR F F
AND 0 1
AND 0 1
C
D D
0 1
0 1
1
E C
D D
0 1
1
B
0
E
F F
0 1
C
1
E C
class2 Padova
-
#CSP – AND/OR Search Graph (Caching Goods)
A
E
C
B
F
D
A
D
B
E C
F
A B C RABC 0 0 0 1 0 0 1 1 0 1 0 0 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
A B E RABE 0 0 0 1 0 0 1 0 0 1 0 1 0 1 1 1 1 0 0 0 1 0 1 1 1 1 0 1 1 1 1 0
A E F RAEF 0 0 0 0 0 0 1 1 0 1 0 1 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
B C D RBCD 0 0 0 1 0 0 1 1 0 1 0 1 0 1 1 0 1 0 0 1 1 0 1 0 1 1 0 1 1 1 1 1
A OR
0 AND
B OR
0 AND
OR E
OR F F
AND 0 1
AND 0 1
C
D D
0 1
0 1
1
E C
D D
0 1
1
B
0
E
F F
0 1
C
1
E C
Time and Space O(exp(w*))
class2 Padova
-
All Four Search Spaces
Full OR search tree
126 nodes
Full AND/OR search tree
54 AND nodes
Context minimal OR search graph
28 nodes
Context minimal AND/OR search graph
18 AND nodes
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1
C D
F E
B A 0 1
A OR 0 AND B OR
0 AND
OR E
OR F F AND 0 1 0 1
AND 0 1
C
D D
0 1 0 1
0 1
1
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
1
B
0
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
1
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1
0 1
0 1 0 1
0 1 0 1
C D
F E
B A 0 1
A OR 0 AND B OR
0 AND OR E
OR F F AND 0 1
AND 0 1 C
D D 0 1
0 1
1
E C
D D
0 1
1
B
0
E
F F
0 1
C
1
E C
A
E
C
B
F
D
class2 Padova
-
AND/OR vs. OR DFS Algorithms
n AND/OR tree n Space: O(n) n Time: O(n km)
O(n kw* log n) (Freuder85; Bayardo95; Darwiche01)
n AND/OR graph n Space: O(n kw*) n Time: O(n kw*)
k = domain size m = tree depth n = # of variables w*= induced width pw*= path width l OR tree
• Space: O(n) • Time: O(kn)
l OR graph • Space: O(n kpw*) • Time: O(n kpw*)
class2 Padova
-
Road Map
n Graphical models n Constraint networks Model n Inference
n Variable elimination: n Tree-clustering n Constraint propagation
n Search n Probabilistic Networks
class2 Padova