exact reasoning: conditioning search and hybrids
DESCRIPTION
Exact Reasoning: conditioning search and hybrids. COMPSCI 276 Fall 2007. Probabilistic Inference Tasks. Belief updating: Finding most probable explanation (MPE) Finding maximum a-posteriory hypothesis Finding maximum-expected-utility (MEU) decision. Belief Updating. Smoking. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/1.jpg)
Exact Reasoning:conditioning search and hybrids
COMPSCI 276Fall 2007
![Page 2: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/2.jpg)
CS 276 2
Probabilistic Inference Tasks
X/A
a
*k
*1 e),xP(maxarg)a,...,(a
evidence)|xP(X)BEL(X iii
Belief updating:
Finding most probable explanation (MPE)
Finding maximum a-posteriory hypothesis
Finding maximum-expected-utility (MEU) decision
e),xP(maxarg*xx
)xU(e),xP(maxarg)d,...,(d X/D
d
*k
*1
variableshypothesis: XA
function utilityx variablesdecision
: )( :
UXD
![Page 3: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/3.jpg)
CS 276 3
Belief Updating
lung Cancer
Smoking
X-ray
Bronchitis
Dyspnoea
P (lung cancer=yes | smoking=no, dyspnoea=yes ) = ?
![Page 4: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/4.jpg)
CS 276 4
Conditioning generates the probability tree
0
),|(),|()|()|()()0,(ebcb
cbePbadPacPabPaPeaP
Complexity of conditioning: exponential time, linear space
![Page 5: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/5.jpg)
CS 276 5
Conditioning+Elimination
0
),|(),|()|()|()()0,(edcb
cbePbadPacPabPaPeaP
Idea: conditioning until of a (sub)problem gets small*w
![Page 6: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/6.jpg)
CS 276 6
Loop-cutset decomposition You condition until you get a
polytree
B
CB
F
A
BCB
F
A=0 A=0 A=0
BCB
F
A=1 A=1 A=1
P(B|F=0) = P(B, A=0|F=0)+P(B,A=1|F=0)
A=0 A=1
Loop-cutset method is time exp in loop-cutset sizeAnd linear space
![Page 7: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/7.jpg)
CS 276 9
OR search spaceA
D
B C
E
F
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1
E
C
F
D
B
A 0 1
Ordering: A B E C D F
![Page 8: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/8.jpg)
CS 276 10
AND/OR search space
AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
A
D
B C
E
F
A
D
B
CE
F
Primal graph DFS tree
A
D
B C
E
F
A
D
B C
E
F
![Page 9: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/9.jpg)
CS 276 12
OR vs AND/OR AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
AND/OR
OR
A
D
B C
E
F
A
D
B
CE
F
1
1
1
0
1
0
![Page 10: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/10.jpg)
CS 276 13
AND/OR vs. OR
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
AND/OR
OR
A
D
B C
E
F
A
D
B
CE
F
AND/OR size: exp(4), OR size exp(6)
![Page 11: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/11.jpg)
CS 276 14
AND/OR vs. OR
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
AND/OR
OR
A
D
B C
E
F
A
D
B
CE
F
No-goods(A=1,B=1)(B=0,C=0)
![Page 12: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/12.jpg)
CS 276 15
AND/OR vs. OR
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0
A 0 1
AOR
0AND 1
BOR B
0AND 1 0
EOR C E C E C
OR D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 10 1 0 10 1 10 1
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0
A 0 1
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0
A 0 1
AND/OR
OR
A
D
B C
E
F
A
D
B
CE
F
(A=1,B=1)(B=0,C=0)
![Page 13: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/13.jpg)
CS 276 16
OR space vs. AND/OR space
width heightOR space AND/OR space
time(sec.) nodes backtracks time(sec.) AND nodes OR nodes
5 10 3.154 2,097,150 1,048,575 0.03 10,494 5,247
4 9 3.135 2,097,150 1,048,575 0.01 5,102 2,551
5 10 3.124 2,097,150 1,048,575 0.03 8,926 4,463
4 10 3.125 2,097,150 1,048,575 0.02 7,806 3,903
5 13 3.104 2,097,150 1,048,575 0.1 36,510 18,255
5 10 3.125 2,097,150 1,048,575 0.02 8,254 4,127
6 9 3.124 2,097,150 1,048,575 0.02 6,318 3,159
5 10 3.125 2,097,150 1,048,575 0.02 7,134 3,567
5 13 3.114 2,097,150 1,048,575 0.121 37,374 18,687
5 10 3.114 2,097,150 1,048,575 0.02 7,326 3,663
![Page 14: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/14.jpg)
CS 276 17
AND/OR search tree for graphical models
The AND/OR search tree of R relative to a spanning-tree, T, has:
Alternating levels of: OR nodes (variables) and AND nodes (values)
Successor function: The successors of OR nodes X are all its consistent values along
its path The successors of AND <X,v> are all X child variables in T
A solution is a consistent subtree Task: compute the value of the root node
0
A
B
0
E
OR
AND
OR
AND
OR
AND
C
0
OR
AND
D
0 1
F
0 1
1
D
0 1
F
0 1
0 1
1
E C
0
D
0 1
F
0 1
1
D
0 1
F
0 1
0 1
1
B
0
E C
0
D
0 1
F
0 1
1
D
0 1
F
0 1
0 1
1
E C
0
D
0 1
F
0 1
1
D
0 1
F
0 1
0 1
A
D
B C
E
F
A
D
B
CE
F
![Page 15: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/15.jpg)
CS 276 18
From DFS trees to pseudo-trees (Freuder 85, Bayardo 95)
(a) Graph
4 61
3 2 7 5
(b) DFS treedepth=3
(c) pseudo- treedepth=2
(d) Chaindepth=6
4 6
1
3
2 7
5 2 7
1
4
3 5
6
4
6
1
3
2
7
5
![Page 16: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/16.jpg)
CS 276 19
From DFS trees to Pseudo-trees
a
1
2
a
3
a
4
OR
AND
OR
AND
OR
AND
AND
OR
ba
b c
c
b
4
ba c
c
4
ba c
b
3
a
4
ba c
b
4
ba c
c
4
ba c
c
3
a
4
ba c
b
4
ba c
c
4
ba c
7
a
5
a
6
ba c
b
6
ba c
c
6
ba c
b
5
a
6
ba c
b
6
ba c
c
6
ba c
c
5
a
6
ba c
b
6
ba c
c
6
ba c
a
1
3
a
4
b c
OR
AND
OR
AND
OR
AND
b c
a
2
b ca
b
4
b ca
2
b ca
c
4
b ca
2
b ca
5
a
7
b ca
6
b ca
b
7
b ca
6
b ca
c
7
b ca
6
b ca
DFS tree
depth = 3
pseudo- tree
depth = 2
![Page 17: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/17.jpg)
CS 276 20
Finding min-depth backbone trees
Finding min depth DFS, or pseudo tree is NP-complete, but:
Given a tree-decomposition whose tree-width is w*, there exists a pseudo tree T of G whose depth, satisfies (Bayardo and Mirankar, 1996, bodlaender and Gilbert, 91):
m <= w* log n,
![Page 18: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/18.jpg)
CS 276 21
Generating pseudo-trees from Bucket trees
FA
C
B D
E
E
A
C
B
D
F (AF) (EF)
(A)
(AB)
(AC) (BC)
(AE)
(BD) (DE)
Bucket-tree based on dd: A B C E D F
E
A
C
B
D
F
Induced graph
E
A
C
B
D F
Bucket-tree used as pseudo-tree
AND
AND
AND
AND
0 1
BOR B
0 1 0 1
COR E C E C E C E
OR D F D F D F D F D F D F D F D F
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 10 1 0 10 1 0 10 1 0 10 1
AOR
AND/OR search tree
Bucket-tree
ABE
A
ABC
AB
BDE AEF
bucket-A
bucket-E
bucket-B
bucket-C
bucket-D bucket-F
(AE) (BE)
![Page 19: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/19.jpg)
CS 276 23
AND/OR Search-tree properties (k = domain size, m = pseudo-tree depth. n = number of
variables)
Theorem: Any AND/OR search tree based on a pseudo-tree is sound and complete (expresses all and only solutions)
Theorem: Size of AND/OR search tree is O(n km)Size of OR search tree is O(kn)
Theorem: Size of AND/OR search tree can be bounded by O(exp(w* log n))
Related to: (Freuder 85; Dechter 90, Bayardo et. al. 96, Darwiche 1999, Bacchus 2003)
When the pseudo-tree is a chain we get an OR space
![Page 20: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/20.jpg)
CS 276 24
Tasks and value of nodes V( n) is the value of the tree T(n) for the task:
Counting: v(n) is number of solutions in T(n) Consistency: v(n) is 0 if T(n) inconsistent, 1 othewise. Optimization: v(n) is the optimal solution in T(n) Belief updating: v(n), probability of evidence in T(n). Partition function: v(n) is the total probability in
T(n).
Goal: compute the value of the root node recursively using dfs search of the AND/OR tree.
Theorem: Complexity of AO dfs search is Space: O(n) Time: O(n km) Time: O(exp(w* log n))
![Page 21: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/21.jpg)
CS 276 26
A
C
B
DE
A
D
B C
E
0
A
B
0
E D
0 1
C
0 1
C
0 1 0 1
1
E D
0 1
C
0 1
C
0 1 0 1
1
B
0
E D
0 1
C
0 1
C
0 1 0 1
1
E D
0 1
C
0 1
C
0 1 0 1
Belief-updating on example
![Page 22: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/22.jpg)
CS 276 27
0
A
B
0
E D
0 1
C
0 1
C
0 1 0 1
1
E D
0 1
C
0 1
C
0 1 0 1
1
B
0
E D
0 1
C
0 1
C
0 1 0 1
1
E D
0 1
C
0 1
C
0 1 0 1
PA(0) PA(1)
PB|A(0|0) PB|A(1|0) PB|A(0|1) PB|A(1|1)
PE|AB(0|0,0) PE|AB(1|0,0) PE|AB(0|0,1) PE|AB(1|0,1) PE|AB(0|1,0) PE|AB(1|1,0) PE|AB(0|1,1) PE|AB(1|1,1)
PD|BC(0|0,0)×PC|A(0|0)
A
C
B
DE
A
D
B C
E
![Page 23: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/23.jpg)
CS 276 28
0
A
B
0
E D
0 1
C
0
0 1
1
C
0 1
1
E D
0 1
C
0
0 1
1
C
0 1
P(A=0)
P(B=0|A=0) P(B=1|A=0)
P(E=0|A=0,B=0)
P(D=0|B=0,C=0)×P(C=0|A=0)
P(D=1|B=1,C=1)×P(C=1|A=0)
P(D=0|B=0,C=1)×P(C=1|A=0)
P(D=1|B=0,C=0)×P(C=0|A=0)
P(D=1|B=0,C=1)×P(C=1|A=0)
P(D=0|B=1,C=0)×P(C=0|A=0)
P(D=0|B=1,C=1)×P(C=1|A=0)
P(D=1|B=1,C=0)×P(C=0|A=0)
P(E=1|A=0,B=0) P(E=0|A=0,B=1) P(E=1|A=0,B=1)
A
C
B
DE
A
D
B C
E
![Page 24: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/24.jpg)
CS 276 29
0
A
B
0
E C
0 1
D
0
0 1
1
D
0 1
1
E C
0 1
D
0
0 1
1
D
0 1
P(A=0)
P(B=0|A=0) P(B=1|A=0)
P(E=0|A=0,B=0)
P(D=0|B=0,C=0) P(D=1|B=1,C=1)P(D=0|B=0,C=1) P(D=1|B=0,C=0) P(D=1|B=0,C=1) P(D=0|B=1,C=0) P(D=0|B=1,C=1) P(D=1|B=1,C=0)
P(E=1|A=0,B=0) P(E=0|A=0,B=1) P(E=1|A=0,B=1)
P(C=0|A=0) P(C=1|A=0) P(C=0|A=0) P(C=1|A=0)
(d)
![Page 25: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/25.jpg)
CS 276 30
AND/OR Tree DFS Algorithm (Belief Updating)
AND node: Combination operator (product)
OR node: Marginalization operator (summation)
Value of node = updated belief for sub-problem below
0
A
B
0
E
OR
AND
OR
AND
OR
AND
C
0
OR
AND
D
0 1
1
D
0 1
0 1
1
E C
0
D
0 1
1
D
0 1
0 1
1
B
0
E C
0
D
0 1
1
D
0 1
0 1
1
E C
0
D
0 1
1
D
0 1
0 1
A
D
B C
E
A
D
B
CE
B C D=0
D=1
0 0 .2 .80 1 .1 .91 0 .3 .71 1 .5 .5
),|( CBDP
.7.8 .9 .5 .7.8 .9 .5
Evidence: D=1
A B E=0
E=1
0 0 .4 .60 1 .5 .51 0 .7 .31 1 .2 .8
),|( BAEP
Evidence: E=0
.4 .5 .7 .2
A B=0
B=1
0 .4 .61 .1 .9
)|( ABPA C=
0C=1
0 .2 .81 .7 .3
)|( ACPA P(A
)0 .61 .4
)(AP
.2 .8 .2 .8 .1 .9 .1 .9
.4 .6 .1 .9
.6 .4
.8 .9
.8 .9
.7 .5
.7 .5
.8 .9
.8 .9
.7 .5
.7 .5
.4 .5 .7 .2.88 .54 .89 .52
.352 .27 .623 .104
.3028 .1559
.24408
.3028 .1559
Result: P(D=1,E=0)
![Page 26: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/26.jpg)
CS 276 31
Complexity of AND/OR Tree Search
AND/OR tree OR tree
Space O(n) O(n)
Time
O(n km)O(n kw* log n)
[Freuder & Quinn85], [Collin, Dechter & Katz91], [Bayardo & Miranker95], [Darwiche01]
O(kn)
k = domain sizem = depth of pseudo-treen = number of variablesw*= treewidth
![Page 27: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/27.jpg)
CS 276 32
From Search Trees to Search Graphs
Any two nodes that root identical subtrees (subgraphs) can be merged
![Page 28: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/28.jpg)
CS 276 33
From Search Trees to Search Graphs
Any two nodes that root identical subtrees (subgraphs) can be merged
![Page 29: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/29.jpg)
CS 276 35
AND/OR Tree
A
D
B C
E
F
A
D
B
CE
F
G H
J
K
G
H
J
KAOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND
AND 0 10 1 0 10 1 0 10 1 0 10 1
OR
OR
AND
AND
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
![Page 30: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/30.jpg)
CS 276 36
An AND/ORgraph
A
D
B C
E
F
A
D
B
CE
F
G H
J
K
G
H
J
KAOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND
AND 0 10 1 0 10 1 0 10 1 0 10 1
OR
OR
AND
AND
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
![Page 31: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/31.jpg)
CS 276 37
Merging Based on ContextOne way of recognizing nodes that can be
merged:
context (X) = ancestors of X in pseudo tree that are connected to X, or to descendants of X
[ ]
[A]
[AB]
[AE][BC]
[AB]
A
D
B
EC
F
pseudo tree
A
E
C
B
F
D
A
E
C
B
F
D
![Page 32: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/32.jpg)
CS 276 38
AND/OR Search Graph
A
E
C
B
F
D
A
D
B
EC
F
A B C RABC
0 0 0 10 0 1 10 1 0 00 1 1 11 0 0 11 0 1 11 1 0 11 1 1 0
A B E RABE
0 0 0 10 0 1 00 1 0 10 1 1 11 0 0 01 0 1 11 1 0 11 1 1 0
A E F RAEF
0 0 0 00 0 1 10 1 0 10 1 1 11 0 0 11 0 1 11 1 0 11 1 1 0
B C D RBCD
0 0 0 10 0 1 10 1 0 10 1 1 01 0 0 11 0 1 01 1 0 11 1 1 1
Constraint Satisfaction – Counting Solutions
[ ]
[A]
[AB]
[AE][BC]
[AB]
Context
pseudo tree
context minimal graph
AOR
0AND
BOR
0AND
OR E
OR
AND
AND 0 1
C
D D
0 1 0 1
0 1
1
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
1
B
0
E
F F
0 1 0 1
0 1
C
0 1
1
E
0 1
C
0 1
![Page 33: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/33.jpg)
CS 276 39
.8 .9 .7.7.8
AND/OR Tree DFS Algorithm (Belief Updating)
AND node: Combination operator (product)OR node: Marginalization operator (summation)
Value of node = updated belief for sub-problem below
0
A
B
0
E C
0
D
0 1
1
D
0 1
0 1
1
E C
0
D
0 1
1
D
0 1
0 1
1
B
0
E C
0
D
0 1
1
D
0 1
0 1
1
E C
0
D
0 1
1
D
0 1
0 1
B C D=0
D=1
0 0 .2 .80 1 .1 .91 0 .3 .71 1 .5 .5
),|( CBDP
.7.8 .9 .5 .7.8 .9 .5
Evidence: D=1
A B E=0
E=1
0 0 .4 .60 1 .5 .51 0 .7 .31 1 .2 .8
),|( BAEP
Evidence: E=0
.4 .5 .7 .2
A B=0
B=1
0 .4 .61 .1 .9
)|( ABPA C=
0C=1
0 .2 .81 .7 .3
)|( ACPA P(A
)0 .61 .4
)(AP
.2 .8 .2 .8 .1 .9 .1 .9
.4 .6 .1 .9
.6 .4
.9
.8 .9
.5
.7 .5 .8 .9
.5
.7 .5
.4 .5 .7 .2.88 .54 .89 .52
.352 .27 .623 .104
.3028 .1559
.24408
.3028 .1559
Result: P(D=1,E=0)
A
D
B
CE
[ ]
[A]
[AB]
[BC]
[AB]
ContextA
D
B C
E
![Page 34: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/34.jpg)
CS 276 40
.7.8
AND/OR Graph DFS Algorithm (Belief Updating)
0
A
B
0
E C
0
D
0 1
1
D
0 1
0 1
1
E C
0
D
0 1
1
D
0 1
0 1
1
B
0
E C
0 10 1
1
E C
0 10 1
A
D
B C
E
B C D=0
D=1
0 0 .2 .80 1 .1 .91 0 .3 .71 1 .5 .5
),|( CBDP.7.8 .9 .5
Evidence: D=1
A B E=0
E=1
0 0 .4 .60 1 .5 .51 0 .7 .31 1 .2 .8
),|( BAEP
Evidence: E=0
.4 .5 .7 .2
A B=0
B=1
0 .4 .61 .1 .9
)|( ABPA C=
0C=1
0 .2 .81 .7 .3
)|( ACPA P(A
)0 .61 .4
)(AP
.2 .8 .2 .8 .1 .9 .1 .9
.4 .6 .1 .9
.6 .4
.9
.8 .9
.5
.7 .5 .8 .9 .7 .5
.4 .5 .7 .2.88 .54 .89 .52
.352 .27 .623 .104
.3028 .1559
.24408
.3028 .1559
A
D
B
CE
[ ]
[A]
[AB]
[BC]
[AB]
Context
B C Value0 0 .80 1 .91 0 .71 1 .1
Cache table for D
Result: P(D=1,E=0)
![Page 35: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/35.jpg)
CS 276 41
AND/OR context minimal graph
C
0
K
0
H
0
L
0 1
N N
0 1 0 1
F F F
1 1
0 1 0 1
F
G
0 1
1
A
0 1
B B
0 1 0 1
E EE E
0 1 0 1
J JJ J
0 1 0 1
A
0 1
B B
0 1 0 1
E EE E
0 1 0 1
J JJ J
0 1 0 1
G
0 1
G
0 1
G
0 1
M
0 1
M
0 1
M
0 1
M
0 1
P
0 1
P
0 1
O
0 1
O
0 1
O
0 1
O
0 1
L
0 1
N N
0 1 0 1
P
0 1
P
0 1
O
0 1
O
0 1
O
0 1
O
0 1
D
0 1
D
0 1
D
0 1
D
0 1
K
0
H
0
L
0 1
N N
0 1 0 1
1 1
A
0 1
B B
0 1 0 1
E EE E
0 1 0 1
J JJ J
0 1 0 1
A
0 1
B B
0 1 0 1
E EE E
0 1 0 1
J JJ J
0 1 0 1
P
0 1
P
0 1
O
0 1
O
0 1
O
0 1
O
0 1
L
0 1
N N
0 1 0 1
P
0 1
P
0 1
O
0 1
O
0 1
O
0 1
O
0 1
D
0 1
D
0 1
D
0 1
D
0 1
B A
C
E
F G
HJ
D
K M
L
N
OP
C
HK
D
M
F
G
A
B
E
J
O
L
N
P
[AB]
[AF][CHAE]
[CEJ]
[CD]
[CHAB]
[CHA]
[CH]
[C]
[ ]
[CKO]
[CKLN]
[CKL]
[CK]
[C]
(C K H A B E J L N O D P M F G)
C
HK
D
M
F
G
A
B
E
J
O
L
N
P
[AB]
[AF][CHAE]
[CEJ]
[CD]
[CHAB]
[CHA]
[CH]
[C]
[ ]
[CKO]
[CKLN]
[CKL]
[CK]
[C]
(C K H A B E J L N O D P M F G)
![Page 36: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/36.jpg)
CS 276 42
How Big Is the Context?
Theorem: The maximum context size for a pseudo tree is equal to the treewidth of the graph along the pseudo tree.
C
HK
D
M
F
G
A
B
E
J
O
L
N
P
[AB]
[AF][CHAE]
[CEJ]
[CD]
[CHAB]
[CHA]
[CH]
[C]
[ ]
[CKO]
[CKLN]
[CKL]
[CK]
[C]
(C K H A B E J L N O D P M F G)
B A
C
E
F G
HJ
D
K M
L
N
OP
max context size = treewidth
![Page 37: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/37.jpg)
CS 276 43
G
EK
F
L
H
C
BA
M
J
D
Treewidth vs. Pathwidth
EK
L
H
C
A
M
J
ABC
BDEF
BDFG
EFH
FHK
HJ KLM
treewidth = 3 = (max cluster size) - 1
ABC
BDEFG
EFH
FHKJ
KLM
pathwidth = 4 = (max cluster size) - 1
D
G
B
F
TREE
CHAIN
![Page 38: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/38.jpg)
CS 276 44
Complexity of AND/OR Graph Search
AND/OR graph OR graph
Space O(n kw*) O(n kpw*)
TimeO(n kw*)
O(n kpw*)
k = domain sizen = number of variablesw*= treewidthpw*= pathwidth
w* ≤ pw* ≤ w* log n
![Page 39: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/39.jpg)
CS 276 45
AND/OR search algorithms AO(i)
i = the max size of a cache table (i.e. number of variables in a context)
i = 0 i = w* treewidth
Space:
Time:
O(n exp w* )
O(n exp w* )
O( n )
O( exp (w* log n) )
![Page 40: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/40.jpg)
CS 276 46
Searching AND/OR Graphs AO(i): searches depth-first, cache i-context
i = the max size of a cache table (i.e. number of variables in a context)
i=0
i=w*
Space: O(n)
Time: O(exp(w* log n))
Space: O(exp w*)
Time: O(exp w*)Space: O(exp(i) )
Time: O(exp(m_i+i )
i
![Page 41: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/41.jpg)
CS 276 47
Caching
A
D
B C
E
F
A
D
B
CE
F
G H
J
K
G
H
J
KAOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND
AND 0 10 1 0 10 1 0 10 1 0 10 1
OR
OR
AND
AND
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
context(D)={D}
context(F)={F}
![Page 42: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/42.jpg)
CS 276 48
Caching
A
D
B C
E
F
A
D
B
CE
F
G H
J
K
G
H
J
KAOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND
AND 0 10 1 0 10 1 0 10 1 0 10 1
OR
OR
AND
AND
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
context(D)={D}
context(F)={F}
![Page 43: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/43.jpg)
CS 276 52
All four search spaces
E
C
F
D
B
A
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1
0 1
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1
0 1
0 1 0 1
E
C
F
D
B
A 0 1
0 1
0 1 0 1 0 1
AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
ORD F D F D F D F D F D F D F D F
AND 0 1 0 1
AND0 10 1 0 1 0 1 0 1
Full OR search tree
Full AND/OR search tree
Context minimal OR search graph
Context minimal AND/OR search graph
A
D
B C
E
F
A
D
B
CE
F
![Page 44: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/44.jpg)
CS 276 53
All four search spaces
E
C
F
D
B
A
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1
0 1
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1
0 1
0 1 0 1
E
C
F
D
B
A 0 1
0 1
0 1 0 1 0 1
AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
ORD F D F D F D F D F D F D F D F
AND 0 1 0 1
AND0 10 1 0 1 0 1 0 1
Full OR search tree
Full AND/OR search tree
Context minimal OR search graph
Context minimal AND/OR search graph
A
D
B C
E
F
A
D
B
CE
F
Time-space
![Page 45: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/45.jpg)
CS 276 54
Available code http://graphmod.ics.uci.edu/group/
Software
![Page 46: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/46.jpg)
CS 276 55
AND/OR w-cutset
A
C
B K
G L
D F
H
M
J
E
AC
B K
G
L
D
FH
M
J
E
A
C
B K
G L
D F
H
M
J
E
C
B K
G
L
D
FH
M
J
E
3-cutset
A
C
B K
G L
D F
H
M
J
E
C
K
G
L
D
FH
M
J
E
2-cutset
A
C
B K
G L
D F
H
M
J
E
L
D
FH
M
J
E
1-cutset
![Page 47: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/47.jpg)
CS 276 56
AND/OR w-cutset
AC
B K
G
L
D
FH
M
J
E
AC
B K
G
L
D
FH
M
J
E
AC
B K
G
L
D
FH
M
J
E
pseudo tree 1-cutset treegrahpical model
![Page 48: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/48.jpg)
CS 276 57
Searching AND/OR Graphs AO(i): searches depth-first, cache i-context
i = the max size of a cache table (i.e. number of variables in a context)
i=0
i=w*
Space: O(n)
Time: O(exp(w* log n))
Space: O(exp w*)
Time: O(exp w*)Space: O(exp(i) )
Time: O(exp(m_i+i )
i
![Page 49: Exact Reasoning: conditioning search and hybrids](https://reader033.vdocument.in/reader033/viewer/2022051402/56815809550346895dc577de/html5/thumbnails/49.jpg)
CS 276 58
w-cutset Trees Over AND/OR space
Definition: T_w is a w-cutset tree relative to backbone tree T, iff
T_w is roots T and when removed, yields tree-width w.
Theorem: AO(i) time complexity for backbone T is time
O(exp(i+m_i)) and space O(i), m_i is the depth of the T_i tree.
Better than w-cutset: O(exp(i+c_i)) when c_i is the number of nodes in T_i