daphne koller message passing belief propagation algorithm probabilistic graphical models inference
TRANSCRIPT
Daphne Koller
Passing Messages
2: B,C
1: A,B
3: C,D
4: A,D
C
A
B D2,1 = 1
y1(A,B) y4(A,D)
y2(B,C) y3(C,D)
1,4 = B 2,1(B) y1(A,B)
Daphne Koller
Passing Messages
2: B,C
1: A,B
3: C,D
4: A,D
C
A
B D1,2 =2,1 =
2,3 =3,2 =
3,4 =4,3 =
1,4 =4,1 =
B 2,1(B) y1(A,B)D 3,4(D) y4(A,D)
C 2,3(C) y3(C,D)A 1,4(A) y4(A,D)
B 1,2(B) y2(B,C)D 4,3(D) y3(C,D)
A 4,1(A) y1(A,B)C 3,2(C) y2(B,C)
Daphne Koller
Cluster Graphs• Undirected graph such that:– nodes are clusters Ci {X1,…,Xn}
– edge between Ci and Cj associated with sepset Si,j Ci Cj
• Given set of factors , we assign each k to a cluster C(k) s.t. Scope[k] C(k)
• Define
Daphne Koller
Example Cluster Graph
1: A, B, C
3: B,D,F
2: B, C, D 5: D, E
4: B, EBB
BC
DD
E
1 = 1
2 = 2 3 6
4 = 5
5 = 4
3 = 7
Daphne Koller
Belief Propagation Algorithm• Assign each factor k to a cluster C(k)
• Construct initial potentials• Initialize all messages to be 1• Repeat
– Select edge (i,j) and pass message
• Compute
Daphne Koller
Belief Propagation Run
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0 5 10 15 20Iteration #
P(a
1)
True posterior
Daphne Koller
Summary• Graph of clusters connected by sepsets • Adjacent clusters pass information to each
other about variables in sepset– Message from i to j summarizes everything i
knows, except information obtained from j
• Algorithm may not converge• The resulting beliefs are pseudo-marginals• Nevertheless, very useful in practice