on distributing a bayesian network
DESCRIPTION
On Distributing a Bayesian Network. Thor Whalen Metron, Inc. Major discussion points. Functionality afforded by old code Issues and problems with old approach Functionality trying to add Functionality successfully added What is not yet working correctly What sill have to try to add. - PowerPoint PPT PresentationTRANSCRIPT
1
On Distributing a Bayesian On Distributing a Bayesian NetworkNetwork
Thor Whalen
Metron, Inc.
2
Major discussion points
• Functionality afforded by old code
• Issues and problems with old approach
• Functionality trying to add
• Functionality successfully added
• What is not yet working correctly
• What sill have to try to add
3
Outline
• Need a Review of JT?
• Distributing a BN using the JT: Issues.
• Directions for Solutions
• Matlab: New functionality and experiments
• And now what?
4
a
b c
d e
f
g
h
Secondary Structure/Junction Tree• multi-dim. random variables• joint probabilities (potentials)
Bayesian Network• one-dim. random variables• conditional probabilities
abd
ade
ace
ceg
eghdef
ad ae ce
de eg
5
Building a Junction Tree
DAG
Moral Graph
Triangulated Graph
Junction Tree
Identifying Cliques
6
Step 1: Moralization
a
b c
d e
f
g
h
a
b c
d e
f
g
h
a
b c
d e
f
g
h
1. For all w V:• For all u,vpa(w) add an edge e=u-v.
2. Undirect all edges.
GMG = ( V , E )
7
Step 2: Triangulation
Add edges to GM such that there is no cyclewith length 4 that does not contain a chord.
NO YES
a
b c
d e
f
g
h
a
b c
d e
f
g
h
GM GT
8
a
b
d
a
c
e
d e
f
a
d e
e
g
h
c
e
g
a
b c
d e
f
g
h
Bayesian NetworkG = ( V , E )
a
b c
d e
f
g
h
a
b c
d e
f
g
h
Moral graph GM Triangulated graph GT
abd
ade
ace
ceg
eghdef
ad ae ce
de eg
seperators
Junction graph GJ (not complete)e.g. ceg egh = eg Cliques
e
e
e
a
e
9
Junction graph GJ (not complete)
abd
ade
ace
ceg
eghdef
ad ae ce
de eg
e
e
e
a
e
abd
ade
ace
ceg
eghdef
ad ae ce
de eg
Junction tree GJT
There are several methods to find MST.
Kruskal’s algorithm: choose successively a link of
maximal weight unless it creates a cycle.
10
a
b
d
a
c
e
d e
f
a
d e
e
g
h
c
e
g
abd
ade
ace
ceg
eghdef
ad ae ce
de eg
sepsets
In JT cliquesbecomesvertices
GJT
Ex: ceg egh = eg
11
Message Passing from clique A to clique B
1. Project the potential of A into SAB
2. Absorb the potential of SAB into B
Projection Absorption
Propagating potentialsPropagating potentials
12
1. COLLECT-EVIDENCE messages 1-52. DISTRIBUTE-EVIDENCE messages 6-10
Global PropagationGlobal Propagation
32
5
1 48 10
9
7
6
Root
abd
ade
ace
ceg
eghdef
ad ae ce
de eg
13
Using the JT for the MSBN
• How?
• Issue: Clique Granularity
• Issue: Clique Association
• Issue: MSBN must have tree structure
14
Take a JT
15
Partition JT into sub-trees
16
Issue: Clique Granularity
17
Issue: Clique Granularity
18
Issue: Clique association
19
Issue: Clique association
20
Issue: MSBN has a tree structure
21
Issue: MSBN has a tree structure
Can’t communicate here!
22
Issue: MSBN has a tree structure
Can’t communicate here!
Though these two subnets may share many variables.
23
Directions for Solutions
• Controlling Granularity
• Increasing Granularity
• Choosing the JT
• Alternative communication graphs
24
Controlling Granularity
• Moralization and Triangulation → Cliques
• Moralization: No choice.
• Triangulation: Some choice.
• Triangulate keeping the subnets in mind.
25
Increasing Granularity
• Can we break down cliques?
Conjecture:Given two sets of random variables X and Y, let
dist(P(X-Y|Y),P(X-Y|X Y))(X,Y) =
vol(X Y)
Given a clique Z whose set of random variables Z is covered by sets X and Y. If ∆(X,Y) is small enough then we may replace Z by X and Y.
Z X Y
26
Choosing The JT
• Once the cliques have been formed (hence the JG), any maximal weight spanning tree of the JG will do for the JT.
• Again, we should choose this tree so as to reduce the overhead when connecting subnets internally.
27
Alternative communication Graphs
We raise the question as to wether it is possible to perform intra- and extra-subnet calibration using some non-tree sub-graph of the JG…
28
MatLab: The new Stuff:
• Netica can talk to the Matlab tool.
• View/Interact tool is nicer
• JG for communication graph… (or not)
• Query cliques
29
Propagating with cycles
ABC BCE
BDE DEF
BC
EBE
B
DE
30
Mysterious convergence
ABC CE
BDE DEF
C
EE
B
DE