Tree Decomposition methodsTree Decomposition methods

Chapter 9Chapter 9

ICS-275

Spring 2007

Graph concepts reviews:Hyper graphs and dual graphs A hypergraph is H = (V,S) , V=

{v_1,..,v_n} and a set of subsets Hyperegdes: S={S_1, ..., S_l }.

Dual graphs of a hypergaph: The nodes are the hyperedges and a pair of nodes is connected if they share vertices in V. The arc is labeled by the shared vertices.

A primal graph of a hypergraph H = (V,S) has $V$ as its nodes, and any two nodes are connected by an arc if they appear in the same hyperedge.

if all the constraints of a network R are binary, then its hypergraph is identical to its primal graph.

Acyclic Networks The running intersection property

(connectedness): An arc can be removed from the dual graph if the variables labeling the arcs are shared along an alternative path between the two endpoints.

Join graph: An arc subgraph of the dual graph that satisfies the connectedness property.

Join-tree: a join-graph with no cycles Hypertree: A hypergraph whose dual

graph has a join-tree. Acyclic network: is one whose

hypergraph is a hypertree.

Solving acyclic networks

Algorithm acyclic-solving applies a tree algorithm to the join-tree. It applies (a little more than) directional relational arc-consistency from leaves to root.

Complexity: acyclic-solving is O(r l log l) steps, where r is the number of constraints and l bounds the number of tuples in each constraint relation

(It assumes that join of two relations when one’s scope is a subset of the other can be done in linear time)

Example

Constraints are: R_{ABC} = R_{AEF} = R_{CDE} = {(0,0,1) (0,1,0)(1,0,0)} and R_{ACE} = { (1,1,0) (0,1,1) (1,0,1) }.

d= (R_{ACE},R_{CDE},R_{AEF},R_{ABC}). • When processing R_{ABC}, its parent relation is R_{ACE}; we generate \

pi_{ACE} (R_{ACE} x R_{ABC}), yielding R_{ACE} = {(0,1,1)(1,0,1)}. • processing R_{AEF} we generate relation R_{ACE} = \pi_{ACE} ( R_{ACE} x

R_{AEF} ) = {(0,1,1)}. • processing R_{CDE} we generate: R_{ACE} = \pi_{ACE} ( R_{ACE} x R_{CDE}

) = {(0,1,1)}. A solution is generated by picking the only allowed tuple for R_{ACE},

A=0,C=1,E=1, extending it with a value for D that satisfies R_{CDE}, which is only D=0, and then similarly extending the assignment to F=0 and B=0, to satisfy R_{AEF} and R_{ABC}.

Recognizing acyclic networks

Dual-based recognition: • perform maximal spanning tree over the dual graph

and check connectedness of the resulting tree.

• Dual-acyclicity complexity is O(e^3) Primal-based recognition:

• Theorem (Maier 83): A hypergraph has a join-tree iff its primal graph is chordal and conformal.

• A chordal primal graph is conformal relative to a constraint hypergraph iff there is a one-to-one mapping between maximal cliques and scopes of constraints.

Primal-based recognition

Check cordality using max-cardinality ordering.

Test conformality Create a join-tree:

connect every clicque to an earlier clique sharing maximal number of variables

Tree-based clustering Convert a constraint problem to an acyclic-one: group subset of

constraints to clusters until we get an acyclic problem.

Hypertree embedding of a hypergraph H = (X,H) is a hypertree S = (X, S) s.t., for every h in H there is h_1 in S s.t. h is included in h_1.

This yield algorithm join-tree clustering

Hypertree partitioning of a hypergraph H = (X,H) is a hypertree S = (X, S) s.t., for every h in H there is h_1 in S s.t. h is included in h_1 and X is the union of scopes in h_1.

Join-tree clustering Input: A constraint problem R =(X,D,C) and its primal graph G = (X,E). Output: An equivalent acyclic constraint problem and its join-tree: T= (X,D, {C ‘}) 1. Select an d = (x_1,...,x_n) 2. Triangulation(create the induced graph along $d$ and call it G^*: ) for j=n to 1 by -1 do E E U {(i,k)| (i,j) in E,(k,j) in E } 3. Create a join-tree of the induced graph G^*: a. Identify all maximal cliques (each variable and its parents is a clique). Let C_1,...,C_t be all such cliques, b. Create a tree-structure T over the cliques: Connect each C_{i} to a C_{j} (j < I) with whom it shares largest subset of variables. 4. Place each input constraint in one clique containing its scope, and let P_i be the constraint subproblem associated with C_i. 5. Solve P_i and let {R'}_i $ be its set of solutions. 6. Return C' = {R'}_1,..., {R'}_t the new set of constraints and their join-tree, T.

Theorem: join-tree clustering trnasforms a constraint network into an acyclic network

Example of tree-clustering

Complexity of JTC

THEOREM 5 (complexity of JTC) join-tree clustering is O(r k^ (w*(d)+1))

time and O(nk^(w*(d)+1)) space, where k is the maximum domain size and w*(d) is the induced width of the ordered graph.

The complexity of acyclic-solving is O(nw*(d) log k k^(w*(d)+1))

Let P=<X,D,C}> be an automated reasoning problem. A tree decomposition is <T,,>, such that

T=(V,E) is a tree associates a set of variables (v)X with each node associates a set of functions (v)C with each node

such that

1. RiC, there is exactly one v such that Ri(v) and scope(Ri)(v).2. xX, the set {vV|x(v)} induces a connected subtree.

Unifying tree-decompositions

Let P=<X,D,C}> be an automated reasoning problem. A tree decomposition is <T,,>, such that

T=(V,E) is a tree associates a set of variables (v)X with each node associates a set of functions (v)C with each node

such that1. RiC, there is exactly one v such that Ri(v) and scope(Ri)(v).1a. v, (v) scope((v)).2. xX, the set {vV|x(v)} induces a connected subtree.

HyperTree Decomposition

w (tree-width) is maxvV |(v)| hw (hypertree width) is maxvV | (v)|

sep (max separator size) is max(u,v) |sep(u,v)|

Example of two join-trees again

hyperTree-decomposition

Treedecomposition

Cluster Tree Elimination

Cluster Tree Elimination (CTE) works by passing messages along a tree-decomposition

Basic idea:• Each node sends one message to each of its neighbors

• Node u sends a message to its neighbor v only when u received messages from all its other neighbors

)(

:message theCompute

)(),(),( iuclusterRvusepvu Rmi

Constraint Propagation

u v

x1

x2

xn

m(u,v)

)},(),,(),...,,(),,({)()( 21 uvmuxmuxmuxmuucluster n

Implementation tradeoffs Implementing Equation 1: The particular implementation of equation (1) in CTE

can vary. One option is to generate the combined relation (1Ri2clusterv(u) Ri) before sending

messages to neighbor v. The other option, which we assume here, is that the message sent to

each neighbor is created without recording the relation (1Ri2clusterv(u) Ri). Rather, each tuple in the

join is projected on the separator immediately after being created. This will yields a better memory

utilization. Furthermore, when u sends a message to v its cluster may contain the message it

received from v. Thus in a synchronized message passing we can allow a single enumeration of

the tuples in cluster(u) when the messages are sent back towards the leaves, each of which be

projected in parallel on the separators of the outgoing messages. The output of CTE is the original tree-decomposition where

Cluster-Tree Elimination (CTE)

...m(v1,u)

m(v2,u)m(vi,u)

(u)(u)

m(u,w) = sep(u,w) [j {m(vj,u)} (u)]

Cluster Tree Elimination - properties

Correctness and completeness: Algorithm CTE is correct, i.e. it computes the exact joint probability of every single variable and the evidence.

Time complexity: O ( deg (n+N) d w*+1 )

Space complexity: O ( N d sep)where deg = the maximum degree of a node

n = number of variables (= number of CPTs)N = number of nodes in the tree decompositiond = the maximum domain size of a variablew* = the induced widthsep = the separator size

Time and space by hyperwidth only if hypertree decomposition:

time and O(N t^w*) space,),( 2hwNtO

Cluster Tree Elimination - properties

Correctness and completeness: Algorithm CTE is correct, i.e. it computes the exact joint probability of a single variable and the evidence.

Time complexity: O ( deg (r+N) kw*+1 ) Space complexity: O ( N d sep)

where deg = the maximum degree of a node

nr= number of of CPTs

N = number of nodes in the tree decomposition

k= the maximum domain size of a variable

w* = the induced width

sep = the separator size

JTC is O ( r k w*+1 ) time and space

Example of CTE message propagation

Each function in a cluster

Satisfy running intersection property

Tree-width: number of variables in a cluster-1

Equals induced-width

G

E

F

C D

B

AA B C

R(a), R(b,a), R(c,a,b)

B C D FR(d,b), R(f,c,d)

B E FR(e,b,f)

E F GR(g,e,f)

2

4

1

3

EF

BC

BF

Join-Tree Decomposition(Dechter and Pearl 1989)

),,(),()(),()2,1( bacRabRaRcbh a

A B C R(a), R(b,a), R(c,a,b)

B C D FR(d,b), R(f,c,d),h(1,2)(b,c)

B E FR(e,b,f), h(2,3)(b,f)

E F GR(g,e,f)

),(),,(),(),( )2,1(,)3,2( cbhdcfRbdRfbh dc

2

4

1

3

EF

BC

BFsep(2,3)={B,F}

elim(2,3)={C,D}

Cluster Tree Elimination

joinproject

),,(),()(),()2,1( bacRabRaRcbh a

),(),,(),(),( )2,3(,)1,2( fbhdcfRbdRcbh fd

),(),,(),(),( )2,1(,)3,2( cbhdcfRbdRfbh dc

),(),,(),( )3,4()2,3( fehfbeRfbh e

),(),,(),( )3,2()4,3( fbhfbeRfeh b

),,(),()3,4( fegRfeh g

ABC

2

4

1

3 BEF

EFG

EF

BF

BC

BCDF

Time: O ( exp(w*+1 ))Space: O ( exp(sep))Time: O(exp(hw)) (Gottlob et. Al., 2000)

CTE: Cluster Tree Elimination

G

E

F

C D

B

A

Decomposable subproblems

DEFINITION 8 (decomposable subproblem) Given a constraint problem R = (X;D;C) and a

subset of variables Y µ X, a subproblem over Y , RY = (Y;DY ;CY ), is decomposable relative

to R iff sol(RY ) = ¼Y sol(R) where sol(R) is the set of all solutions of network R

A123

A C1 23 2

A B1 21 32 1

2 3

3 13 2

B C F1 2 33 2 1

A B D1 2 31 3 22 1 32 3 13 1 23 2 1

D F G1 2 32 1 3

1R

2R

4R

3R

5R

6R

Distributed relational arc-consistency example

A

B C

D F

G

The message that R2 sends to R1 is

R1 updates its relation and domains and sends messages to neighbors

Distributed Arc-Consistency

b) Constraint network

DR-AC can be applied to the dual problem of any constraint network.

A

AB AC

ABD BCF

DFG

A

AB

A

A

AB

C

B

D F

A

A1

2

3

A C1 23 2

A B1 21 32 12 33 13 2

B C F1 2 33 2 1

A B D1 2 31 3 22 1 32 3 13 1 23 2 1 D F G

1 2 32 1 3

1R

2R

4R

3R

5R

6R

A

AB AC

ABD BCF

DFG

B

4 5

3

6

2

B

D F

A

A

A

C

1

DR-AC on a dual join-graph

A123

A C1 23 2

A B1 21 32 12 33 13 2

B C F1 2 33 2 1

A B D1 2 31 3 22 1 32 3 13 1 23 2 1 D F G

1 2 32 1 3

A123

A123

A123

A13

B123

13

B

A123

D12

F13

D123

C2

B123

C2

F13

A123

B123

Iteration 1

A

AB AC

ABD BCF

DFG

B

4 5

3

6

2

B

D F

A

A

A

C

1

21h

31h

13h1

2h 14h

35h

24h2

5h

41h4

6h 42h

65h6

4h

52h 5

3h 56h

1R

2R

4R

3R

5R

6R

A13

A C1 23 2

A B1 32 12 33 1

B C F1 2 33 2 1

A B D1 3 22 3 13 1 23 2 1

D F G2 1 3

1R

2R

4R

3R

5R

6R

A

AB AC

ABD BCF

DFG

B

4 5

3

6

2

B

D F

A

A

A

C

1

Iteration 1

A13

A C1 23 2

A B1 32 12 33 1

B C F1 2 33 2 1

A B D1 3 22 3 13 1 23 2 1

D F G2 1 3

A13

A13

A123

A13

A13

D2

F13

D12

C2

B13

C2

F1

A123

B13

B123

A

AB AC

ABD BCF

DFG

B

4 5

3

6

2

B

D F

A

A

A

C

1

21h

31h

13h1

2h 14h

35h

24h2

5h

41h4

6h 42h

65h6

4h

52h 5

3h 56h

1R

2R

4R

3R

5R

6R

Iteration 2

13

B

A13

A C1 23 2

A B1 33 1

B C F3 2 1

A B D1 3 23 1 2

D F G2 1 3

Iteration 2

1R

2R

4R

3R

5R

6R

A

AB AC

ABD BCF

DFG

B

4 5

3

6

2

B

D F

A

A

A

C

1

A13

A13

A13

B3

A13

D2

F1

D2

C2

B13

C2

F1

A13

A13

A13

A C1 23 2

A B1 33 1

B C F3 2 1

A B D1 3 23 1 2

D F G2 1 3

B13

B13

21h

31h

13h1

2h 14h

35h

24h2

5h

41h4

6h 42h

65h6

4h

52h 5

3h 56h

1R

2R

4R

3R

5R

6R

A

AB AC

ABD BCF

DFG

B

4 5

3

6

2

B

D F

A

A

A

C

1

Iteration 3

A13

A C1 23 2

A B1 3

B C F3 2 1

A B D1 3 23 1 2

D F G2 1 3

1R

2R

4R

3R

5R

6R

A

AB AC

ABD BCF

DFG

B

4 5

3

6

2

B

D F

A

A

A

C

1

Iteration 3

A13

A13

A1

B3

A13

D2

F1

D2

C2

B3

C2

F1

A13

A13

A13

A C1 23 2

A B1 3

B C F3 2 1

A B D1 3 23 1 2

D F G2 1 3

B13

B13

21h

31h

13h1

2h 14h

35h

24h2

5h

41h4

6h 42h

65h6

4h

52h 5

3h 56h

1R

2R

4R

3R

5R

6R

A

AB AC

ABD BCF

DFG

B

4 5

3

6

2

B

D F

A

A

A

C

1

Iteration 4

A1

A C1 23 2

A B1 3

B C F3 2 1

A B D1 3 2

D F G2 1 3

1R

2R

4R

3R

5R

6R

A

AB AC

ABD BCF

DFG

B

4 5

3

6

2

B

D F

A

A

A

C

1

Iteration 4

A1

A1

A1

B3

B3

B3

A1

D2

F1

D2

C2

B3

C2

F1

A1

A1

A1

A C1 23 2

A B1 3

B C F3 2 1

A B D1 3 2

D F G2 1 3

21h

31h

13h1

2h 14h

35h

24h2

5h

41h4

6h 42h

65h6

4h

52h 5

3h 56h

1R

2R

4R

3R

5R

6R

A

AB AC

ABD BCF

DFG

B

4 5

3

6

2

B

D F

A

A

A

C

1

Iteration 5

A1

A C1 2

A B1 3

B C F3 2 1

A B D1 3 2

D F G2 1 3

1R

2R

4R

3R

5R

6R

A

AB AC

ABD BCF

DFG

B

4 5

3

6

2

B

D F

A

A

A

C

1

Iteration 5

Join-tree clustering is a restricted tree-decomposition:example

Adaptive-consistency as tree-decomposition

Adaptive consistency is a message-passing along a bucket-tree

Bucket trees: each bucket is a node and it is connected to a bucket to which its message is sent.• The variables are the clicue of the triangulated graph

• The funcions are those placed in the initial partition

Bucket EliminationAdaptive Consistency (Dechter and Pearl, 1987)

d ordering along widthinduced -(d) ,

*

*

w(d)))exp(w O(n :Complexity

E

D

A

C

B

}2,1{

}2,1{}2,1{

}2,1{ }3,2,1{

:)(AB :)(BC :)(AD :)(

BE C,E D,E :)(

ABucketBBucketCBucketDBucketEBucket

A

E

D

C

B

:)(EB :)(

EC , BC :)(ED :)(

BA D,A :)(

EBucketBBucketCBucketDBucketABucket

E

A

D

C

B

|| RDBE ,

|| RE

|| RDB

|| RDCB

|| RACB

|| RAB

RA

RCBE

From bucket-elimination to bucket-tree propagation

The bottom up messages

Adaptive-Tree-Consistency as tree-decomposition

Adaptive consistency is a message-passing along a bucket-tree

Bucket trees: each bucket is a node and it is connected to a bucket to which its message is sent.

Theorem: A bucket-tree is a tree-decomposition Therefore, CTE adds a bottom-up message passing

to bucket-elimination.

The complexity of ATC is O(r deg k^(w*+1))

A

ABDE

FGI

ABC

BCE

GHIJ

CDEF

FGH

C

H

A C

A AB BC

BE

C

CDE CE

FH

FFG GH H

GI

A

ABDE

FGI

ABC

BCE

GHIJ

CDEF

FGH

C

H

A

AB BC

CDE CE

H

FF GH

GI

ABCDE

FGI

BCE

GHIJ

CDEF

FGH

BC

DE CE

FF GH

GI

ABCDE

FGHI GHIJ

CDEF

CDE

F

GHI

more accuracy

less complexity

Join-Graphs and Iterative Join-graph propagation (IJGP)