independence, decomposability and functions which take values into an abelian group adrian silvescu...
TRANSCRIPT
![Page 1: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/1.jpg)
Independence, Decomposability and functions which take values into an
Abelian Group
Adrian Silvescu Vasant Honavar
Department of Computer ScienceIowa State University
![Page 2: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/2.jpg)
Decomposition and Independence
Decomposition renders problems more tractable.
Apply recursively Decomposition is enabled
by “independence” Decomposition and
independence are dual notions
A B
A B
A B
A B
![Page 3: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/3.jpg)
Conditional Decomposition and Independence
Seldom are the two sub-problems disjoint
All is not lost Conditional
Decomposition / Independence
Conditioning on C C a.k.a. separator
CA B
A BA C C B
CA B
C
=
![Page 4: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/4.jpg)
Formalization of the intuitions
Problem P = (D, S, solP) D = Domain, S = Solutions solP : D S
A BsolP
Example: Determinant_Computation(M2, R, det)
![Page 5: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/5.jpg)
Conditional Independence / Decomposition Formalization (Variable Based)
P = (D = A X B X C, S, solP)
P1 = (A X C, S1, solP1), P2 = (B X C, S2, solP2)
solP(A, B, C) = solP1(A, C) solP2(B, C)
PPP 2,1
![Page 6: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/6.jpg)
Probabilities
I(A, B|C) iff P(A, B| C) = P(A|C) P(B|C) Equivalently P(A, B, C) = P(A, C) P(B|C) P(A, B, C) = f1(A, C) f2(B, C) Independencies can be represented by a
graph where we do not draw edges between variables that are independent conditioned on the rest of the variables.
A C B
![Page 7: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/7.jpg)
The Hammersley-Clifford Theorem: From Pairwise to Holistic Decomposability
)(
)()(GMaxCliquesCC CfVp
![Page 8: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/8.jpg)
Outline
Generalized Conditional Independence with respect to a function f and properties
Theorems Conclusions and Discussion
![Page 9: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/9.jpg)
Conditional Independence with respect to a function f - If(A,B|C)
solP(A, B, C) = solP1(A, C) solP2(B, C)
Assumptions: – S = S1 = S2 [= G]– .
– A, B, C is a partition of the set of all variables– Saturated independence statements – from now on
PPP 2,1
PPP 2,1
f(A, B, C) = f1(A, C) f2(B, C)
If(A,B|C)
![Page 10: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/10.jpg)
Conditional Independence with respect to a function f If(A,B|C) – cont’d
A B C f
0 0 0 .25
0 0 1 .3
… … … …
=
A C f1
0 0 .5
0 1 .3
… … …
B C f2
0 0 .5
0 1 .3
… … …
If(A,B|C) iff
f(A, B, C) = f1(A, C) f2(B, C)
![Page 11: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/11.jpg)
Examples of If(A,B|C )
Multiplicative (probabilities)
Additive (fitness, energy, value functions)
Relational (relations)
![Page 12: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/12.jpg)
Properties of If(A,B|C )
1.Trivial Independence If(A, Φ|C) 2. Symmetry If(A, B|C) => If(B, A|C) 3. Weak Union If(A, B U D|C) => If(A, B|C U D) 4. Intersection If(A, B|C U D) & If(A, D|C U B) => If(A, B U D|C)
A CD
B
![Page 13: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/13.jpg)
Abelian Groups
(G, +, 0, -) is an Abelian Group iff– + is associative and commutative– 0 is a neutral element – - is an inversion operator
Examples:– (R, + , 0, - ) - additive (value func.)– ((0, ∞), · , 1, -¹) - multiplicative (prob.)– ({0, 1}, mod2, 0, id) - relational (relations)
![Page 14: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/14.jpg)
Outline
Generalized Conditional Independence with respect to a function f
Properties and Theorems Conclusions and Discussion
![Page 15: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/15.jpg)
Markov Properties [Pearl & Paz ‘87]
If Axioms 1-4 then the following are equivalent
Pairwise – (α,β) G => If(α, β|V\{α,β})
Local -
If(α, V\(N(α)U{α})| N(α)) Global – If C=V\{A, B}
separates A and B in G If(A, B| C=V\{A, B})
α βV\{α,β}
N(α)α
A BC
![Page 16: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/16.jpg)
Factorization – Main Theorem
![Page 17: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/17.jpg)
The Factorization Theorem: From Pairwise to Holistic Decomposability
)()()(
CfVf CGMaxCliquesC
![Page 18: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/18.jpg)
Particular Cases - Factorization
Probabilistic – Hammersley-Clifford
Additive Decomposability
Relational Decomposability
![Page 19: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/19.jpg)
Graph Separability and Independence [Geiger & Pearl ‘ 93]
If Axioms 1-4 hold then
SepG(A, B|C) If(A, B|C)
for all saturated independence statements
![Page 20: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/20.jpg)
Completeness
Axioms 1-4 provide a complete axiomatic characterization of independence statements for functions which take values over Abelian groups
![Page 21: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/21.jpg)
Outline
Generalized Conditional Independence with respect to a function f
Properties and Theorems Conclusions and Discussion
![Page 22: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/22.jpg)
Conclusions (1)
Introduced a very general notion of Conditional Independence / Decomposability.
Developed it into a notion of Conditional Independence relative to a function f which takes values into an Abelian Group If(.,.|.).
We proved that If(.,.|.) satisfies the following important independence properties:
– 1. Trivial independence, – 2. Symmetry, – 3. Weak union – 4. Intersection
![Page 23: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/23.jpg)
Conclusions (2)
Axioms 1-4 imply the equivalence of the Global, Local and Pairwise Markov Properties for our notion conditional independence relation If(.,.|.)) based on the result from [Pearl and Paz '87].
We proved a natural generalization of the Hammersley-Clifford which allows us to factorize the function f over the cliques of an associated Markov Network which reflects the Conditional Independencies of subsets of variables with respect to f.
Completeness Theorem, Graph Separability Eq. Theorem The theory developed in this paper subsumes: probability
distributions, additive decomposable functions and relations, as particular cases of functions over Abelian Groups.
![Page 24: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/24.jpg)
Discussion: Relation to Graphoids
(-) Decomposition
(-) Contraction
(+) Weak Contraction
Graphoids – No finite axiomatic charact. [Studeny ’92]
Intersection Discussion – noninvertible elms.
![Page 25: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/25.jpg)
Discussion – cont’d
Graph Separability Independence Completeness Seems that
– Trivial Independence– Symmetry– Weak Union– Intersection
Strong Axiomatic core for Independence
![Page 26: Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa](https://reader036.vdocument.in/reader036/viewer/2022081603/5697bf7b1a28abf838c83ae3/html5/thumbnails/26.jpg)
Applications