1 mpe and partial inversion in lifted probabilistic variable elimination rodrigo de salvo braz...
TRANSCRIPT
![Page 1: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/1.jpg)
1
MPE and Partial Inversion inLifted Probabilistic Variable Elimination
Rodrigo de Salvo Braz
University of Illinois at
Urbana-Champaignwith
Eyal Amir and Dan Roth
![Page 2: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/2.jpg)
Page 2
Lifted Probabilistic Inference
We assume probabilistic statements such as8 Person, DiseaseP(sick(Person,Disease) | epidemics(Disease)) = 0.3
Typical approach is grounding. We seek to do inference at first-order level,
like it is done in logic. Faster and more intelligible. Two contributions:
Partial inversion: more general technique than previous work (IJCAI '05)
MPE and Lifted assignments
![Page 3: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/3.jpg)
Page 3
Representing structure
sick(mary,measles)
epidemic(measles) epidemic(flu)
sick(mary,flu)
…
… sick(bob,measles) sick(bob,flu)……
… …
sick(P,D)
epidemic(D)
Poole (2003) named these parfactors,
for “parameterized factors”
Atom
Logical variabl
e
![Page 4: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/4.jpg)
Page 4
Parfactor
sick(Person,Disease)
epidemic(Disease)
8 Person, Disease sick(Person,Disease), epidemic(Disease))
![Page 5: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/5.jpg)
Page 5
Parfactor
sick(Person,Disease)
epidemic(Disease)
8 Person, Disease sick(Person,Disease), epidemic(Disease)),
Person mary, Disease flu
Person mary, Disease flu
![Page 6: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/6.jpg)
Page 6
Joint Distribution
As in propositional case, proportional to product of all factors But here, “all factors” means all instantiations of all parfactors:
P(...) X (p(X)) X,Y (p(X),q(X,Y))
![Page 7: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/7.jpg)
Page 7
Inference task - Marginalization
q(X,Y) X (p(X)) X,Y (p(X),q(X,Y))
Marginal on all random variables in p(X):summation over all assignments to all instances of q(X,Y)
![Page 8: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/8.jpg)
Page 8
The FOVE Algorithm
First-Order Variable Elimination (FOVE): a generalization of Variable Elimination in propositional graphical models.
Eliminates classes of random variables at once.
![Page 9: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/9.jpg)
Page 9
FOVE
P(hospital(mary)) = ?
sick(mary,measles)
hospital(mary)
sick(mary, D)
D measles
epidemic(measles) epidemic(D)
D measles
![Page 10: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/10.jpg)
Page 10
FOVE
P(hospital(mary)) = ?
sick(mary,measles)
hospital(mary)
sick(mary, D)
D measles
epidemic(D)
D measles
![Page 11: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/11.jpg)
Page 11
FOVE
hospital(mary)
sick(mary, D)
D measles
epidemic(D)
D measles
P(hospital(mary)) = ?
![Page 12: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/12.jpg)
Page 12
FOVE
P(hospital(mary)) = ?
hospital(mary)
sick(mary, D)
D measles
D measles
![Page 13: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/13.jpg)
Page 13
FOVE
P(hospital(mary)) = ?
hospital(mary)
![Page 14: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/14.jpg)
Page 14
e(D) D1D2 (e(D1),e(D2))
= e(D) (0,0)#(0,0) in assignment (0,1)#(0,1) in assignment
(1,0)#(1,0) in assignment
(1,1)#(1,1) in assignment
Let i be the number of e(D)’s assigned 1:
= i v1,v2 (v1,v2)#(v1,v2) given i
(number of assignments with |{D : e(D)=1}| = i)
Counting Elimination - A Combinatorial Approach
![Page 15: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/15.jpg)
Page 15
It does not work oneliminating class epidemic from(epidemic(D1, Region), epidemic(D2, Region), donations).
In general, counting elimination does not apply when atoms share logical variables.
Here, Region is shared between atoms.
Counting Elimination - Conditions
![Page 16: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/16.jpg)
Page 16
Partial Inversion
Provides a way of not sharing logical variables
e(D,R) D1D2,R e(D1,R), e(D2,R), d )
R e(D,r) D1D2 e(D1,r), e(D2,r), d )
(R is now bound, so not a variable anymore)
R ’d ) = ’d )|R| = ’’d )
![Page 17: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/17.jpg)
Page 17
Partial Inversion, graphically
epidemic(D2,r1)
epidemic(D1,r1)
D1 D2
donations
epidemic(D2,R)
epidemic(D1,R)
D1 D2 donations
epidemic(D2,r10)
epidemic(D1,r10)
D1 D2…
…
Each instance a counting
elimination problem
![Page 18: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/18.jpg)
Page 18
Another (not so partial) inversion
q(X,Y) X,Y (p(X),q(X,Y)) (expensive)
=X,Y q(X,Y) (p(X),q(X,Y)) (propositional)
= X,Y '(p(X))
= X 'Y(p(X))
= X ''(p(X)) (marginal on p(X))
![Page 19: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/19.jpg)
Page 19
Another (not so partial) inversion
…q(x1,y1)
p(x1)
q(xn,yn)
p(xn)…
q(X,Y)
p(X)Each instance a
propositional elimination
problem
![Page 20: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/20.jpg)
Page 20
Partial inversion conditions
friends(X,Y), friends(Y,X))Cannot partially invert on X,Y because friends(bob,mary) appears in more than one instance of parfactor.
friends(mary,bob)
friends(bob,mary)
friends(Y,X)
friends(X,Y)
friends(bob,mary)
…X Y
friends(mary,bob)
…
![Page 21: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/21.jpg)
Page 21
Summary of Partial Inversion
More general than previousInversion Elimination.
Generates Counting Elimination or Propositional sub-problems.
Cannot be applied to “entangled parfactors”.
Does not depend on domain size.
![Page 22: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/22.jpg)
Page 22
Second contribution: Lifted MPE
In propositional case,MPE done by factors containing MPE of eliminated variables.
A B
C
D
![Page 23: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/23.jpg)
Page 23
MPE
A B
D
B D MPE
0 0 0.3 C=1
0 1 0.2 C=1
1 0 0.5 C=0
1 1 0.9 C=1
In propositional case,MPE done by factors containing MPE of eliminated variables.
![Page 24: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/24.jpg)
Page 24
MPE
A B
B MPE
0 0.5 C=1,D=0
1 1.4 C=1,D=1
In propositional case,MPE done by factors containing MPE of eliminated variables.
![Page 25: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/25.jpg)
Page 25
MPE
A
A MPE(B,C,D)
0 0.9 B=0,C=1,D=0
1 0.7 B=1,C=1,D=1
In propositional case,MPE done by factors containing MPE of eliminated variables.
![Page 26: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/26.jpg)
Page 26
MPE
MPE
0.9 A=0,B=1,C=1,D=1
In propositional case,MPE done by factors containing MPE of eliminated variables.
![Page 27: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/27.jpg)
Page 27
MPE
Same idea in First-order case But factors are quantified and so are assignments:
p(X) q(X,Y) MPE
0 0 0.3 r(X,Y) = 1
0 1 0.2 r(X,Y) = 1
1 0 0.5 r(X,Y) = 0
1 1 0.9 r(X,Y) = 1
8 X, Y (p(X), q(X,Y))
![Page 28: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/28.jpg)
Page 28
MPE
After Inversion Elimination of q(X,Y):
p(X) q(X,Y) MPE
0 0 0.3 r(X,Y) = 1
0 1 0.9 r(X,Y) = 1
1 0 0.5 r(X,Y) = 0
1 1 0.3 r(X,Y) = 1
8 X, Y (p(X), q(X,Y))
p(X) ’ MPE
0 0.05 8 Y q(X,Y) = 1, r(X,Y) = 1
1 0.02 8 Y q(X,Y) = 0, r(X,Y) = 1
8 X ’(p(X))
Liftedassignment
s
![Page 29: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/29.jpg)
Page 29
MPE
After Inversion Elimination of p(X):
8 X ’(p(X))
’’ MPE
0.009 8 X 8 Y p(X) = 0, q(X,Y) = 1, r(X,Y) = 0
’’()
p(X) ’ MPE
0 0.05 8 Y q(X,Y) = 1, r(X,Y) = 1
1 0.02 8 Y q(X,Y) = 0, r(X,Y) = 1
![Page 30: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/30.jpg)
Page 30
MPE
After Counting Elimination of e:
e(D1) e(D2) MPE
0 0 0.3 r(D1,D2) = 1
0 1 0.9 r(D1,D2) = 1
1 0 0.5 r(D1,D2) = 0
1 1 0.3 r(D1,D2) = 1
8 D1, D2 (e(D1), e(D2))
’ MPE
0.05 938 D1,D2 e(D1)=0, e(D2) = 0, r(D1,D2) = 1
912 D1,D2 e(D1)=0, e(D2) = 1, r(D1,D2) = 1
915 D1,D2 e(D1)=1, e(D2) = 0, r(D1,D2) = 0
925 D1,D2 e(D1)=1, e(D2) = 1, r(D1,D2) = 1
’()
![Page 31: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/31.jpg)
Page 31
Conclusions
Partial Inversion:More general algorithm, subsumes Inversion elimination
Lifted Most Probable Explanation (MPE) same idea as in propositional VE, but with
Lifted assignments: describe sets of basic assignments universally quantified comes from Partial Inversion existentially quantified comes from Counting elimination
Ultimate goal: to perform lifted probabilistic inference in way similar to
logic inference: without grounding and at a higher level.
![Page 32: 1 MPE and Partial Inversion in Lifted Probabilistic Variable Elimination Rodrigo de Salvo Braz University of Illinois at Urbana-Champaign with Eyal Amir](https://reader036.vdocument.in/reader036/viewer/2022062511/55144e735503462d4e8b4fc9/html5/thumbnails/32.jpg)
Page 32