review of bayes’ rule
DESCRIPTION
BIOL 301 Guest Lecture: Reasoning Under Uncertainty (Intro to Bayes Networks) Simon D. Levy CSCI Department 8 April 2010. Rev. Thomas Bayes (1702-1761). Review of Bayes’ Rule. From the Product Rule:. P( A | B ) = P( A & B ) / P(B). P( A & B ) = P( A | B ) * P( B ). - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Review of Bayes’ Rule](https://reader035.vdocument.in/reader035/viewer/2022062500/568159a6550346895dc70760/html5/thumbnails/1.jpg)
BIOL 301 Guest Lecture: Reasoning Under Uncertainty
(Intro to Bayes Networks)
Simon D. LevyCSCI Department
8 April 2010
![Page 2: Review of Bayes’ Rule](https://reader035.vdocument.in/reader035/viewer/2022062500/568159a6550346895dc70760/html5/thumbnails/2.jpg)
Review of Bayes’ Rule
• From the Product Rule:
• P(A|B) = P(A & B) / P(B)• P(A &B) = P(A|B) * P(B) = P(B|A) * P(A)
Rev. Thomas Bayes (1702-1761)• We derive Bayes’ Rule by
substitution:• P(A|B) = P(A & B) / P(B) = P(B|A) * P(A) / P(B)
![Page 3: Review of Bayes’ Rule](https://reader035.vdocument.in/reader035/viewer/2022062500/568159a6550346895dc70760/html5/thumbnails/3.jpg)
Real-World Problems May Involve Many Variables
http://www.bayesia.com/assets/images/content/produits/blab/tutoriel/en/chapitre-3/image026.jpg
![Page 4: Review of Bayes’ Rule](https://reader035.vdocument.in/reader035/viewer/2022062500/568159a6550346895dc70760/html5/thumbnails/4.jpg)
Real-World Problems May Involve Many Variables
![Page 5: Review of Bayes’ Rule](https://reader035.vdocument.in/reader035/viewer/2022062500/568159a6550346895dc70760/html5/thumbnails/5.jpg)
Variables Are TypicallyOberserved Simultaneously
(Confounded)Fever Ache Virus PNo No No .950No No Yes .002No Yes No .032No Yes Yes .002Yes No No .002Yes No Yes .001Yes Yes No .010Yes Yes Yes .001
So how do we compute P(V=Yes), P(F=Yes & A=No), etc.?
![Page 6: Review of Bayes’ Rule](https://reader035.vdocument.in/reader035/viewer/2022062500/568159a6550346895dc70760/html5/thumbnails/6.jpg)
Marginalization
Fever Ache Virus PNo No No .950No No Yes .002No Yes No .032No Yes Yes .002Yes No No .002Yes No Yes .001Yes Yes No .010Yes Yes Yes .001 ______ Sum = .006
P(V=Yes):
![Page 7: Review of Bayes’ Rule](https://reader035.vdocument.in/reader035/viewer/2022062500/568159a6550346895dc70760/html5/thumbnails/7.jpg)
Marginalization
Fever Ache Virus PNo No No .950No No Yes .002No Yes No .032No Yes Yes .002Yes No No .002Yes No Yes .001Yes Yes No .010Yes Yes Yes .001 ______ Sum = .003
P(F=Yes & A=No):
![Page 8: Review of Bayes’ Rule](https://reader035.vdocument.in/reader035/viewer/2022062500/568159a6550346895dc70760/html5/thumbnails/8.jpg)
Combinatorial Explosion(The “Curse of Dimensionality”)
Assuming (unrealistically) only two values (Yes/No) per variable:
# Variables # of Rows in Table1 22 43 84 165 326 64: :20 1,048,576
![Page 9: Review of Bayes’ Rule](https://reader035.vdocument.in/reader035/viewer/2022062500/568159a6550346895dc70760/html5/thumbnails/9.jpg)
Solution: Local Causality + Belief Propagation
![Page 10: Review of Bayes’ Rule](https://reader035.vdocument.in/reader035/viewer/2022062500/568159a6550346895dc70760/html5/thumbnails/10.jpg)
Local Causality
![Page 11: Review of Bayes’ Rule](https://reader035.vdocument.in/reader035/viewer/2022062500/568159a6550346895dc70760/html5/thumbnails/11.jpg)
Recover Joint From Prior & Posterior
B E P(A)T T .95T F .94F T .29F F .001
P(B).001
P(E).002
B E A ProbT T T .001*.002*.95 = .000001900T T F .001*.002*.05 = .000000100T F T .001*.998*.94 = .000938120T F F .001*.998*.06 = .000059880F T T .999*.002*.29 = .000579420F T F .999*.002*.71 = .001418580F F T .999*.998*.001 = .000997002F F F .999*.998*.999 = .996004998
![Page 12: Review of Bayes’ Rule](https://reader035.vdocument.in/reader035/viewer/2022062500/568159a6550346895dc70760/html5/thumbnails/12.jpg)
Belief Propagation
• Consider just B → A → J
• P(J=T | B=T) = P(J=T | A=T) * P(A=T | B=T) • Then use Bayes’ Rule and marginalization to answer more sophisticated queries like
P(B=T | J=F & E=F & M=T)
![Page 13: Review of Bayes’ Rule](https://reader035.vdocument.in/reader035/viewer/2022062500/568159a6550346895dc70760/html5/thumbnails/13.jpg)
Multiply-Connected Networks
Wet Grass
Cloudy
Rain Sprinkler
![Page 14: Review of Bayes’ Rule](https://reader035.vdocument.in/reader035/viewer/2022062500/568159a6550346895dc70760/html5/thumbnails/14.jpg)
Clustering (“Mega Nodes”)
Cloudy Sprinkler Rain
Sprinkler Rain Wet Grass
Sprinkler Rain
![Page 15: Review of Bayes’ Rule](https://reader035.vdocument.in/reader035/viewer/2022062500/568159a6550346895dc70760/html5/thumbnails/15.jpg)
A
B
D
F
E
C G
H
Junction Tree Algorithm (Huang & Darwiche 1994)
![Page 16: Review of Bayes’ Rule](https://reader035.vdocument.in/reader035/viewer/2022062500/568159a6550346895dc70760/html5/thumbnails/16.jpg)
A
B
D
F
E
C G
H
A
B
D
F
E
C G
H
Junction Tree Algorithm (Huang & Darwiche 1994)
“Moralize””
![Page 17: Review of Bayes’ Rule](https://reader035.vdocument.in/reader035/viewer/2022062500/568159a6550346895dc70760/html5/thumbnails/17.jpg)
A
B
D
F
E
C G
H
Junction Tree Algorithm (Huang & Darwiche 1994)
![Page 18: Review of Bayes’ Rule](https://reader035.vdocument.in/reader035/viewer/2022062500/568159a6550346895dc70760/html5/thumbnails/18.jpg)
A
B
D
F
E
C G
H
Junction Tree Algorithm (Huang & Darwiche 1994)
A
B
D
F
E
C G
HTriangulate
![Page 19: Review of Bayes’ Rule](https://reader035.vdocument.in/reader035/viewer/2022062500/568159a6550346895dc70760/html5/thumbnails/19.jpg)
Junction Tree Algorithm (Huang & Darwiche 1994)
A
B
D
F
E
C G
H
![Page 20: Review of Bayes’ Rule](https://reader035.vdocument.in/reader035/viewer/2022062500/568159a6550346895dc70760/html5/thumbnails/20.jpg)
Junction Tree Algorithm (Huang & Darwiche 1994)
A
B
D
F
E
C G
H
ABD ADE
DEF
ACE CEG
EGH
AD
DE
AE CE
EG
![Page 21: Review of Bayes’ Rule](https://reader035.vdocument.in/reader035/viewer/2022062500/568159a6550346895dc70760/html5/thumbnails/21.jpg)
“Message-Passing”
ABD ADE
DEF
ACE CEG
EGH
AD
DE
AE CE
EG
Observe A=T
![Page 22: Review of Bayes’ Rule](https://reader035.vdocument.in/reader035/viewer/2022062500/568159a6550346895dc70760/html5/thumbnails/22.jpg)
“Message-Passing”
ABD ADE
DEF
ACE CEG
EGH
AD
DE
AE CE
EG
Pick a cluster containing A:
![Page 23: Review of Bayes’ Rule](https://reader035.vdocument.in/reader035/viewer/2022062500/568159a6550346895dc70760/html5/thumbnails/23.jpg)
“Message-Passing”
ABD ADE
DEF
ACE CEG
EGH
AD
DE
AE CE
EG
Pass messages to propagate evidence: