ece 6504: advanced topics in machine learnings14ece6504/slides/l3_bn_semantics.pptx.pdfece 6504:...
TRANSCRIPT
ECE 6504: Advanced Topics in Machine Learning
Probabilistic Graphical Models and Large-Scale Learning
Dhruv Batra Virginia Tech
Topics: – Bayes Nets: Representation/Semantics
– v-structures – Probabilistic influence, Active Trails
Readings: Barber 3.3; KF 3.3.1-3.3.2
Plan for today • Notation Clarification
• Errata #1: Number of parameters in disease network
• Errata #2: Car start v-structure example
• Bayesian Networks – Probabilistic influence & active trails – d-separation – General (conditional) independence assumptions in a BN
(C) Dhruv Batra 2
A general Bayes net • Set of random variables
• Directed acyclic graph – Encodes independence assumptions
• CPTs – Conditional Probability Tables
• Joint distribution:
(C) Dhruv Batra 3
Factorized distributions • Given
– Random vars X1,…,Xn
– P distribution over vars – BN structure G over same vars
• P factorizes according to G if
Flu Allergy
Sinus
Headache Nose
(C) Dhruv Batra 4 Slide Credit: Carlos Guestrin
How many parameters in a BN? • Discrete variables X1, …, Xn
• Graph – Defines parents of Xi, PaXi
• CPTs – P(Xi| PaXi)
(C) Dhruv Batra 5
Independencies in Problem
BN:
Graph G encodes local
independence
assumptions
World, Data, reality:
True distribution P contains independence
assertions
(C) Dhruv Batra 6 Slide Credit: Carlos Guestrin
Bayes Nets • BN encode (conditional) independence assumptions.
– I(G) = {X indep of Y given Z}
• Which ones? • And how can we easily read them?
(C) Dhruv Batra 7
Local Structures • What’s the smallest Bayes Net?
(C) Dhruv Batra 8
Local Structures
(C) Dhruv Batra 9
Z
Y X
Z Y X
Z Y X
Z Y X
Indirect causal effect:
Indirect evidential effect:
Common cause:
Common effect:
Car starts BN
• 18 binary attributes
• Inference – P(BatteryAge|Starts=f)
• 218 terms, why so fast?
(C) Dhruv Batra 10 Slide Credit: Carlos Guestrin
Bayes Ball Rules • Flow of information
– on board
(C) Dhruv Batra 11
Active trails formalized • Let variables O ⊆ {X1,…,Xn} be observed
• A path X1 – X2 – · · · –Xk is an active trail if for each consecutive triplet:
– Xi-1→Xi→Xi+1, and Xi is not observed (Xi∉O)
– Xi-1←Xi←Xi+1, and Xi is not observed (Xi∉O)
– Xi-1←Xi→Xi+1, and Xi is not observed (Xi∉O)
– Xi-1→Xi←Xi+1, and Xi is observed (Xi∈O), or one of its descendents is observed
Slide Credit: Carlos Guestrin (C) Dhruv Batra 12
Active trails and Independence
• Theorem: Variables Xi and Xj are independent given Z if – no active trail between Xi and Xj
when variables Z⊆{X1,…,Xn} are observed
A
H
C E
G
D
B
F
K
J
I
(C) Dhruv Batra 13 Slide Credit: Carlos Guestrin
Naïve Bayes:
Slide Credit: Erik Sudderth
Name That Model
(C) Dhruv Batra 14
Slide Credit: Erik Sudderth
Name That Model
Tree-Augmented Naïve Bayes (TAN)
(C) Dhruv Batra 15
Name That Model
Hidden Markov Model (HMM)
Y1 = {a,…z}
X1 =
Y5 = {a,…z} Y3 = {a,…z} Y4 = {a,…z} Y2 = {a,…z}
X2 = X3 = X4 = X5 =
Figure Credit: Carlos Guestrin (C) Dhruv Batra 16