directedgraphicalmodels undirected graphical modelsmgormley/courses/10418/slides... · 2020. 5....
TRANSCRIPT
![Page 1: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/1.jpg)
Directed Graphical Models+
Undirected Graphical Models
1
10-418 / 10-618 Machine Learning for Structured Data
Matt GormleyLecture 7
Sep. 18, 2019
Machine Learning DepartmentSchool of Computer ScienceCarnegie Mellon University
![Page 2: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/2.jpg)
Q&A
2
Q: How will I earn the 5% participation points?
A: Very gradually. There will be a few aspects of the course (polls, surveys, meetings with the course staff) that we will attach participation points to.
That said, we might not actually use the whole 5% that is being held out.
![Page 3: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/3.jpg)
Q&A
3
Q: When should I prefer a directed graphical model to an undirected graphical model?
A: As we’ll see today, the primary differences between them are:
1. the conditional independence assumptions they define
2. the normalization assumptions they make (Bayes Nets are locally normalized)
(That said, we’ll also tie them together via a single framework: factor graphs.)
There are also some practical differences (e.g. ease of learning) that result from the locally vs. globally normalized difference.
![Page 4: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/4.jpg)
Reminders
• Homework 1: DAgger for seq2seq – Out: Thu, Sep. 12– Due: Thu, Sep. 26 at 11:59pm
4
![Page 5: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/5.jpg)
SUPERVISED LEARNING FOR BAYES NETS
5
![Page 6: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/6.jpg)
Recipe for Closed-form MLE1. Assume data was generated i.i.d. from some model
(i.e. write the generative story)x(i) ~ p(x|θ)
2. Write log-likelihoodl(θ) = log p(x(1)|θ) + … + log p(x(N)|θ)
3. Compute partial derivatives (i.e. gradient)!l(θ)/!θ1 = …!l(θ)/!θ2 = ……!l(θ)/!θM = …
4. Set derivatives to zero and solve for θ!l(θ)/!θm = 0 for all m ∈ {1, …, M}θMLE = solution to system of M equations and M variables
5. Compute the second derivative and check that l(θ) is concave down at θMLE
6
![Page 7: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/7.jpg)
Machine Learning
7
The data inspires the structures
we want to predict It also tells us
what to optimize
Our modeldefines a score
for each structure
Learning tunes the parameters of the
model
Inference finds {best structure, marginals,
partition function} for a new observation
Domain Knowledge
Mathematical Modeling
OptimizationCombinatorial Optimization
ML
(Inference is usually called as a subroutine
in learning)
![Page 8: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/8.jpg)
Machine Learning
8
DataModel
Learning
Inference
(Inference is usually called as a subroutine
in learning)
3
A
l
i
c
e
s
a
w
B
o
b
o
n
a
h
i
l
l
w
i
t
h
a
t
e
l
e
s
c
o
p
e
A
l
i
c
e
s
a
w
B
o
b
o
n
a
h
i
l
l
w
i
t
h
a
t
e
l
e
s
c
o
p
e
4
t
i
m
e
fl
i
e
s
l
i
k
e
a
n
a
r
r
o
w
t
i
m
e
fl
i
e
s
l
i
k
e
a
n
a
r
r
o
w
t
i
m
e
fl
i
e
s
l
i
k
e
a
n
a
r
r
o
w
t
i
m
e
fl
i
e
s
l
i
k
e
a
n
a
r
r
o
w
t
i
m
e
fl
i
e
s
l
i
k
e
a
n
a
r
r
o
w
2
Objective
X1
X3X2
X4 X5
![Page 9: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/9.jpg)
Learning Fully Observed BNs
9
X1
X3X2
X4 X5
p(X1, X2, X3, X4, X5) =
p(X5|X3)p(X4|X2, X3)
p(X3)p(X2|X1)p(X1)
![Page 10: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/10.jpg)
p(X1, X2, X3, X4, X5) =
p(X5|X3)p(X4|X2, X3)
p(X3)p(X2|X1)p(X1)
Learning Fully Observed BNs
10
X1
X3X2
X4 X5
![Page 11: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/11.jpg)
p(X1, X2, X3, X4, X5) =
p(X5|X3)p(X4|X2, X3)
p(X3)p(X2|X1)p(X1)
Learning Fully Observed BNs
How do we learn these conditional and marginal distributions for a Bayes Net?
11
X1
X3X2
X4 X5
![Page 12: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/12.jpg)
Learning Fully Observed BNs
12
X1
X3X2
X4 X5
p(X1, X2, X3, X4, X5) =
p(X5|X3)p(X4|X2, X3)
p(X3)p(X2|X1)p(X1)
X1
X2
X1
X3
X3X2
X4
X3
X5
Learning this fully observed Bayesian Network is equivalent to learning five (small / simple) independent networks from the same data
![Page 13: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/13.jpg)
Learning Fully Observed BNs
13
X1
X3X2
X4 X5
✓⇤= argmax
✓log p(X1, X2, X3, X4, X5)
= argmax
✓log p(X5|X3, ✓5) + log p(X4|X2, X3, ✓4)
+ log p(X3|✓3) + log p(X2|X1, ✓2)
+ log p(X1|✓1)
✓⇤1 = argmax
✓1
log p(X1|✓1)
✓⇤2 = argmax
✓2
log p(X2|X1, ✓2)
✓⇤3 = argmax
✓3
log p(X3|✓3)
✓⇤4 = argmax
✓4
log p(X4|X2, X3, ✓4)
✓⇤5 = argmax
✓5
log p(X5|X3, ✓5)
✓⇤= argmax
✓log p(X1, X2, X3, X4, X5)
= argmax
✓log p(X5|X3, ✓5) + log p(X4|X2, X3, ✓4)
+ log p(X3|✓3) + log p(X2|X1, ✓2)
+ log p(X1|✓1)
How do we learn these conditional and marginal
distributions for a Bayes Net?
![Page 14: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/14.jpg)
Learning Fully Observed BNs
14
![Page 15: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/15.jpg)
INFERENCE FOR BAYESIAN NETWORKS
16
![Page 16: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/16.jpg)
A Few Problems for Bayes NetsSuppose we already have the parameters of a Bayesian Network…
1. How do we compute the probability of a specific assignment to the variables?P(T=t, H=h, A=a, C=c)
2. How do we draw a sample from the joint distribution?t,h,a,c ∼ P(T, H, A, C)
3. How do we compute marginal probabilities?P(A) = …
4. How do we draw samples from a conditional distribution? t,h,a ∼ P(T, H, A | C = c)
5. How do we compute conditional marginal probabilities?P(H | C = c) = …
17
![Page 17: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/17.jpg)
GRAPHICAL MODELS:DETERMINING CONDITIONAL INDEPENDENCIES
![Page 18: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/18.jpg)
What Independencies does a Bayes Net Model?
• In order for a Bayesian network to model a probability distribution, the following must be true:
Each variable is conditionally independent of all its non-descendants in the graph given the value of all its parents.
• This follows from
• But what else does it imply?
P(X1…Xn ) = P(Xi | parents(Xi ))i=1
n
∏
= P(Xi | X1…Xi−1)i=1
n
∏
Slide from William Cohen
![Page 19: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/19.jpg)
Common Parent V-StructureCascade
What Independencies does a Bayes Net Model?
20
Three cases of interest…
Z
Y
X
Y
X Z
ZX
YY
![Page 20: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/20.jpg)
Common Parent V-StructureCascade
What Independencies does a Bayes Net Model?
21
Z
Y
X
Y
X Z
ZX
YY
X �� Z | Y X �� Z | Y X ��� Z | Y
Knowing Y decouples X and Z
Knowing Y couples X and Z
Three cases of interest…
![Page 21: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/21.jpg)
Whiteboard
(The other two cases can be shown just as easily.)
22
Common Parent
Y
X Z
X �� Z | Y
Proof of conditional independence
![Page 22: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/22.jpg)
The �Burglar Alarm� example
• Your house has a twitchy burglar alarm that is also sometimes triggered by earthquakes.
• Earth arguably doesn’t care whether your house is currently being burgled
• While you are on vacation, one of your neighbors calls and tells you your home’s burglar alarm is ringing. Uh oh!
Burglar Earthquake
Alarm
Phone Call
Slide from William Cohen
Quiz: True or False?
Burglar �� Earthquake | PhoneCall
![Page 23: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/23.jpg)
Markov Blanket (Directed)
25
Def: the Markov Blanket of a node in a directed graphical model is the set containing the node’s parents, children, and co-parents.
Def: the co-parents of a node are the parents of its children
X1
X4X3
X6 X7
X9
X12
X5
X2
X8
X10
X13
X11
![Page 24: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/24.jpg)
Markov Blanket (Directed)
26
Def: the Markov Blanket of a node in a directed graphical model is the set containing the node’s parents, children, and co-parents.
Def: the co-parents of a node are the parents of its children
X1
X4X3
X6 X7
X9
X12
X5
X2
X8
X10
X13
X11
Example: The Markov Blanket of X6 is {X3, X4, X5, X8, X9, X10}
ParentsChildren
ParentsCo-parents
ParentsParents
![Page 25: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/25.jpg)
Markov Blanket (Directed)
27
Def: the Markov Blanket of a node in a directed graphical model is the set containing the node’s parents, children, and co-parents.
Def: the co-parents of a node are the parents of its children
Theorem: a node is conditionally independent of every other node in the graph given its Markov blanket
X1
X4X3
X6 X7
X9
X12
X5
X2
X8
X10
X13
X11
Example: The Markov Blanket of X6 is {X3, X4, X5, X8, X9, X10}
ParentsChildren
ParentsCo-parents
ParentsParents
![Page 26: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/26.jpg)
D-Separation
Definition #1: Variables X and Z are d-separated given a set of evidence variables E iff every path from X to Z is “blocked”.
A path is “blocked” whenever:1. ∃Y on path s.t. Y ∈ E and Y is a “common parent”
2. ∃Y on path s.t. Y ∈ E and Y is in a “cascade”
3. ∃Y on path s.t. {Y, descendants(Y)} ∉ E and Y is in a “v-structure”
28
If variables X and Z are d-separated given a set of variables EThen X and Z are conditionally independent given the set E
YX Z… …
YX Z… …
YX Z… …
![Page 27: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/27.jpg)
D-Separation
Definition #2: Variables X and Z are d-separated given a set of evidence variables E iff there does not exist a path in the undirected ancestral moral graph with E removed.
1. Ancestral graph: keep only X, Z, E and their ancestors2. Moral graph: add undirected edge between all pairs of each node’s parents3. Undirected graph: convert all directed edges to undirected4. Givens Removed: delete any nodes in E
29
If variables X and Z are d-separated given a set of variables EThen X and Z are conditionally independent given the set E
⇒A and B connected⇒ not d-separated
A B
C
D E
F
Original:
A B
C
D E
Ancestral:
A B
C
D E
Moral:
A B
C
D E
Undirected:
A B
C
Givens Removed:
Example Query: A ⫫ B | {D, E}
![Page 28: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/28.jpg)
Learning ObjectivesBayesian Networks
You should be able to…1. Identify the conditional independence assumptions given by a
generative story or a specification of a joint distribution2. Draw a Bayesian network given a set of conditional
independence assumptions3. Define the joint distribution specified by a Bayesian network4. User domain knowledge to construct a (simple) Bayesian
network for a real-world modeling problem5. Depict familiar models as Bayesian networks6. Use d-separation to prove the existence of conditional
independencies in a Bayesian network7. Employ a Markov blanket to identify conditional independence
assumptions of a graphical model8. Develop a supervised learning algorithm for a Bayesian network
30
![Page 29: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/29.jpg)
TYPES OF GRAPHICAL MODELS
31
![Page 30: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/30.jpg)
Three Types of Graphical Models
32
X1 X1 X1
X1
X1 X1
X1 X1 X1
X1
X1 X1
X1 X1 X1
X1
X1 X1
Directed Graphical Model
Undirected Graphical Model
Factor Graph
![Page 31: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/31.jpg)
Key Concepts for Graphical Models
Graphical Models in General1. A graphical model defines a
family of probability distributions
2. That family shares in common a set of conditional independence assumptions
3. By choosing a parameterization of the graphical model, we obtain a single model from the family
4. The model may be either locally or globally normalized
Ex: Directed G.M.1. Family:
2. Conditional Independencies:
3. Example parameterization:
4. Normalization:
33
![Page 32: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/32.jpg)
Key Concepts for Graphical Models
Graphical Models in General1. A graphical model defines a
family of probability distributions
2. That family shares in common a set of conditional independence assumptions
3. By choosing a parameterization of the graphical model, we obtain a single model from the family
4. The model may be either locally or globally normalized
Ex: Undirected G.M.1. Family:
2. Conditional Independencies:
3. Example parameterization:
4. Normalization:
34
![Page 33: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/33.jpg)
Key Concepts for Graphical Models
Graphical Models in General1. A graphical model defines a
family of probability distributions
2. That family shares in common a set of conditional independence assumptions
3. By choosing a parameterization of the graphical model, we obtain a single model from the family
4. The model may be either locally or globally normalized
Ex: Factor Graph1. Family:
2. Conditional Independencies:
3. Example parameterization:
4. Normalization:
35
![Page 34: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/34.jpg)
UNDIRECTED GRAPHICAL MODELSMarkov Random Fields
36
![Page 35: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/35.jpg)
Undirected Graphical ModelsWhiteboard– Conditional independence assumptions for
undirected graphical model (graph separation)– Definition: clique– Definition: maximal clique– Cliques and potential functions– Non-negativity of potential functions– Definition of model family (i.e. joint distribution)– Global normalization and the partition function– Example: Binary Variables for MRF
37
![Page 36: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/36.jpg)
Markov Blanket (Directed)
38
Def: the Markov Blanket of a node in a directed graphical model is the set containing the node’s parents, children, and co-parents.
Def: the co-parents of a node are the parents of its children
Theorem: a node is conditionally independent of every other node in the graph given its Markov blanket
X1
X4X3
X6 X7
X9
X12
X5
X2
X8
X10
X13
X11
Example: The Markov Blanket of X6 is {X3, X4, X5, X8, X9, X10}
ParentsChildren
ParentsCo-parents
ParentsParents
![Page 37: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/37.jpg)
Markov Blanket (Undirected)
39
X1
X4X3
X6 X7
X9
X12
X5
X2
X8
X10
X13
X11
Example: The Markov Blanket of X6 is {X3, X4, X9, X10}
Def: the Markov Blanket of a node in an undirected graphical model is the set containing the node’s neighbors.
Theorem: a node is conditionally independent of every other node in the graph given its Markov blanket
![Page 38: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/38.jpg)
Non-equivalence of Directed / Undirected Graphical Models
There does not exist an undirected graphical model that can capture the conditional independence assumptions of this directedgraphical model:
There does not exist a directed graphical model that can capture the conditional independence assumptions of this undirected graphical model:
40
A C
B
D
A C
B
![Page 39: DirectedGraphicalModels Undirected Graphical Modelsmgormley/courses/10418/slides... · 2020. 5. 14. · Slide from William Cohen Quiz: True or False? Burglar Earthquake| PhoneCall](https://reader036.vdocument.in/reader036/viewer/2022062610/61173bd256576461b0766a60/html5/thumbnails/39.jpg)
Undirected Graphical Models
Whiteboard– Parameterization (e.g. tabular vs. log-linear)– Pairwise Markov Random Field (MRF)
41