Download - Graphical Models 4dummies
![Page 1: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/1.jpg)
Graphical Models for dummies
Max Khesin, Data Strategist, Liquidnet Inc.
![Page 2: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/2.jpg)
Graphical Models ForBy Dummies
![Page 3: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/3.jpg)
Grand Theme
• “Probabilistic graphical models are an elegant framework which combines uncertainty (probabilities) and logical structure (independence constraints) to compactlyrepresent complex, real-world phenomena”. (Koller 2007)
![Page 4: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/4.jpg)
Trying to guess if family is home.
• When wife leaves the house she leaves the outdoor light on (but sometimes leaves it on for a guest)
• When wife leaves the house, she usually puts the dog out
• When dog has a bowel problem, she goes to the backyard
• If the dog is in the backyard, I will probably hear it (but it might be the neighbor's dog)
![Page 5: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/5.jpg)
![Page 6: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/6.jpg)
These causal connections are not absolute
Three causes of uncertainty (Norvig, Russell 2009):
- Laziness
- Theoretical Ignorance
- Practical Ignorance
![Page 7: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/7.jpg)
Problem with probability
• Too many parameters
• For binary random variables, 2^n-1
![Page 8: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/8.jpg)
Bayesean Network - definition
• A Bayesian network is a directed graph in which each node is annotated with quantitative probability information. The full specification is as follows:
1. Each node corresponds to a random variable, which may be discrete or continuous.
2. A set of directed links or arrows connects pairs of nodes. If there is an arrow from node X to node Y , X is said to be a parent of Y. The graph has no directed cycles (and hence is a directed acyclic graph, or DAG.
3. Each node Xi has a conditional probability distribution P(Xi | Parents(Xi)) that quantifies the effect of the parents on the node. (Russell, Norvig 2009)
![Page 9: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/9.jpg)
![Page 10: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/10.jpg)
Compressed distribution (factorization)
• In our Bayesian system, we only have 10 parameters
• The compression is due to independence
• Independence is how causality manifests itself in the distribution
![Page 11: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/11.jpg)
D-connection
![Page 12: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/12.jpg)
Definition: conditional probability
![Page 13: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/13.jpg)
Decomposing a joint distribution
…
![Page 14: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/14.jpg)
Topologic sort:
• “topological ordering of a directed acyclic graph (DAG) is a linear ordering of its nodes in which each node comes before all nodes to which it has outbound edges” - Wikipedia.
![Page 15: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/15.jpg)
Sorting the family-home example
• Family-out(1), Bowel-problem(2), Ligths-On(3), Dog-out(4), Hear-bark(4)
• Bowel-problem(1), Family-out(2), Ligths-On(3), Dog-out(4), Hear-bark(5)
• Right away, we got all the non-descendants to the left of the variable
![Page 16: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/16.jpg)
Parents only matter
![Page 17: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/17.jpg)
Chain rule for Bayesian Networks
![Page 18: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/18.jpg)
![Page 19: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/19.jpg)
![Page 20: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/20.jpg)
Markov Networks
• These models are useful in modeling a variety of phenomena where one cannot naturally ascribe a directionality to the interaction between variables
![Page 21: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/21.jpg)
Markov Networks
![Page 22: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/22.jpg)
Markov Networks
• D is a set of random variables
• Factor to be a function from Val(D) to IR+.
![Page 23: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/23.jpg)
Markov Networks
• H is a Markov network structure
• set of subsets D1, . . . ,Dm, where each Di is a complete subgraph of H
• factors
![Page 24: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/24.jpg)
Markov Network - factorization
Where the unnormalized measure is
And normalization factor is
![Page 25: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/25.jpg)
Factor Product (pointwise multiplication)
![Page 26: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/26.jpg)
![Page 27: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/27.jpg)
Decision Networks
- Combine Bayesian Networks with Utility Theory
![Page 28: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/28.jpg)
Utility-based agent
![Page 29: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/29.jpg)
Decision network
![Page 30: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/30.jpg)
Evaluating decision network
![Page 31: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/31.jpg)
Applications – Bayes Nets
• Expert systems. • “…A later evaluation showed that the diagnostic accuracy of
Pathfinder IV was at least as good as that of the expert used to design the system. When used with less expert pathologists, the system significantly improved the diagnostic accuracy of the physicians alone. Moreover, the system showed greater ability to identify important findings and to integrate these findings into a correct diagnosis. Unfortunately, multiple reasons prevent the widespread adoption of Bayesian networks as anaid for medical diagnosis, including legal liability issues for misdiagnoses and incompatibility with the physicians' workflow” (Koller 2009)
![Page 32: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/32.jpg)
Applications – Markov Networks
• Computer vision – segmentation
• Regions are contiguous. Glove is next to the arm
“superpixels”
![Page 33: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/33.jpg)
Application – Markov Nets (combining logic and probability)
)()(),(,
)()(
ySmokesxSmokesyxFriendsyx
xCancerxSmokesx
1.1
5.1
Cancer(A)
Smokes(A)Friends(A,A)
Friends(B,A)
Smokes(B)
Friends(A,B)
Cancer(B)
Friends(B,B)
Two constants: Anna (A) and Bob (B)
![Page 34: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/34.jpg)
Road to AGI…
![Page 35: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/35.jpg)
Tools
• Netica - http://www.norsys.com/netica.html
• Really, look here http://www.cs.ubc.ca/~murphyk/Bayes/bnsoft.html
![Page 36: Graphical Models 4dummies](https://reader033.vdocument.in/reader033/viewer/2022051322/5457923baf795998788b517a/html5/thumbnails/36.jpg)
References
• Russell, Norvig 2009: Artificial Intelligence: A Modern Approach (AIMA) http://aima.cs.berkeley.edu/ (amazon)
• Getoor, Taskar 2007: Introduction to Statistical Relational Learning http://www.cs.umd.edu/srl-book/ (amazon)
• Koller, Friedman 2009: Probabilistic Graphical Models: Principles and Techniques http://pgm.stanford.edu/ (amazon)
• Charniak 1991: Bayesian Networks without Tears. www.cs.ubc.ca/~murphyk/Bayes/Charniak_91.pdf
• CS228: http://www.stanford.edu/class/cs228/ (course available via SCPD)
• Domingos, practical statistical learning in AI http://www.cs.cmu.edu/~tom/10601_sp08/slides/mlns-april-28.ppt, see also http://www.youtube.com/watch?v=bW5DzNZgGxY
• Koller 2007: “Graphical Models in a Nutshell”, a chapter of Getoor, Taskar 2007, availavle online http://www.seas.upenn.edu/~taskar/pubs/gms-srl07.pdf