lec 9: april 25th, 2006ee512 - graphical models - j. bilmespage 1 jeff a. bilmes university of...
Post on 20-Dec-2015
214 views
TRANSCRIPT
![Page 1: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/1.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 1
University of WashingtonDepartment of Electrical Engineering
EE512 Spring, 2006 Graphical Models
Jeff A. Bilmes <[email protected]>Jeff A. Bilmes <[email protected]>
Lecture 9 Slides
April 25th, 2006
![Page 2: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/2.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 2
• READING: – M. Jordan: Chapters 4,10,12,17,18
• Reminder: TA discussions and office hours:– Office hours: Thursdays 3:30-4:30, Sieg Ground Floor
Tutorial Center– Discussion Sections: Fridays 9:30-10:30, Sieg Ground Floor
Tutorial Center Lecture Room
• Reminder: take-home Midterm: May 5th-8th, you must work alone on this.
Announcements
![Page 3: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/3.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 3
• L1: Tues, 3/28: Overview, GMs, Intro BNs.• L2: Thur, 3/30: semantics of BNs + UGMs• L3: Tues, 4/4: elimination, probs, chordal I• L4: Thur, 4/6: chrdal, sep, decomp, elim• L5: Tue, 4/11: chdl/elim, mcs, triang, ci props.• L6: Thur, 4/13: MST,CI axioms, Markov prps.• L7: Tues, 4/18: Mobius, HC-thm, (F)=(G)• L8: Thur, 4/20: phylogenetic trees, HMMs• L9: Tue, 4/25: HMMs, inference on trees• L10: Thur, 4/27
• L11: Tues, 5/2• L12: Thur, 5/4• L13: Tues, 5/9• L14: Thur, 5/11• L15: Tue, 5/16• L16: Thur, 5/18• L17: Tues, 5/23• L18: Thur, 5/25• L19: Tue, 5/30• L20: Thur, 6/1: final presentations
Class Road Map
![Page 4: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/4.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 4
• L1: Tues, 3/28: • L2: Thur, 3/30:• L3: Tues, 4/4: • L4: Thur, 4/6:• L5: Tue, 4/11:• L6: Thur, 4/13:• L7: Tues, 4/18:• L8: Thur, 4/20: Team Lists, short abstracts I• L9: Tue, 4/25: today• L10: Thur, 4/27: short abstracts II
• L11: Tues, 5/2• L12: Thur, 5/4: abstract II + progress• L13: Tues, 5/9• L14: Thur, 5/11: 1 page progress report• L15: Tue, 5/16• L16: Thur, 5/18: 1 page progress report• L17: Tues, 5/23• L18: Thur, 5/25: 1 page progress report• L19: Tue, 5/30• L20: Thur, 6/1: final presentations
• L21: Tue, 6/6 4-page papers due (like a conference paper).
Final Project Milestone Due Dates
• Team lists, abstracts, and progress reports must be turned in, in class and using paper (dead tree versions only).
• Final reports must be turned in electronically in PDF (no other formats accepted).
• Progress reports must report who did what so far!!
![Page 5: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/5.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 5
• d-Separation, (DL), (DO), and equivalence of all Markov properties on BNs.
• Phylogenetic Trees and Chordal Models• Mixture Models• Hidden Markov Models (HMMs)• Forward () recursion and elimination
Summary of Last Time
![Page 6: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/6.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 6
• What queries to we want from an HMM?• Forward () recursion and elimination• Backwards () recursion and elimination• Why do we want these queries anyway?• More on inference in HMMs• Inference on chains• Start of inference on trees.
Outline of Today’s Lecture
![Page 7: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/7.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 7
Books and Sources for Today
• M. Jordan: Chapters 4,10,12,17,18• “What HMMs can do” handout on web.
![Page 8: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/8.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 8
HMMs and Bayesian Networks
![Page 9: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/9.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 9
Goals for HMMs
![Page 10: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/10.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 10
Learning HMMs with EM, what queries are used?
![Page 11: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/11.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 11
Learning HMMs with Gradient Descent, what probabilistic queries are needed?
![Page 12: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/12.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 12
Learning HMMs with Gradient Descent, what probabilistic queries are needed?
![Page 13: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/13.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 13
Learning HMMs with Gradient Descent, what probabilistic queries are needed?
![Page 14: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/14.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 14
HMMs, elimination orders, and forward recursion
![Page 15: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/15.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 15
HMMs, elimination orders, and forward recursion
![Page 16: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/16.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 16
What does the forward computation mean
![Page 17: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/17.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 17
HMMs, elimination orders, and backward recursion
![Page 18: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/18.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 18
HMMs, elimination orders, and backward recursion
![Page 19: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/19.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 19
What does the backward computation mean
![Page 20: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/20.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 20
Forward/Backward Elimination
![Page 21: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/21.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 21
Clique posteriors
![Page 22: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/22.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 22
Clique posteriors
![Page 23: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/23.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 23
HMM message passing
![Page 24: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/24.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 24
HMM message passing
1 2
3 4
1
2
3
4
![Page 25: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/25.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 25
HMM message passing
![Page 26: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/26.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 26
HMM message passing
![Page 27: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/27.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 27
Slightly different HMM junction tree
![Page 28: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/28.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 28
HMM message passing
1 2
3 4
1
2
3
4
![Page 29: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/29.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 29
HMM message passing
![Page 30: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/30.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 30
recursion
![Page 31: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/31.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 31
Towards More General Inference
![Page 32: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/32.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 32
Towards More General Inference
![Page 33: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/33.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 33
Towards More General Inference
![Page 34: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/34.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 34
Towards More General Inference
![Page 35: Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d485503460f94a2318f/html5/thumbnails/35.jpg)
Lec 9: April 25th, 2006 EE512 - Graphical Models - J. Bilmes Page 35
Inference in undirected trees