Bayes Nets & HMMs Tamara Berg CS 560 Artificial Intelligence Many slides throughout the course adapted from Svetlana Lazebnik, Dan Klein, Stuart Russell, Andrew Moore, Percy Liang, Luke Zettlemoyer, Rob Pless, Killian Weinberger, Deva Ramanan 1 Announcements HW3 due Oct 29, 11:59pm Today: Talk by Vijay Kumar (Aerial Robot Swarms), 4pm in SN011 Reminder: HW3 Example BN Nave Bayes C Class F - Features We only specify (parameters): prior over class labels how each feature depends on the class Nave Bayes Representation Goal: estimate likelihoods P(message | spam) and P(message | ham) and priors P(spam) and P(ham) Likelihood: bag of words representation The message is a sequence of words (w 1, , w n ) The order of the words in the message is not important Each word is conditionally independent of the others given message class (spam or not spam) Thus, the problem is reduced to estimating marginal likelihoods of individual words P(w i | spam) and P(w i | ham) along with the priors P(spam), P(ham) General MAP rule for Nave Bayes: Classify the message as spam if Decision rule likelihood priorposterior Classify the message as spam if Decision rule Example The quick brown fox jumped over the lazy dog How would you decide what class to predict? Why is this called a bag of words representation? Slide from Dan Klein Percentage of documents in training set labeled as spam/ham Slide from Dan Klein In the documents labeled as spam, occurrence percentage of each word (e.g. # times the occurred/# total words). Slide from Dan Klein In the documents labeled as ham, occurrence percentage of each word (e.g. # times the occurred/# total words). Slide from Dan Klein Parameter estimation Model parameters: feature likelihoods P(word | spam) and P(word | ham) and priors P(spam) and P(ham) How do we obtain the values of these parameters? Need training set of labeled samples from both classes P(word | spam) = # of word occurrences in spam messages total # of words in spam messages P(word | ham) = # of word occurrences in ham messages total # of words in ham messages Parameter estimation Parameter estimate: Parameter smoothing: dealing with words that were never seen or seen too few times Laplacian smoothing: pretend you have seen every vocabulary word m more times than you actually did P(word | spam) = # of word occurrences in spam messages total # of words in spam messages P(word | spam) = # of word occurrences in spam messages + m total # of words in spam messages + m*V (V: total number of unique words) Summary of model and parameters Nave Bayes model: Model parameters: P(spam) P(spam) P(w 1 | spam) P(w 2 | spam) P(w n | spam) P(w 1 | spam) P(w 2 | spam) P(w n | spam) Feature likelihoods given spam prior Feature likelihoods given spam Classification The class that maximizes: Classification In practice Multiplying lots of small probabilities can result in floating point underflow Since log(xy) = log(x) + log(y), we can sum log probabilities instead of multiplying probabilities. Since log is a monotonic function, the class with the highest score does not change. So, what we usually compute in practice is: Nave Bayes on images Visual Categorization with Bags of Keypoints Gabriella Csurka, Christopher R. Dance, Lixin Fan, Jutta Willamowski, Cdric Bray Bag of words for images Represent images as a bag of words Bag-of-word models for images Csurka et al. (2004), Willamowski et al. (2005), Grauman & Darrell (2005), Sivic et al. (2003, 2005) Bag-of-word models for images 1.Extract image features Bag-of-word models for images 1.Extract image features 2.Learn visual vocabulary Bag-of-word models for images 1.Extract image features 2.Learn visual vocabulary 3.Map image features to visual words Bag-of-word models for images Method Steps: Detect and describe image patches. Assign patch descriptors to a set of visual words. Apply nave Bayes, treating the images as a bag of visual words. Predict which category the image belongs to as the MAP estimate. Results Next: Adding time! 29 Reasoning over Time Often, we want to reason about a sequence of observations Speech recognition Robot localization Medical monitoring DNA sequence recognition Need to introduce time into our models Basic approach: hidden Markov models (HMMs) More general: dynamic Bayes nets 30 Reasoning over time Basic idea: Agent maintains a belief state representing probabilities over possible states of the world From the belief state and a transition model, the agent can predict how the world might evolve in the next time step From observations and an emission model, the agent can update the belief state Markov Model 32 Markov Models A Markov model is a chain-structured BN Value of X at a given time is called the state As a BN: . P(X t |X t-1 ).. Parameters: initial probabilities and transition probabilities or dynamics (specify how the state evolves over time) X2X2 X1X1 X3X3 X4X4 Conditional Independence Basic conditional independence: Past and future independent of the present Each time step only depends on the previous This is called the (first order) Markov property Note that the chain is just a (growing) BN We can always use generic BN reasoning on it if we truncate the chain at a fixed length X2X2 X1X1 X3X3 X4X4 34 Example: Markov Chain Weather: States: X = {rain, sun} Transitions: Initial distribution: 1.0 sun Whats the probability distribution after one step? rainsun X2X2 X1X1 X3X3 X4X4 Mini-Forward Algorithm Question: Whats P(X) on some day t? sun rain sun rain sun rain sun rain Forward simulation 36 Example From initial observation of sun From initial observation of rain P(X 1 )P(X 2 )P(X 3 )P(X ) P(X 1 )P(X 2 )P(X 3 )P(X ) 37 Stationary Distributions If we simulate the chain long enough: What happens? Uncertainty accumulates Eventually, we have no idea what the state is! Stationary distributions: For most chains, the distribution we end up in is independent of the initial distribution Called the stationary distribution of the chain Usually, can only predict a short time out Hidden Markov Model 39 Hidden Markov Models Markov chains not so useful for most agents Eventually you dont know anything anymore Need observations to update your beliefs Hidden Markov models (HMMs) Underlying Markov chain over states S You observe outputs (effects) at each time step As a Bayes net: X5X5 X2X2 E1E1 X1X1 X3X3 X4X4 E2E2 E3E3 E4E4 E5E5 Example An HMM is defined by: Initial distribution: Transitions: Emissions: Conditional Independence HMMs have two important independence properties: Markov hidden process, future depends on past via the present Current observation independent of all else given current state Quiz: does this mean that observations are independent given no evidence? [No, correlated by the hidden state] X5X5 X2X2 E1E1 X1X1 X3X3 X4X4 E2E2 E3E3 E4E4 E5E5 Real HMM Examples Speech recognition HMMs: Observations are acoustic signals (continuous valued) States are specific positions in specific words (so, tens of thousands) Machine translation HMMs: Observations are words (tens of thousands) States are translation options Robot tracking: Observations are range readings (continuous) States are positions on a map (continuous) Filtering / Monitoring Filtering, or monitoring, is the task of tracking the distribution B(X) (the belief state) over time We start with B(X) in an initial setting, usually uniform As time passes, or we get observations, we update B(X) The Kalman filter was invented in the 60s and first implemented as a method of trajectory estimation for the Apollo program Example: Robot Localization t=0 Sensor model: never more than 1 mistake Motion model: may not execute action with small prob. 10 Prob Example from Michael Pfeiffer Example: Robot Localization t=1 10 Prob Example: Robot Localization t=2 10 Prob Example: Robot Localization t=3 10 Prob Example: Robot Localization t=4 10 Prob Example: Robot Localization t=5 10 Prob Filtering Filtering update belief state given all evidence (observations) to date Recursive estimation - given the result of filtering up to time t, the agent computes the result for t+1 from the new evidence: Filtering consists of two parts: Projecting the current belief state distribution forward from t to t+1 Updating the belief using the new evidence Forward Algorithm Transition model Current belief state Emission model New belief state Normalizing constant used to make probabilities sum to 1. Decoding: Finding the most likely sequence Query: most likely seq: X5X5 X2X2 E1E1 X1X1 X3X3 X4X4 E2E2 E3E3 E4E4 E5E5 53 [video] Speech Recognition with HMMs Particle Filtering Sometimes |X| is too big to use exact inference |X| may be too big to even store B(X) E.g. X is continuous |X| 2 may be too big to do updates Solution: approximate inference Track samples of X, not all values Samples are called particles Time per step is linear in the number of samples But: number needed may be large In memory: list of particles, not states This is how robot localization works in practice Representation: Particles Our representation of P(X) is now a list of N particles (samples) P(x) approximated by number of particles with value x Initially, all particles have a weight of 1 64 Particles: (3,3) (2,3) (3,3) (3,2) (3,3) (3,2) (2,1) (3,3) (2,1) Particle Filtering: Elapse Time Each particle is moved by sampling its next position based on the transition model This captures the passage of time Particle Filtering: Observe Based on observations: Similar to likelihood weighting, we down weight our samples based on the evidence Note that, the probabilities wont sum to one, since most have been down weighted Particle Filtering: Resample Rather than tracking weighted samples, we resample N times, we choose from our weighted sample distribution (i.e. draw with replacement) This is equivalent to renormalizing the distribution Now the update is complete for this time step, continue with the next one Old Particles: (3,3) w=0.1 (2,1) w=0.9 (3,1) w=0.4 (3,2) w=0.3 (2,2) w=0.4 (1,1) w=0.4 (3,1) w=0.4 (2,1) w=0.9 (3,2) w=0.3 New Particles: (2,1) w=1 (3,2) w=1 (2,2) w=1 (2,1) w=1 (1,1) w=1 (3,1) w=1 (2,1) w=1 (1,1) w=1 Robot Localization In robot localization: We know the map, but not the robots position Observations may be vectors of range finder readings State space and readings are typically continuous (works basically like a very fine grid) and so we cannot store B(X) Particle filtering is a main technique [Demos,Fox]DemosFox SLAM SLAM = Simultaneous Localization And Mapping We do not know the map or our location Our belief state is over maps and positions! Main techniques: Kalman filtering (Gaussian HMMs) and particle methods DP-SLAM, Ron Parr Particle Filters for Localization and Video Tracking Exact Inference 71 The Forward Algorithm We are given evidence at each time and want to know We can derive the following updates We can normalize as we go if we want to have P(x|e) at each time step, or just once at the end Online Belief Updates Every time step, we start with current P(X | evidence) We update for time: We update for evidence: The forward algorithm does both at once (and doesnt normalize) Problem: space is |X| and time is |X| 2 per time step X2X2 X1X1 X2X2 E2E2 Best Explanation Queries Query: most likely seq: X5X5 X2X2 E1E1 X1X1 X3X3 X4X4 E2E2 E3E3 E4E4 E5E5 74 State Path Trellis State trellis: graph of states and transitions over time Each arc represents some transition Each arc has weight Each path is a sequence of states The product of weights on a path is the seqs probability Can think of the Forward algorithm as computing sums of all paths in this graph sun rain sun rain sun rain sun rain 75 Viterbi Algorithm sun rain sun rain sun rain sun rain 76 Example 77 Summary Filtering Push Belief through time Incorporate observations Inference: Forward Algorithm (/ Variable Elimination) Particle Filter (/ Sampling) Algorithms: Forward Algorithm (Belief given evidence) Viterbi Algorithm (Most likely sequence) 78 Inference Recap: Simple Cases E1E1 X1X1 X2X2 X1X1 Passage of Time Assume we have current belief P(X | evidence to date) Then, after one time step passes: Or, compactly: Basic idea: beliefs get pushed through the transitions With the B notation, we have to be careful about what time step t the belief is about, and what evidence it includes X2X2 X1X1 Observation Assume we have current belief P(X | previous evidence): Then: Or: Basic idea: beliefs reweighted by likelihood of evidence Unlike passage of time, we have to renormalize E1E1 X1X1 Example HMM The Forward Algorithm We are given evidence at each time and want to know We can derive the following updates We can normalize as we go if we want to have P(x|e) at each time step, or just once at the end Online Belief Updates Every time step, we start with current P(X | evidence) We update for time: We update for evidence: The forward algorithm does both at once Problem: space is |X| and time is |X| 2 per time step X2X2 X1X1 X2X2 E2E2 Exact Inference 85 Just like Variable Elimination 86 X2X2 e1e1 X1X1 X3X3 X4X4 e 2 e3e3 e4e4 XEP (E|X) rainumbrella0.9 rainno umbrella0.1 sunumbrella0.2 sunno umbrella0.8 X1X1 P( X 1 ) rain0.1 sun0.9 XP (e|X) rain0.9 sun0.2 evidence X1X1 P( X 1,e 1 ) rain0.09 sun0.18 Emission probability: Just like Variable Elimination 87 X2X2 X 1,e 1 X3X3 X4X4 e2e2 e3e3 e4e4 X1X1 P( X 1,e 1 ) rain0.09 sun0.18 XtXt X t+1 P (X t+1 |X t ) rain 0.9 rainsun0.1 sunrain0.2 sun 0.8 Transition probability: X1X1 X2X2 P (X 2,X 1,e 1 ) rain rainsun0.009 sunrain0.036 sun 0.144 Just like Variable Elimination 88 X 1,X 2,e 1 X3X3 X4X4 e2e2 e3e3 e4e4 Marginalize out X 1... X1X1 X2X2 P (X 2,X 1, e 1 ) rain rainsun0.009 sunrain0.036 sun X2X2 P( X 2,e 1 ) rain0.117 sun0.153 Just like Variable Elimination 89 X 2,e 1 X3X3 X4X4 e2e2 e3e3 e4e4 X2X2 P( X 2,e 1 ) rain0.117 sun0.153 XEP (E|X) rainumbrella0.9 rainno umbrella0.1 sunumbrella0.2 sunno umbrella0.8 Emission probability: Just like Variable Elimination 90 X 2,e 1, e 2 X3X3 X4X4 e3e3 e4e4 Approximate Inference 91 HMMs: MLE Queries HMMs defined by States X Observations E Initial distr: Transitions: Emissions: Query: most likely explanation: X5X5 X2X2 E1E1 X1X1 X3X3 X4X4 E2E2 E3E3 E4E4 E5E5 92