hybrid mode estimation and gaussian filtering with hybrid...

16.412 / 6.834 Lecture, 15 March 2004 1

Hybrid Mode Estimation andGaussian Filtering with Hybrid HMMs

Stanislav Funiak

16.412 / 6.834 Lecture, 15 March 2004 2Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models

References

Hofbaur, M. W., and Williams, B. C. (2002). Mode estimation of probabilistic hybrid systems. In: Hybrid Systems: Computation and Control, HSCC 2002.Funiak, S., and Williams, B. C. (2003). Multi-modal particle filtering for hybrid systems with autonomous mode transitions. In: DX-2003, SafeProcess 2003.Lerner, U., R. Parr, D. Koller and G. Biswas (2000). Bayesian fault detection and diagnosis in dynamic systems. In: Proc. of the 17th

National Conference on A. I.. pp. 531-537.V. Pavlovic. J. Rehg, T.-J. Cham, and K. Murphy. A Dynamic Bayesian Network Approach to Figure Tracking Using Learned Dynamic Models. In: Proc. ICCV, 1999.H.A.P. Blom and Y. Bar-Shalom. The interacting multiple model algorithm for systems with Markovian switching coefficients. IEEE Transactions on Automatic Control, 33, 1988.


Hybrid Models

Hidden Markov model

on

failed

off

p(x0)p(xt | xt-1)p(zt | xt)

Dynamic systems

?

Applications:- target tracking- localization and mapping- …

Applications:- topological localization


Outline

Applications: fault diagnosis, visual trackingSwitching linear Gaussian models + exact filteringProbabilistic Hybrid Automata + filteringApproximate Gaussian filtering with hybrid HMM models

fx(t1)|z(t1)(x|z1)

fx(t2)|z(t2)(x|z2)

fx(t2)|z(t1),z(t2)(x|z1,z2)

XZ1

x1

x2

Z1 Z2 Z1 Z2 X

σ

σ

σ

µ

Process model isxt = Axt-1 + But-1 + qt-1

Measurement model iszt = Hxt-1 + rt

Image adapted from Maybeck.


Scenario 1: Wheel monitoring for planetary rovers

Discrete variable: wheel failed (if any)Continuous variables: linear and angular velocity

Normal trajectory and trajectorieswith fault at each wheel

Courtesy NASA JPL Courtesy of Vandi Verma. Used with permission.


Scenario 2: Diagnosing subtle faults [H&W 2002]

Discrete variables: operational modeContinuous variables: CO2 flow, CO2 & O2 conc.

{closed, open, stuck-closed, stuck-open}

Courtesy NASA JSC


Scenario 3: Visual pose tracking Discrete variables: type of movementContinuous variables: head, legs, and torso position

Courtesy Pavlovic. J. Rehg, T.-J. Cham, and K. Murphy


Scenarios 1-3: Common properties

1. Continuous dynamics2. Finite set of behaviors, determines dynamics

Continuous state hiddenNoisy observationsUncertainty in the model

System may switchbetween behaviors

Need continuous statistical estimation

Need to track discrete changes

Need both


Outline



Hybrid models – Desired properties

State evolution:Stochastic continuous evolution (uncertain model)Gaussian noise (for KF)Probabilistic discrete transitionsContinuous observations, discrete and continuous actions

Interaction of discrete and continuous state:Discrete state affects continuous evolutionContinuous state affects discrete evolution

Large systems

1 2

uc1

ud1

ud2

wc1

3

yc2

yc1

vs1 vs3

vo1

vo2

A A

CA

A

vs2


Graphical models revisited

a1

b1

Z1

x1

b2

Z2

x2

),|( iij xaxp

Actions

Beliefs

Observations

States

ObservableHidden

)|( ii xzp

Model:

Transition distribution

Observation distribution)|( ii xzp

),|( iij xaxp


Review: Hidden Markov models

Discrete states, actions, and observationsTransition & observation p. written as tables

a1

b1

Z1

x1

b2

Z2

x2

),|( iij xaxp

Actions

Beliefs

Observations

States

Observable

Hidden

117

5

9

77MassAve

Belief update:


Review: Linear models, Kalman Filter

Continuous states, actions, and observationsLinear (linearized) process and measurement model

a1

b1

Z1

x1

b2

Z2

x2

),|( iij xaxp

Actions

Beliefs

Observations

States

Observable

Hidden

111 tttt qBuAxx

ttt rHxz

Belief update:


Switching Linear Systems (SLDS)

Discrete and continuous stateAlso known as jump Markov linear Gaussian model

a1

b1

Z1

x1

b2

Z2

x2

),|( 1 tt xaxp

Actions

Beliefs

Observations

Continuousstates

ObservableHidden

)|( tt xzp

d1 d2Discretestates

Discrete states (modes):

00

1

)Pr(

)|Pr(

d

dd tt

Continuous state:

)(

)(

)()(

000

11

111

dvx

rHxy

dq

udBxdAx

ttt

tt

ttttt

)|( 1 tt ddp


Example: Acrobatic robot tracking

ok

failed

0.005

0.995

1.0

µ

µ

T

2121 ,,,

noisetf

noisetf

noiset

noiset

kkkkyesk

kkkkyesk

kkk

kkk

),,,,(

),,,,(

,2,2,1,1,21,2

,2,2,1,1,11,1

,2,21,2

,1,11,1

noisetf

noisetf

noiset

noiset

kkkknok

kkkknok

kkk

kkk

),,,,(

),,,,(

,2,2,1,1,21,2

,2,2,1,1,11,1

,2,21,2

,1,11,1

)(okA

)( failedA

)(okB

)( failedB

)(okq

)( failedq

)|Pr( 1 tt dd

)01.0,0(,

0

0

1

0

NrH


Example cont’d

The actuator works until t=20, then breaks

ok ok ok ok

failed

)(okA)(okB)(okq

)( failedA)( failedB)( failedq

)(okA)(okB)(okq

)(okA)(okB)(okq

)(okA)(okB)(okq

t=20t=19t=18t=17

…

t=21


What questions?

Mode estimation:Given a1z1 … atzt, estimate dt

Application: fault diagnosis

Continuous state estimationGiven a1z1 … atzt, estimate xt

Application: tracking

Hybrid state estimationGiven a1z1 … atzt, estimate xt, dt


Exact filtering for SLDS

Main idea:Do estimation for each sequence of mode assignmentsSum up the assignments that end in the same modeEach Gaussian has an associated weight (probability)

ok

ok

failed

ok

failed

ok

failed

Maths:

)...|...,(

)...|,(

11...

1

11

11

ttdd

tt

tttt

zazaddxp

zazadxp

t

)...|...(

)...,...|(

)...|...,(

111

111

111

ttt

tttt

tttt

zazaddp

zazaddxp

zazaddxp

Continuous tracking

Probability of a mode sequence


Continuous tracking

Back to our example:

ok ok ok ok

failed)(okA)(okB)(okq

)( failedA)( failedB)( failedq

)(okA)(okB)(okq

)(okA)(okB)(okq

)(okA)(okB)(okq

t=19t=18t=17

…

Know:1. Model A,B each each t2. Observations3. Actions

Can do Kalman filteringas before

)()( ,ˆ it

it Cx

Sequence (i)

Kalman filter: ),ˆ()...,...|( )()(111

it

ittttt CxNzazaddxp

)(ˆ itx

)(itC

)(1ˆ i

tx

t=20


Probability of a mode sequence (1/2)

Challenge: Can we update efficiently?)...|...( 111 ttt zazaddp

a1

b1

Z1

x1

b2

Z2

x2

),|( 1 tt xaxp

Actions

Beliefs

Observations

Continuousstates

ObservableHidde

n

)|( tt xzp

d1 d2Discretestates )|( 1 tt ddp

ok ok ok ok

failed

)...|......( 111111 ttt zazaokokddp

)...|......( 111 ttt zazafailedokokddp

1. Prediction:

)...,...|(

)...|...(

)...|...(

111111

111111

11111

tttt

ttt

ttt

zazadddp

zazaddp

zazaddp

)|()...|...( 1111111 ttttt ddpzazaddp

conditionalprobability

independence

discrete transition probability


Probability of a mode sequence (2/2)

Challenge: Can we incorporate the latest observation?

ok ok ok ok

failed

)...|......( 111 ttt zazafailedokokddp2. Update:Bayes rule + Kalman Filter

)...|......( 11111 ttt zazafailedokokddp

)...|...( 11111 ttttt zazazaddp

Observation likelihoodgiven the mode sequence Prediction

),ˆ()...,...|( )()(11111

iittttt CxNzazaddxp1.

Sequence (i)

2. ),ˆ()...,...|( )()(11111 RHHCxHNzazaddzp Tii

ttttt

S

TrrSeS

15.02/1|2|

1

constzazaddpzazaddzp ttttttt )...|...()...,...|( 1111111111


Outline



Probabilistic Hybrid AutomataSLDS + nonlinear dynamics, guarded transitionsHofbaur & Williams 2002

has-ball

true

false

7.01

0.1

0.9

7.01

0.5

7.010.5

7.01

1.0

0.5

0.5

2121 ,,,

Continuous dynamics:

noisetf

noisetf

noiset

noiset

kkkkyesk

kkkkyesk

kkk

kkk

),,,,(

),,,,(

,2,2,1,1,21,2

,2,2,1,1,11,1

,2,21,2

,1,11,1

noisetf

noisetf

noiset

noiset

kkkknok

kkkknok

kkk

kkk

),,,,(

),,,,(

,2,2,1,1,21,2

,2,2,1,1,11,1

,2,21,2

,1,11,1

µ

µ

T

ball:

a1

b1

Z1

x1

b2

Z2

x2

),|( 1 tt xaxp

Actions

Beliefs

Observations

Continuousstates

ObservableHidd

en

)|( tt xzp

d1 d2Discretestates

Discrete transition now dependson the continuous state

),|( 1 ttt xddp (also, the observation function g depends on discrete state)


Discrete prediction in PHA

Main idea:compute transition probability from continuous estimate

constant for all xt-1 thatsatisfy a given guard

)(1

)(1,ˆ i

ti

t Cx

true true true true

false

)...,...|()...|...(

)...|...(

111111111111

11111

ttttttt

ttt

zazadddpzazaddp

zazaddp

)...|...( 111111 ttt zazaddp

)...|...( 111111 tttt zazadddp

O

def

tttttX ttt Pdxzazaddxpxddp 1111111111 )...,...|(),|(

Cannot simplify as easily as before

Instead:

= p(transition ) p(guard for satisfied | previous estimate)


Example

0.7 1meanof estimated 1 guard boundary

probabilityof guard c

7.01

7.01

7.01

7.01

)(1

)(1,ˆ i

ti

t Cx

P(true -> false | estimate i) = 0.1* + 0.5*


Concurrent Probabilistic Hybrid Automata

PHA + composition

2121 ,,,

µ

µ

T

ball:

Gaussian filtering:Merge the difference equations in component models

e.g., equation solver

Track sequences of full mode assignments as beforeCompute the transition probability component-wise


Putting it all together

Estimate the continuous state for each mode sequenceUpdate the posterior probability of each mode sequence1. prediction (transition expansion)2. observation

Hybrid update equationsHofbaur & Williams 2002

Po

Pt

prediction observation

)(1

)(111 ,ˆ),...( i

ti

tt CxddbOld estimate: New estimate:

)()(1 ,ˆ),...( i

ti

tt Cxddb

ottt PPddbddb )...()...( 111


Outline



Gaussian filtering for hybrid HMMs

Tracks mode sequences with a bank of Kalman Filters

Problem: the number of possible mode sequences increases exponentiallyin time(also exponentially in # components)Exponential computational complexity

2 strategies:1. pruning (truncating)2. collapsing (merging)


Pruning: Selecting relevant mode sequences

Two methodsK-best filtering

Looking for a set of leading sequencesObtained efficiently with A* search

Particle filteringSelecting trajectories probabilistically by samplingRao-Blackwellised particle filtering

x2x2

x2

x2

x2

x2

x2

x1

x1x0

x2

x2(1)

(2)

(3) x2(3)

(1)

(2)

x1(3)

(1)

(4)

(5)

(6)

(7)

(8)

(9)


K-best Filtering

A* search through space of possible successors [H&W 2002]Evaluated in the order of

)()()( nhngnf

Probability (cost) oftrajectory so far

Admissible heuristics:upperbound on probabilityfrom nv onward


Rao-Blackwellised Particle Filtering

Principle: Decrease the computational complexity by reducing dimensionality of sampled space

Sample variables rClosed-form solution for variables s

r

s

r

)|( )(:0itrp s

s

component 1component 2

PT2

nv

PT1PT1 Po

h(k)h(k-1)(k-1)

transition expansion estimation

component l

xx(k)


RBPF for Hybrid HMMs

mok,0

(1) Initialization step

(2) Importance sampling step

(3) Selection (resampling) step

mok,0 mok,1mok,1

mok,0mok,1 mf,0 mf,1

(4) Exact (Kalman Filtering) step

draw samples from the initial distribution over the modes

initialize the corresponding continuous state estimates

evolve each sample trajectory according to the transition model and previous continuous estimates

determine transition & observ-ation model for each sample

update continuous estimates

mok,0mok,0

mok,0mok,1

mok,1mok,1

mok,1mf,1

mok,1mok,1

mok,0mok,0

mok,0mok,1

mok,1mok,1

Funiak & Williams 2003


Do we really need hybrid?

Alternatives:HMM, grid-based methods: Course discretizationineffective for tracking a dynamic system! Kalman Filter: Unimodal distribution too weakParticle filter: Sample size too large


What you should know

The form of hybrid modelsThe “exact” algorithm for computing the belief state over mode sequences + continuous estimates

And how to use it for common diagnostic tasks

The strategies for pruning mode sequences(conceptually)

hybrid mode estimation and gaussian filtering with hybrid...

Documents