a friendly approach to particle filters in computer vision
DESCRIPTION
This is a friendly approach to particle filters. Some hints, examples, and good practices to be able to successfully apply particle filters to solve your computer vision proTRANSCRIPT
1
A FRIENDLY APPROACH TO
PARTICLE FILTERS IN
COMPUTER VISION
Concepts, hints and examples
12/2/2011
Dr.-Ing. Marcos Nieto DoncelInvestigador/Researcher
2
Outline
Motivation
Bayesian framework
Sampling solution
Examples
2Marcos Nieto, PhD - [email protected]/2/2011
3
Motivation
For what?
Obtain estimates of a recursive/dynamic system
W
H
(x0,y0)
Let’s stay in computer vision
applications
3Marcos Nieto, PhD - [email protected]/2/2011
4
Motivation
Why?
Deterministicapproach vs
Probabilisticapproach
4Marcos Nieto, PhD - [email protected]/2/2011
5
Motivation
How?
1. Define your target
2. Define your functions
3. Select a type of filter adapted to 1) and 2)
4. Implement and run
5. Optionally: Write your paper and share : )
5Marcos Nieto, PhD - [email protected]/2/2011
6
Bayesian filtering
Target: xk
It evolves through time according to somedynamics, properties, interaction, etc.
W
H
(x0,y0)
W
H
x0y0
p(xk|xk-1)
Prior / Dynamics / Transition…
6Marcos Nieto, PhD - [email protected]/2/2011
7
Bayesian filtering
Observations: z1:k
Noisy, distorted, indirect
Typically, different dimensionaliy
p(zk|xk)
Likelihood / Observation model / Measurements…
7Marcos Nieto, PhD - [email protected]/2/2011
8
Bayesian filtering
Posterior distribution: p(xk|z1:k)
Probability density function
This is all you can expect to know
Typically we want a point-estimate of thisdistribution At each time instant: x*k
At the end
8Marcos Nieto, PhD - [email protected]/2/2011
9
Bayesian filtering
K-1 K K+1 TIME
MEASUREMENTS
(VISIBLE)
STATES
(HIDDEN)xk-1
zk-1
xk
zk
xk+1
zk+1
p(xk|xk-1): Dynamic model
p(zk|xk): Observation model
9Marcos Nieto, PhD - [email protected]/2/2011
10
Bayesian filtering
How?
PredictionUse the dynamics, guess future according to
CorrectionObtain a new observation, and apply Bayes’ rule
p(xk|xk-1)
Likelihood Prediction
Posterior
10Marcos Nieto, PhD - [email protected]/2/2011
11
Particle filters
Solve through sampling!
Let us approximate posterior as a set of samples
Samples / Particles / Hypotheses
WEIGHTED UNWEIGHTED
11Marcos Nieto, PhD - [email protected]/2/2011
12
Particle filters
Understanding particles Each particle represents a hypothesis
Remember! we will typically want just one point-estimate Best particle, mean particle, mode, median…
W
H
(x0,y0)
W
H
x0
y0
12Marcos Nieto, PhD - [email protected]/2/2011
13
Particle filters
How to sample?
Importance sampling
Markov Chain Monte Carlo
Gibbs sampling
Slice sampling
…
How many samples?
As much as required to track the posterior!
13Marcos Nieto, PhD - [email protected]/2/2011
14
Particle filters
Sequential importance sampling (SIR)
14Marcos Nieto, PhD - [email protected]/2/2011
15
Particle filters
Sequential importance sampling (SIR)
15Marcos Nieto, PhD - [email protected]/2/2011
17
SIR – example (I)
Linear-Gaussian dynamics
Generate N samples starting fromprevious state adding estimatedvelocity
And some Gaussian noise
The noise makes thatsamples are different!
T),,,( yxyxkx
11 kkk A vxx),;()|( 111 kkkk QANp xxxx
1000
0100
010
001
k
k
A
17Marcos Nieto, PhD - [email protected]/2/2011
18
SIR - example (I)
Likelihood based on segmentation or color histogram
Evaluate each predicted sample according to this value
Likelihood function shouldreturn high values for
“good” hypotheses, and lowfor “bad” hypotheses
18Marcos Nieto, PhD - [email protected]/2/2011
19
SIR – example (II)
Eye-tracking
Linear prediction won’t work The projection of the eye movement on the screen is
difficult to predict
Define a combination of linear-Gaussian + uniform
19Marcos Nieto, PhD - [email protected]/2/2011
2020Marcos Nieto, PhD - [email protected]/2/2011
2121Marcos Nieto, PhD - [email protected]/2/2011
22
SIR – example (II)
• Conclusion:
Try to adapt the dynamic model you use with what youthink is the real dynamics of what you want to track
This may imply using mixture models, accelerations, etc.
Also, a good likelihood model should include somecontinuous term (like a uniform), in order to cope withocclusions, so that the track is not lost
22Marcos Nieto, PhD - [email protected]/2/2011
23
SIR
Problems Required number of samples increase exponentially with
problem dimension
Several objects/elements?a) Define a multimodal posterior and generate multiple point-
estimatesb) Cluster particlesc) Increase state vector dimension
Variable number of objects?a) Add external handlerb) Include the number of objects as another variable to estimate
23Marcos Nieto, PhD - [email protected]/2/2011
24
Particle filters
MCMC
More flexible
The problem of dimension is softened
Directly sample from the posterior
Researchers are focusing in MCMCMany excellent works that propose solutions to multiple
object, interaction, entering-exiting, number of samplesreduction, etc.
24Marcos Nieto, PhD - [email protected]/2/2011
25
Particle filters
MCMC
Generate a Markov chain of samples directly fromthe posterior
25Marcos Nieto, PhD - [email protected]/2/2011
26
MCMC
Metropolis-Hastings
Start somehow
Propose a movement
Accept with probabilityequal to the ratio betweenproposed value and previousone
Prob. = 1 if proposed is betterthan previous
Prob. = ratio if not
Metropolis-Hastings allowsobtaining samples for anarbitrary distribution bymaking a chain which acceptsor rejects movements
26Marcos Nieto, PhD - [email protected]/2/2011
27
MCMC
Multiple objects
State vector including all thedimensions of all objects
Metropolis-Hastings: Generate a chain of N samples
For each sample, use theinformation of all the samplesat the previous time instant
Each sample is a hypothesis of the state
of all objects
After the chain is completed, we have
the sample-basedapproximation of the
posterior
27Marcos Nieto, PhD - [email protected]/2/2011
28
MCMC
Marginalized proposal moves
Propose movement of a single dimension at each new sample
E.g. don’t propose a move in all dimensions for all objects
Choose a dimension randomly and update it
x W y
x WHx xW L
…
…
Burn-in period Stop when stationary function is
reached.Or when maximum number of
samples is reached.
28Marcos Nieto, PhD - [email protected]/2/2011
29
MCMC
Interaction between objects
Markov Random Field (MRF) factor
29Marcos Nieto, PhD - [email protected]/2/2011
30
MCMC
Variable number of objects
a) Add an external detector, and modify state size
b) Reversible-Jump MCMC Define an Enter move (creates an object)
Define an Exit move (removes an object)
Define an Update move (updates existing objects)
30Marcos Nieto, PhD - [email protected]/2/2011
32
Discussion
If dynamics and observation are linear, and with Gaussian noise
Use Kalman, this is the optimum solution
If not, consider using a particle filter
32Marcos Nieto, PhD - [email protected]/2/2011
33
Discussion
SIR
Use it if target dimension is low (3-5)
Use it if you plan to parallelize processing Remember particles are independent one from another
Would require important design issues forManaging multiple objects
Managing variable number of objects
33Marcos Nieto, PhD - [email protected]/2/2011
34
Discussion
MCMCUse it if dimensions increase
It can not be parallelized Remember that particles form a chain, and each one
depends on the previous one
Adapted to multiple objectsMRF interaction is easy to insert
Metropolis-Hastings can be efficiently adapted tomultiple object
34Marcos Nieto, PhD - [email protected]/2/2011
35
Summary
Define your target
Determine its dynamics
Define the likelihood
Select a filter that adapts to the problem
Implement it
Run it carefully selecting the appropriateparameters of your functions, number of particles, etc.
35Marcos Nieto, PhD - [email protected]/2/2011
362/2/2011 36
Dr.-Ing. Marcos Nieto DoncelInvestigador/Researcher
http://marcosnieto.net/