bramble: the bayesian multiple-blob tracker by michael isard and john maccormick presented by...
TRANSCRIPT
BraMBLe: The Bayesian Multiple-BLob Tracker
By Michael Isard and John MacCormick
Presented by Kristin Branson
CSE 252C, Fall 2003
Problem The goal is to track an unknown number of
blobs from static camera video.
Solution The Bayesian Multiple-BLob (BraMBLe)
tracker is a Bayesian solution. It estimates
),...,,|p( 11 ZZZX ttt
State at frame t Image Sequence
Number,Positions,Shapes,Velocities,…
Bayes Rule
p(p( )) p)| (| Z XX XZ
Posterior State Distribution Observation
Likelihood
Prior
Sequential Bayes
Posterior State Distribution Observation
Likelihood
Prior
Instead of modeling directly, BraMBLe models and .
)|p( :1 tt IX)|p( tt XI )|p( 1tt XX
1 1 1: 1 11: p( | ) p(p( | ) ) | )p( | t tt t tt tt t d X X X Z XZ XX Z
)|p( 1:11 tt ZX
Update Algorithm
)|p( :1 tt ZX
)|p( :1 tt ZX
)|p( 1:1 tt ZX
)|p( 1tt xx
)|p( tt XZ
)|p( :11 tt ZX
)|p( 11 tt XZ)|p( 1 tt XX
1 1 1: 1 11: p( | ) p(p( | ) ) | )p( | t tt t tt tt t d X X X Z XZ XX Z
Outline Observation likelihood model . Prediction model . Estimation of posterior . Results. Discussion.
)|p( tt XZ)|p( 1tt XX
)|p( :1 tt ZX
Outline Observation likelihood model . Prediction model . Estimation of posterior . Results. Discussion.
)|p( tt XZ)|p( 1tt XX
)|p( :1 tt ZX
Image Observations We want to choose our observations so
that we can compute quickly:
Z
)|p( XZ
g gz )|p()|p( XXZ
Individual observations are conditionally independent.
Image Observations We want . A bank of filters is applied at each grid point.
Z
g gz )|p()|p( XXZ
Y
Cr
Cb
Filter plots from http://www.cs.jhu.edu/~wolff/course600.461/week3.2/sld012.htm
Mexican Hat
Gaussian
Image Observations We want to choose our model so that we
can compute quickly:
Z
)|p( Xgz
g ggg g lzz )|p()|p()|p( XXZ
Observation depends on membership of grid point.
0gl
2gl1gl
3gl
Image Observations We want to choose our model so that we
can compute quickly:
We can precompute and quickly evaluate any state x.
Z
)|p( Xgz
g ggg g lzz )|p()|p()|p( XXZ
ggg llz )|p(
Appearance Models The appearance models are learned from
training data.
forek
foref
forefpfpf G
Klz ),(
1)0|p(
Training Data
backk backbackgk
backgkgg G
Klz ),(
1)0|p(
Observation Likelihood
)0|p(
)0|p(log
gg
gg
lz
lz
Observation Likelihood Review We defined our image observations
so that We defined our observation model
so that
We can precompute and quickly evaluate for many choices of x.
Z.)|p()|p(
g gz XXZ
)|p( Xgz
.)|p()|p()|p( g ggg g lzz XXZ
ggg lglz ,)|p( )|p( XZ
Outline Observation likelihood model . Prediction model . Estimation of posterior . Results. Discussion.
)|p( tt XZ)|p( 1tt XX
)|p( :1 tt ZX
Object Model The blob configuration is
mm xxX ...,,, 1Number of objects
ObjectState
1x
2x
3x
3m
X
Object Model The blob configuration is
The object state is
svlx ,,,i
mm xxX ...,,, 1
IdentityLocation
VelocityShape
X
Person Model
Generalized-Cylinder Model
CalibratedCamera
Prediction Model The number of objects can change:
Each object has a constant probability of remaining in the scene.
There is a constant probability that a new object will enter the scene.
)|p( 1tt XX
r
i
In this formulation, hypotheses with different numbers of objects can be compared directly.
Prediction Model Damped constant location velocity:
)|p( 1tt XX
11 1 12p( | , ) ( 0 8 ). ,t tt tt xx x xG xx
1tx 1tx
1 10.8t tx x
Prediction Model Damped constant location velocity:
Auto-regressive shape:
)|p( 1tt XX
11 1 12p( | , ) ( 0 8 ). ,t tt tt xx x xG xx
12
1 1 10.2(p( | ) ( ,) )t t tt t sss sG s s
1ts
1 1 10.2( )t t ts s s
1ts
Model Review
The observation likelihood is fast to compute for different hypotheses .
The prediction model allows generation of from
Estimating requires an efficient way ofRepresenting .Computing the multiplications and integrations.
tX
tX 1.t X
)|p( tt XZ
)|p( 1tt XX
1 1 1: 1 11: p( | ) p(p( | ) ) | )p( | t tt t tt tt t d X X X Z XZ XX Z
)|p( :1 tt ZX
)|p( :1 tt ZX
Outline Observation likelihood model . Prediction model . Estimation of posterior . Results. Discussion.
)|p( tt XZ)|p( 1tt XX
)|p( :1 tt ZX
Efficient Representation is represented by a set of particles,
:
Sampling from the set using the weights approximates sampling from
1tX 2
tX 1Nt
X NtX...
N Points:
N Weights: 1t 2
t 1Nt N
t
)|p( :1 tt ZX
1:p( | ).t tX Z
{( , )}i it tX
Efficient Representation is represented by a set of particles,
:
Sampling from the set using the weights approximates sampling from
)|p( :1 tt ZX
1:p( | ).t tX Z
{( , )}i it tX
Efficient Computation The particle set representing is
computed from by CONDENSATION:
1. Apply dynamics to the particle set:
2. Multiply by the observation likelihood:
)|p( :1 tt ZX
1 1: 1p( | )t t X Z
: 1: 11 p( | p(p( | ) ) | )t tt tt t X ZX XZ Z
1 1 1: 1 11: p( | ) p(p( | ) ) | )p( | t tt t tt tt t d X X X Z XZ XX Z
1: 1 1 1 11: 1p( |p( p( )| ) | ) tt t t t tt d X ZX XXZ X
Applying Dynamics Given particle set , compute1 1: 1p( | )t t X Z
Image from http://www.hpl.hp.com/personal/John_MacCormick/WOMOT03/cal.giftalk/page.018.gif
1 1: 1p( | )t t X Z
1: 1p( | )t t X Z
)|p( 1tt XX
1: 1 1 1 11: 1p( |p( p( )| ) | ) tt t t t tt d X ZX XXZ X
Applying Dynamics Given particle set , compute
1. Resample into
1 1: 1p( | )t t X Z
1 1{( , )}i it t X
1{( ' ,1/ )}it NX
1 1{( , )}i it t X
1{( ' ,1/ )}it NX
Image from http://www.hpl.hp.com/personal/John_MacCormick/WOMOT03/cal.giftalk/page.020.gif
1: 1 1 1 11: 1p( |p( p( )| ) | ) tt t t t tt d X ZX XXZ X
Applying Dynamics Given particle set , compute
1. Resample into
2. Predict, generating to give 1p( | ' )i i
t t t X X X{( ,1/ )}.i
t NX
1{( ' ,1/ )}it NX
{( ,1/ )}it NX
1 1: 1p( | )t t X Z
1 1{( , )}i it t X
1{( ' ,1/ )}it NX
1: 1 1 1 11: 1p( |p( p( )| ) | ) tt t t t tt d X ZX XXZ X
Multiplication by Likelihood Given particle set , compute
Weight particles, setting
1: 1p( | )t t X Z
:1 1 1:p( | ) p( | )p( | .) t tt tt t Z X X ZX Z
1: 1p( | )t t X Z
)|p( tt XZ
)|p( :1 tt ZX
{( ,1/ )}it NX
{( , )}i it tX
p( | ).i it t t Z X
Image from http://www.hpl.hp.com/personal/John_MacCormick/WOMOT03/cal.giftalk/page.021.gif
Efficient Computation Review The particle set representing is
computed from by CONDENSATION:
)|p( :1 tt ZX
1 1: 1p( | )t t X Z
1 1: 1p( | )t t X Z
1 1{( , )}i it t X
1 1: 1p( | )t t X Z
1{( ' ,1/ )}it NX {( ,1/ )}i
t NX
1: 1p( | )t t X Z )|p( :1 tt ZX
{( , )}i it tX
Resample Predict Reweight
Image from http://www.hpl.hp.com/personal/John_MacCormick/WOMOT03/cal.giftalk/page.022.gif
Outline Observation likelihood model . Prediction model . Estimation of posterior . Results. Discussion.
)|p( tt XZ)|p( 1tt XX
)|p( :1 tt ZX
People Tracking
People Tracking Tracking was successful in real time on
this 53s clip except when two people crossed in front of a third.
Outline Observation likelihood model . Prediction model . Estimation of posterior . Results. Discussion.
)|p( tt XZ)|p( 1tt XX
)|p( :1 tt ZX
Algorithm Summary The models chosen
Are a smooth integration of foreground and background models.
Allow hypotheses with differing numbers of objects to be compared directly.
Can be quickly evaluated in a particle filtering implementation.
Relationship to Previous Work Static camera blob tracking:
Classifies pixels as foreground or background.
Application of Stauffer and Grimson’s Adaptive Background Subtraction to video with compression artifacts
Video from http://image.pirl.umd.edu/knkim/research/BGS/compressed_video.htm
Relationship to Previous Work Static camera blob tracking:
Classifies pixels as foreground or background. Static camera blob tracking:
Finds the position in the search area Made up of foreground pixels. Matching the blob in the previous frame.
Frame t - 1 Frame t Frame t + 1
Predicted position in frame t + 1
Search area
Relationship to Previous Work Improvements over blob tracking:
Integrates the foreground and background modeling.
Multiple objects can be tracked through occlusions.
Video from http://robotics.stanford.edu/~birch/headtracker/Video from http://www.robots.ox.ac.uk/~jmac/research/thesis/thesis.html
Weaknesses The algorithm is sensitive to reflections.
The algorithm sometimes switches the labels when one object passes in front of another.
There are a lot of parameters to assign.