a sparse probabilistic learning algorithm for real-time...
TRANSCRIPT
![Page 1: A Sparse Probabilistic Learning Algorithm for Real-Time Trackingmi.eng.cam.ac.uk/~cipolla/archive/Presentations/2003... · 2014-05-19 · A Sparse Probabilistic Learning Algorithm](https://reader034.vdocument.in/reader034/viewer/2022042400/5f0f7dae7e708231d4446d06/html5/thumbnails/1.jpg)
Oliver Williams
University of Cambridge
Andrew Blake
Microsoft Research,
Cambridge
Roberto Cipolla
University of Cambridge
A Sparse Probabilistic Learning Algorithm
for Real-Time Tracking
![Page 2: A Sparse Probabilistic Learning Algorithm for Real-Time Trackingmi.eng.cam.ac.uk/~cipolla/archive/Presentations/2003... · 2014-05-19 · A Sparse Probabilistic Learning Algorithm](https://reader034.vdocument.in/reader034/viewer/2022042400/5f0f7dae7e708231d4446d06/html5/thumbnails/2.jpg)
exploit temporal
coherence
object detector Self starting
Self recovering
Efficient
Robust Tracking
![Page 3: A Sparse Probabilistic Learning Algorithm for Real-Time Trackingmi.eng.cam.ac.uk/~cipolla/archive/Presentations/2003... · 2014-05-19 · A Sparse Probabilistic Learning Algorithm](https://reader034.vdocument.in/reader034/viewer/2022042400/5f0f7dae7e708231d4446d06/html5/thumbnails/3.jpg)
Background: Support Vector Tracking
Avidan [CVPR 2001]
SVM on object class
Treat score as function of
position
-100 -80 -60 -40 -20 0 20 40 60 80 100-3
-2
-1
0
1
2
3
Translation (pixels)S
VM
Sco
re
subimage
image gradient
![Page 4: A Sparse Probabilistic Learning Algorithm for Real-Time Trackingmi.eng.cam.ac.uk/~cipolla/archive/Presentations/2003... · 2014-05-19 · A Sparse Probabilistic Learning Algorithm](https://reader034.vdocument.in/reader034/viewer/2022042400/5f0f7dae7e708231d4446d06/html5/thumbnails/4.jpg)
Learn right versus left
Training for displacement
positive example
negative example
SVM
Translation (pixels)
SVM
Sco
re
-u
+u
Calibrate state update from SVM score
![Page 5: A Sparse Probabilistic Learning Algorithm for Real-Time Trackingmi.eng.cam.ac.uk/~cipolla/archive/Presentations/2003... · 2014-05-19 · A Sparse Probabilistic Learning Algorithm](https://reader034.vdocument.in/reader034/viewer/2022042400/5f0f7dae7e708231d4446d06/html5/thumbnails/5.jpg)
Relevance Vector
Machine
Probabilistic
Statistical filter
Want sparsity
Computational efficiency
Temporal Fusion
Tipping [NIPS 2000]
SVM in Bayesian setting
Learns continuous function from training set {zi,ti}
![Page 6: A Sparse Probabilistic Learning Algorithm for Real-Time Trackingmi.eng.cam.ac.uk/~cipolla/archive/Presentations/2003... · 2014-05-19 · A Sparse Probabilistic Learning Algorithm](https://reader034.vdocument.in/reader034/viewer/2022042400/5f0f7dae7e708231d4446d06/html5/thumbnails/6.jpg)
RVM Evaluation Equation
weights
kernel function
test subimagetraining subimage
Setting some wi to zero sparse solution
![Page 7: A Sparse Probabilistic Learning Algorithm for Real-Time Trackingmi.eng.cam.ac.uk/~cipolla/archive/Presentations/2003... · 2014-05-19 · A Sparse Probabilistic Learning Algorithm](https://reader034.vdocument.in/reader034/viewer/2022042400/5f0f7dae7e708231d4446d06/html5/thumbnails/7.jpg)
Bayesian Training
Sparsity encouraged by zero-mean prior :
Training data modelled by Gaussian likelihood function
hyperparameter
provided target
“true” underlying value noise parameter
-5 -4 -3 -2 -1 0 1 2 3 4 5-5 -4 -3 -2 -1 0 1 2 3 4 5-5 -4 -3 -2 -1 0 1 2 3 4 5
![Page 8: A Sparse Probabilistic Learning Algorithm for Real-Time Trackingmi.eng.cam.ac.uk/~cipolla/archive/Presentations/2003... · 2014-05-19 · A Sparse Probabilistic Learning Algorithm](https://reader034.vdocument.in/reader034/viewer/2022042400/5f0f7dae7e708231d4446d06/html5/thumbnails/8.jpg)
Occam’s Razor
Sometimes, lower Occam
penalty paid by removing an
example…
…and explaining data with
more noise
-5 -4 -3 -2 -1 0 1 2 3 4 5-5 -4 -3 -2 -1 0 1 2 3 4 5-5 -4 -3 -2 -1 0 1 2 3 4 5
![Page 9: A Sparse Probabilistic Learning Algorithm for Real-Time Trackingmi.eng.cam.ac.uk/~cipolla/archive/Presentations/2003... · 2014-05-19 · A Sparse Probabilistic Learning Algorithm](https://reader034.vdocument.in/reader034/viewer/2022042400/5f0f7dae7e708231d4446d06/html5/thumbnails/9.jpg)
Relevance Vector Machine
Posterior
Train RVM by finding hyperparameters
Maximise marginal likelihood
“Prune” vector i when i ! 0
![Page 10: A Sparse Probabilistic Learning Algorithm for Real-Time Trackingmi.eng.cam.ac.uk/~cipolla/archive/Presentations/2003... · 2014-05-19 · A Sparse Probabilistic Learning Algorithm](https://reader034.vdocument.in/reader034/viewer/2022042400/5f0f7dae7e708231d4446d06/html5/thumbnails/10.jpg)
Creating a Training Set
δX1
Select a few “seed” stills
Simulate translation, scaling and rotation
labelled training set
δX3
X2
![Page 11: A Sparse Probabilistic Learning Algorithm for Real-Time Trackingmi.eng.cam.ac.uk/~cipolla/archive/Presentations/2003... · 2014-05-19 · A Sparse Probabilistic Learning Algorithm](https://reader034.vdocument.in/reader034/viewer/2022042400/5f0f7dae7e708231d4446d06/html5/thumbnails/11.jpg)
RVM Tracking
X1 X2 X3 XN
4D RVM learns mapping
{zi}
{ti}
Input Space (images)
State space (u,v,s,)
R400
R4
RVM
![Page 12: A Sparse Probabilistic Learning Algorithm for Real-Time Trackingmi.eng.cam.ac.uk/~cipolla/archive/Presentations/2003... · 2014-05-19 · A Sparse Probabilistic Learning Algorithm](https://reader034.vdocument.in/reader034/viewer/2022042400/5f0f7dae7e708231d4446d06/html5/thumbnails/12.jpg)
RVM Tracking
Frame t Frame t+1
XtXt+1
xt x*t+1 xt+1
![Page 13: A Sparse Probabilistic Learning Algorithm for Real-Time Trackingmi.eng.cam.ac.uk/~cipolla/archive/Presentations/2003... · 2014-05-19 · A Sparse Probabilistic Learning Algorithm](https://reader034.vdocument.in/reader034/viewer/2022042400/5f0f7dae7e708231d4446d06/html5/thumbnails/13.jpg)
RVM Tracking
At each frame…
RVM
X
Statistical filter
Xt+1
Extract subimage
Random variable
New state estimate
Motion Model
x*t+1
xt+1
![Page 14: A Sparse Probabilistic Learning Algorithm for Real-Time Trackingmi.eng.cam.ac.uk/~cipolla/archive/Presentations/2003... · 2014-05-19 · A Sparse Probabilistic Learning Algorithm](https://reader034.vdocument.in/reader034/viewer/2022042400/5f0f7dae7e708231d4446d06/html5/thumbnails/14.jpg)
Initialization & Recovery
Algorithm trains from “seed”
subimages
Provided by localisation algorithm
Localisation algorithm also used for
Initialization (frame 0)
Validation
Re-initialization
![Page 15: A Sparse Probabilistic Learning Algorithm for Real-Time Trackingmi.eng.cam.ac.uk/~cipolla/archive/Presentations/2003... · 2014-05-19 · A Sparse Probabilistic Learning Algorithm](https://reader034.vdocument.in/reader034/viewer/2022042400/5f0f7dae7e708231d4446d06/html5/thumbnails/15.jpg)
Results: Computational Efficiency
Why track?
0 10 20 30 40 50 60 70 80 9010
1
102
103
CP
U t
ime
/ fra
me
(m
s)
Frame Number
Restarting
![Page 16: A Sparse Probabilistic Learning Algorithm for Real-Time Trackingmi.eng.cam.ac.uk/~cipolla/archive/Presentations/2003... · 2014-05-19 · A Sparse Probabilistic Learning Algorithm](https://reader034.vdocument.in/reader034/viewer/2022042400/5f0f7dae7e708231d4446d06/html5/thumbnails/16.jpg)
Results: Computational Efficiency
Localize with boosting [Viola, Jones 2001, Li et al. 2002]
0 10 20 30 40 50 60 70 80
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Time (s)
CP
U U
tiliz
ation
Boosting
RVM
![Page 17: A Sparse Probabilistic Learning Algorithm for Real-Time Trackingmi.eng.cam.ac.uk/~cipolla/archive/Presentations/2003... · 2014-05-19 · A Sparse Probabilistic Learning Algorithm](https://reader034.vdocument.in/reader034/viewer/2022042400/5f0f7dae7e708231d4446d06/html5/thumbnails/17.jpg)
Automatic Camera Management
Use position/scale information to control digital pan and
zoom
![Page 18: A Sparse Probabilistic Learning Algorithm for Real-Time Trackingmi.eng.cam.ac.uk/~cipolla/archive/Presentations/2003... · 2014-05-19 · A Sparse Probabilistic Learning Algorithm](https://reader034.vdocument.in/reader034/viewer/2022042400/5f0f7dae7e708231d4446d06/html5/thumbnails/18.jpg)
Conclusions
Future Work
RVM sparsity-smoothness tradeoff
Robustness to illumination and occlusion
Tracking 3D motion
Hybrid RVM tracking is…
Self starting SVM/Boosting detector
Self recovering SVM/Boosting validation
Efficient Temporal fusion +
sparsity
![Page 19: A Sparse Probabilistic Learning Algorithm for Real-Time Trackingmi.eng.cam.ac.uk/~cipolla/archive/Presentations/2003... · 2014-05-19 · A Sparse Probabilistic Learning Algorithm](https://reader034.vdocument.in/reader034/viewer/2022042400/5f0f7dae7e708231d4446d06/html5/thumbnails/19.jpg)
Questions?
![Page 20: A Sparse Probabilistic Learning Algorithm for Real-Time Trackingmi.eng.cam.ac.uk/~cipolla/archive/Presentations/2003... · 2014-05-19 · A Sparse Probabilistic Learning Algorithm](https://reader034.vdocument.in/reader034/viewer/2022042400/5f0f7dae7e708231d4446d06/html5/thumbnails/20.jpg)
Results: Cars
Algorithm is not specific to any class of objects
![Page 21: A Sparse Probabilistic Learning Algorithm for Real-Time Trackingmi.eng.cam.ac.uk/~cipolla/archive/Presentations/2003... · 2014-05-19 · A Sparse Probabilistic Learning Algorithm](https://reader034.vdocument.in/reader034/viewer/2022042400/5f0f7dae7e708231d4446d06/html5/thumbnails/21.jpg)
cinematographer
START
Establishment
Shot
Medium View
Zoom Withdraw
Pan &
Withdraw
Pan
Detect
New Pos
Big Change
Lost
Lost
![Page 22: A Sparse Probabilistic Learning Algorithm for Real-Time Trackingmi.eng.cam.ac.uk/~cipolla/archive/Presentations/2003... · 2014-05-19 · A Sparse Probabilistic Learning Algorithm](https://reader034.vdocument.in/reader034/viewer/2022042400/5f0f7dae7e708231d4446d06/html5/thumbnails/22.jpg)
1D example
Tutorial: toy data
100 Training
examples
12 Relevant vectors
remain
0 10 20 30 40 50 60 70 80 90 100-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
![Page 23: A Sparse Probabilistic Learning Algorithm for Real-Time Trackingmi.eng.cam.ac.uk/~cipolla/archive/Presentations/2003... · 2014-05-19 · A Sparse Probabilistic Learning Algorithm](https://reader034.vdocument.in/reader034/viewer/2022042400/5f0f7dae7e708231d4446d06/html5/thumbnails/23.jpg)
Results: Smoothness
0 100 200 300 400 500 600 7000
5
10
15
20
25
30
35
0 100 200 300 400 500 600 7000
5
10
15
20
25
30
35
RVM Boosting