sequential learned linear predictors for object tracking - center for machine...
TRANSCRIPT
![Page 1: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/1.jpg)
Sequential Learned Linear Predictors for ObjectTracking
Tomas Svobodajoint work with Karel Zimmermann, Jirı Matas, and David Hurych
Czech Technical University, Faculty of Electrical EngineeringCenter for Machine Perception, Prague, Czech Republic
http://cmp.felk.cvut.cz/~svoboda
June 23, 2009
![Page 2: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/2.jpg)
Object tracking
I Tracking - iterative estimation of an object pose in asequence (e.g., 2D position of Basil’s head).
I Trade-off among accuracy, speed, robustness is explicitlytaken into account.
video: Basil head
![Page 3: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/3.jpg)
Object tracking
I Tracking - iterative estimation of an object pose in asequence (e.g., 2D position of Basil’s head).
I Trade-off among accuracy, speed, robustness is explicitlytaken into account.
video: Basil head
![Page 4: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/4.jpg)
Object tracking
I Tracking - iterative estimation of an object pose in asequence (e.g., 2D position of Basil’s head).
I Trade-off among accuracy, speed, robustness is explicitlytaken into account.
video: Cartoon eye
![Page 5: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/5.jpg)
Tracking
Current imageTemplate
Motion
Observation
Observation (image) I and template J
![Page 6: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/6.jpg)
Tracking of a single point
X
Current position
![Page 7: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/7.jpg)
Tracking of a single point
X
Old position
![Page 8: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/8.jpg)
Tracking of a single point
X
Old position
![Page 9: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/9.jpg)
[Lucas-Kanade1981] - Iterative minimization
argmint‖I(t)− J‖2argmint‖I(t)− J‖2image I alignment t
J← I(t)
optional template updateLucas–Kanade
![Page 10: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/10.jpg)
[Lucas-Kanade1981] - Iterative minimization
argmint‖I(t)− J‖2argmint‖I(t)− J‖2image I alignment t
J← I(t)
optional template updateLucas–Kanade
video: KLT iterations
![Page 11: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/11.jpg)
Regression - learn the mapping in advanceo
H(I− J)H(I− J)image I alignment t
J← I(t)
optional template updateJurie–Dhome
I [Jurie-BMVC-2002] - learned motion and optional hardtemplate update
I [Cootes-PAMI-2001] - learned regression during AAM iterations
I . . .
![Page 12: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/12.jpg)
Learning alignment for one predictor
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
![Page 13: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/13.jpg)
Learning alignment for one predictor
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
![Page 14: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/14.jpg)
Learning alignment for one predictor
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
![Page 15: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/15.jpg)
Learning alignment for one predictor
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
![Page 16: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/16.jpg)
Learning alignment for one predictor
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
![Page 17: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/17.jpg)
Learning alignment for one predictor
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
![Page 18: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/18.jpg)
Generating training examples
[ ]
t − motioni I = I(x ,...x ) − observation1 ci
examples
displacementestimation
generating
+2
+7+9
+4
−8
−4
training set: +2 −4 +4+9 +7 −8
T= I=
Training set: (I, T)I = [I1 − J, I2 − J, . . . Id − J] and T = [t1, t2, . . . td ].
![Page 19: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/19.jpg)
LS learning
ϕ∗ = argminϕ∑
t
‖ϕ(I(t ◦ X
))− t‖2.
Minimizes sum of square errors over all training set. Leads tomatrix pseudoinverse computation.
Example for linear mapping:
H∗ = argminH
d∑i=1
‖H(Ii − J)− ti‖22 = argminH‖HI− T‖2F
after some derivation
H∗ = T I>(II>)−1︸ ︷︷ ︸I+
= TI+.
![Page 20: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/20.jpg)
LS learning
ϕ∗ = argminϕ∑
t
‖ϕ(I(t ◦ X
))− t‖2.
Minimizes sum of square errors over all training set. Leads tomatrix pseudoinverse computation.
Example for linear mapping:
H∗ = argminH
d∑i=1
‖H(Ii − J)− ti‖22 = argminH‖HI− T‖2F
after some derivation
H∗ = T I>(II>)−1︸ ︷︷ ︸I+
= TI+.
![Page 21: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/21.jpg)
LS learning
ϕ∗ = argminϕ∑
t
‖ϕ(I(t ◦ X
))− t‖2.
Minimizes sum of square errors over all training set. Leads tomatrix pseudoinverse computation.
Example for linear mapping:
H∗ = argminH
d∑i=1
‖H(Ii − J)− ti‖22 = argminH‖HI− T‖2F
after some derivation
H∗ = T I>(II>)−1︸ ︷︷ ︸I+
= TI+.
![Page 22: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/22.jpg)
Min-max learning
ϕ∗ = argminϕ maxt‖ϕ(I(t ◦ X
))− t‖∞.
Minimizes the worst case (the biggest estimation error) in thetraining set. Leads to linear programming.
![Page 23: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/23.jpg)
Tracking of a single point by a sequence of predictors
X
t =1 ϕ1
Old position
![Page 24: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/24.jpg)
Tracking of a single point by a sequence of predictors
X
o
1
t =1 ϕ1
Old position
![Page 25: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/25.jpg)
Tracking of a single point by a sequence of predictors
X
o
1
t =1 ϕ1
Old position
![Page 26: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/26.jpg)
Tracking of a single point by a sequence of predictors
X
o
1
t =1 ϕ1
Old position
![Page 27: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/27.jpg)
Tracking of a single point by a sequence of predictors
X
o
1
t =1 ϕ1
t =2 ϕ2
Old position
![Page 28: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/28.jpg)
Tracking of a single point by a sequence of predictors
X
o
o2
1
t =1 ϕ1
t =2 ϕ2
Old position
![Page 29: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/29.jpg)
Tracking of a single point by a sequence of predictors
X
o
o2
1
t =1 ϕ1
t =2 ϕ2
Old position
![Page 30: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/30.jpg)
Tracking of a single point by a sequence of predictors
X
o
o2
1
t =1 ϕ1
t =2 ϕ2
Old position
![Page 31: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/31.jpg)
Tracking of a single point by a sequence of predictors
X
o
o2
1
t =1 ϕ1
t =3 ϕ3
t =2 ϕ2
Old position
![Page 32: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/32.jpg)
Tracking of a single point by a sequence of predictors
X
o
o
o
2
1
3 t =1 ϕ1
t =3 ϕ3
t =2 ϕ2
Old position
![Page 33: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/33.jpg)
Tracking of a single point by a sequence of predictors
X
o
o
o
2
1
3
New position
t =1 ϕ1
t =3 ϕ3
t =2 ϕ2
Old position
![Page 34: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/34.jpg)
Tracking of a single point by a sequence of predictors
1 2Φ=(ϕ,ϕ,ϕ)
3
X
o
o
o
2
1
3
New position
t =1 ϕ1
Motion
t =3 ϕ3
t =2 ϕ2
Old position
![Page 35: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/35.jpg)
Tracking of a single point by a sequence of predictors
1 2Φ=(ϕ,ϕ,ϕ)
3
Ranges
X
o
o
o
2
1
3
New position
t =1 ϕ1
Motion
t =3 ϕ3
t =2 ϕ2
Old position
![Page 36: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/36.jpg)
Learning of sequential predictor
I Learning - searching for the sequence with predefined range,accuracy and minimal computational cost.
I [Zimmermann-PAMI-2009] - Dynamic programming estimatesthe optimal sequence of linear predictors.
I [Zimmermann-IVC-2009] - Branch & bound search allows fortime constrained learning (demo in MATLAB).
[Zimmermann-PAMI-2009] K.Zimmermann, J.Matas, T.Svoboda. Tracking by anOptimal Sequence of Linear Predictors, in IEEE Transactions on Pattern Analysis andMachine Intelligence, IEEE computer society, 2009, vol. 31, No 4, pp 677–692.
[Zimmermann-IVC-2009] K.Zimmermann, T.Svoboda, J.Matas. Anytime learning for
the NoSLLiP tracker. Image and Vision Computing, Elsevier, accepted, available
on-line.
![Page 37: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/37.jpg)
Learning of sequential predictor
I Learning - searching for the sequence with predefined range,accuracy and minimal computational cost.
I [Zimmermann-PAMI-2009] - Dynamic programming estimatesthe optimal sequence of linear predictors.
I [Zimmermann-IVC-2009] - Branch & bound search allows fortime constrained learning (demo in MATLAB).
[Zimmermann-PAMI-2009] K.Zimmermann, J.Matas, T.Svoboda. Tracking by anOptimal Sequence of Linear Predictors, in IEEE Transactions on Pattern Analysis andMachine Intelligence, IEEE computer society, 2009, vol. 31, No 4, pp 677–692.
[Zimmermann-IVC-2009] K.Zimmermann, T.Svoboda, J.Matas. Anytime learning for
the NoSLLiP tracker. Image and Vision Computing, Elsevier, accepted, available
on-line.
![Page 38: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/38.jpg)
Learning of sequential predictor
I Learning - searching for the sequence with predefined range,accuracy and minimal computational cost.
I [Zimmermann-PAMI-2009] - Dynamic programming estimatesthe optimal sequence of linear predictors.
I [Zimmermann-IVC-2009] - Branch & bound search allows fortime constrained learning (demo in MATLAB).
[Zimmermann-PAMI-2009] K.Zimmermann, J.Matas, T.Svoboda. Tracking by anOptimal Sequence of Linear Predictors, in IEEE Transactions on Pattern Analysis andMachine Intelligence, IEEE computer society, 2009, vol. 31, No 4, pp 677–692.
[Zimmermann-IVC-2009] K.Zimmermann, T.Svoboda, J.Matas. Anytime learning for
the NoSLLiP tracker. Image and Vision Computing, Elsevier, accepted, available
on-line.
![Page 39: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/39.jpg)
Learning of the optimal sequence of linear predictors
Range
Complexity
Uncertaintyregion
I Range: the set of admissible motions, r .I Complexity: cardinality of support set, c .I Uncertainty region: the region within which all predictions
lie, λ. Small red circles show acceptable uncertainty.
![Page 40: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/40.jpg)
Learning of the optimal sequence of linear predictors
0
20
40
60
80
100
0
50
100
150
200
0
10
20
30
40
50
r − range
c − complexity
λ −
unc
erta
inty
are
a
Achievablepredictors ω
Inachievablepredictors
λ−minimal predictors
I Range: the set of admissible motions, r .I Complexity: cardinality of support set, c .I Uncertainty region: the region within which all predictions
lie, λ. Small red circles show acceptable uncertainty.
![Page 41: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/41.jpg)
Learning of the optimal sequence of linear predictors
I Range: the set of admissible motions, r .I Complexity: cardinality of support set, c .I Uncertainty region: the region within which all predictions
lie, λ. Small red circles show acceptable uncertainty.
![Page 42: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/42.jpg)
Learning of the optimal sequence of linear predictors
I Range: the set of admissible motions, r .I Complexity: cardinality of support set, c .I Uncertainty region: the region within which all predictions
lie, λ. Small red circles show acceptable uncertainty.
![Page 43: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/43.jpg)
Branch and Bound
0 1 2 3 4 50
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
Graph depth
Pre
dic
tion e
rror
c=600c=320 c=360
c=80
0 1 2 3 4 50
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
Graph depth
Pre
dic
tion e
rror
c=100
c=600c=320
c=380
Don’t forget to show the live demo!
![Page 44: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/44.jpg)
Branch and Bound
0 1 2 3 4 50
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
Graph depth
Pre
dic
tion e
rror
c=600c=320 c=360
c=80
0 1 2 3 4 50
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
Graph depth
Pre
dic
tion e
rror
c=100
c=600c=320
c=380
Don’t forget to show the live demo!
![Page 45: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/45.jpg)
Tracking with one linear predictor.
![Page 46: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/46.jpg)
Modeling motion by number of linear predictors.
![Page 47: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/47.jpg)
Motion blur, fast motion, views from acute angles andother image distortions.
![Page 48: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/48.jpg)
Support set selection
0.065 0.07 0.075 0.08 0.085 0.09 0.095 0.1 0.105 0.110
50
100
150
200
250
300
350
400
450
500
MSE
I Greedy LS selection (red) of an efficient support set.
I Much better than 1%-quantile (green) achieavable byrandomized sampling
![Page 49: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/49.jpg)
Tracking of objects with variable appearance
I Variable appearance - the way how the object looks like inthe camera changes due to illumination, non-rigiddeformation, out-of-plane rotation, ...
![Page 50: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/50.jpg)
Tracking of objects with variable appearance
I Variable appearance - the way how the object looks like inthe camera changes due to illumination, non-rigiddeformation, out-of-plane rotation, ...
![Page 51: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/51.jpg)
Simultaneous learning of motion and appearance
ϕ(I)ϕ(I)image I alignment t
I Introduce feedback which encodes appearance in a lowdimensional space and adjust the predictor.
I Appearance parameters learned in unsupervised way.
I Simultaneous learning of ϕ and γ ⇒ appearance encoded inthe low dimensional space, which is the most suitable for themotion estimation.
![Page 52: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/52.jpg)
Simultaneous learning of motion and appearance
ϕ(I;θ)ϕ(I;θ)
γ(J)γ(J)
image I alignment t
J← I(t) imageθ appearance parameters
I Introduce feedback which encodes appearance in a lowdimensional space and adjust the predictor.
I Appearance parameters learned in unsupervised way.
I Simultaneous learning of ϕ and γ ⇒ appearance encoded inthe low dimensional space, which is the most suitable for themotion estimation.
![Page 53: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/53.jpg)
Simultaneous learning of motion and appearance
ϕ(I;θ)ϕ(I;θ)
γ(J)γ(J)
image I alignment t
J← I(t) imageθ appearance parameters
I Introduce feedback which encodes appearance in a lowdimensional space and adjust the predictor.
I Appearance parameters learned in unsupervised way.
I Simultaneous learning of ϕ and γ ⇒ appearance encoded inthe low dimensional space, which is the most suitable for themotion estimation.
![Page 54: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/54.jpg)
Simultaneous learning of motion and appearance
ϕ(I;θ)ϕ(I;θ)
γ(J)γ(J)
image I alignment t
J← I(t) imageθ appearance parameters
I Introduce feedback which encodes appearance in a lowdimensional space and adjust the predictor.
I Appearance parameters learned in unsupervised way.
I Simultaneous learning of ϕ and γ ⇒ appearance encoded inthe low dimensional space, which is the most suitable for themotion estimation.
![Page 55: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/55.jpg)
Learning appearance
H(I− J)H(I− J)image I alignment t
J← I(t)
optional template updateJurie–Dhome
![Page 56: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/56.jpg)
Learning appearance
H(I− J(θ))H(I− J(θ))H(I− J(θ))H(I− J(θ))
PCAJPCAJ
timage I
I(t)θ
Cootes–Edwards
![Page 57: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/57.jpg)
Learning appearance – our approach
H(I− J(θ))H(I− J(θ))H(I− J(θ))H(I− J(θ))
PCAJPCAJ
timage I
I(t)θ
Cootes–Edwards
ϕ(I;θ)ϕ(I;θ)
γ(I(t))γ(I(t))
image I alignment t
I(t) imageθ appearance parameters
Our algorithm
![Page 58: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/58.jpg)
Learning appearance – our approach
H(I− J(θ))H(I− J(θ))H(I− J(θ))H(I− J(θ))
PCAJPCAJ
timage I
I(t)θ
Cootes–Edwards
ϕ(I;θ)ϕ(I;θ)
γ(I(t))γ(I(t))
image I t
I(t) imageθ appearance parameters
Our algorithm
motion estimator
appearance encoder
![Page 59: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/59.jpg)
Learning the appearance encoder γ
I Current appearance encoded in low-dim parameters.
I γ( ) = θ1 I γ( ) = θ2
![Page 60: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/60.jpg)
Learning the appearance encoder γ
I Current appearance encoded in low-dim parameters.
I γ( ) = θ1 I γ( ) = θ2
![Page 61: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/61.jpg)
Learning the appearance encoder γ
I Current appearance encoded in low-dim parameters.
I γ( ) = θ1 I γ( ) = θ2
![Page 62: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/62.jpg)
Learning the tracker ϕ(I; θ)
I ϕ( ; θ1)= (0, 0)>
I ϕ( ; θ1)= (−25, 0)>
I ϕ( ; θ1)= (25,−15)>
I ϕ( ; θ2)= (0, 0)>
I ϕ( ; θ2)= (−25, 0)>
I ϕ( ; θ2)= (25,−15)>
![Page 63: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/63.jpg)
Learning the tracker ϕ(I; θ)
I ϕ( ; θ1)= (0, 0)>
I ϕ( ; θ1)= (−25, 0)>
I ϕ( ; θ1)= (25,−15)>
I ϕ( ; θ2)= (0, 0)>
I ϕ( ; θ2)= (−25, 0)>
I ϕ( ; θ2)= (25,−15)>
![Page 64: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/64.jpg)
Learning the tracker ϕ(I; θ)
I ϕ( ; θ1)= (0, 0)>
I ϕ( ; θ1)= (−25, 0)>
I ϕ( ; θ1)= (25,−15)>
I ϕ( ; θ2)= (0, 0)>
I ϕ( ; θ2)= (−25, 0)>
I ϕ( ; θ2)= (25,−15)>
![Page 65: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/65.jpg)
Learning the tracker ϕ(I; θ)
I ϕ( ; θ1)= (0, 0)>
I ϕ( ; θ1)= (−25, 0)>
I ϕ( ; θ1)= (25,−15)>
I ϕ( ; θ2)= (0, 0)>
I ϕ( ; θ2)= (−25, 0)>
I ϕ( ; θ2)= (25,−15)>
![Page 66: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/66.jpg)
Learning the tracker ϕ(I; θ)
I ϕ( ; θ1)= (0, 0)>
I ϕ( ; θ1)= (−25, 0)>
I ϕ( ; θ1)= (25,−15)>
I ϕ( ; θ2)= (0, 0)>
I ϕ( ; θ2)= (−25, 0)>
I ϕ( ; θ2)= (25,−15)>
![Page 67: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/67.jpg)
Learning the tracker ϕ(I; θ)
I ϕ( ; θ1)= (0, 0)>
I ϕ( ; θ1)= (−25, 0)>
I ϕ( ; θ1)= (25,−15)>
I ϕ( ; θ2)= (0, 0)>
I ϕ( ; θ2)= (−25, 0)>
I ϕ( ; θ2)= (25,−15)>
![Page 68: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/68.jpg)
Simultaneous learning of ϕ and γ
I Learning = minimization of the least-squares error
(ϕ∗, γ∗) = arg minϕ,γ[ϕ(
; γ( ))−(0, 0)>
]2+[
ϕ(
; γ( ))−(−25, 0)>
]2+[
ϕ(
; γ( ))−(25,−15)>
]2+[
ϕ(
; γ( ))−(0, 0)>
]2+[
ϕ(
; γ( ))−(−25, 0)>
]2+[
ϕ(
; γ( ))−(25,−15)>
]2
![Page 69: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/69.jpg)
Simultaneous learning of ϕ and γ
I Learning = minimization of the least-squares error
(ϕ∗, γ∗) = arg minϕ,γ[ϕ(
; γ( ))−(0, 0)>
]2+[
ϕ(
; γ( ))−(−25, 0)>
]2+[
ϕ(
; γ( ))−(25,−15)>
]2+[
ϕ(
; γ( ))−(0, 0)>
]2+[
ϕ(
; γ( ))−(−25, 0)>
]2+[
ϕ(
; γ( ))−(25,−15)>
]2
![Page 70: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/70.jpg)
Linear mapping
I γ(J) : θ = GJ
I ϕ(I,θ) : t = (H0 + θ1H1 + · · ·+ θnHn)I
I Criterion is sum of squares of bilinear functions.
![Page 71: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/71.jpg)
Linear mapping
I γ(J) : θ = GJ
I ϕ(I,θ) : t = (H0 + θ1H1 + · · ·+ θnHn)I
I Criterion is sum of squares of bilinear functions.
![Page 72: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/72.jpg)
Linear mapping
I γ(J) : θ = GJ
I ϕ(I,θ) : t = (H0 + θ1H1 + · · ·+ θnHn)I
I Criterion is sum of squares of bilinear functions.
![Page 73: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/73.jpg)
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
γϕ
0 2 4 6 8
0.8
1
1.2
1.4
1.6
1.8
2
2.2
2.4
I Global optimality for linear ϕ, γ experimentally shown.
![Page 74: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/74.jpg)
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
γϕ
0 2 4 6 8
0.8
1
1.2
1.4
1.6
1.8
2
2.2
2.4
I Global optimality for linear ϕ, γ experimentally shown.
![Page 75: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/75.jpg)
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
γϕ
[ϕ0, γ0]
0 2 4 6 8
0.8
1
1.2
1.4
1.6
1.8
2
2.2
2.4
I Global optimality for linear ϕ, γ experimentally shown.
![Page 76: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/76.jpg)
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
γϕ [ϕ1, γ0]
0 2 4 6 8
0.8
1
1.2
1.4
1.6
1.8
2
2.2
2.4
I Global optimality for linear ϕ, γ experimentally shown.
![Page 77: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/77.jpg)
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
γϕ [ϕ1, γ1]
0 2 4 6 8
0.8
1
1.2
1.4
1.6
1.8
2
2.2
2.4
I Global optimality for linear ϕ, γ experimentally shown.
![Page 78: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/78.jpg)
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
γϕ [ϕ2, γ1]
0 2 4 6 8
0.8
1
1.2
1.4
1.6
1.8
2
2.2
2.4
I Global optimality for linear ϕ, γ experimentally shown.
![Page 79: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/79.jpg)
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
γϕ [ϕ2, γ2]
0 2 4 6 8
0.8
1
1.2
1.4
1.6
1.8
2
2.2
2.4
I Global optimality for linear ϕ, γ experimentally shown.
![Page 80: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/80.jpg)
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
γϕ
[ϕ3, γ2]
0 2 4 6 8
0.8
1
1.2
1.4
1.6
1.8
2
2.2
2.4
I Global optimality for linear ϕ, γ experimentally shown.
![Page 81: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/81.jpg)
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
γϕ
[ϕ3, γ3]
0 2 4 6 8
0.8
1
1.2
1.4
1.6
1.8
2
2.2
2.4
I Global optimality for linear ϕ, γ experimentally shown.
![Page 82: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/82.jpg)
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
γϕ
[ϕ9, γ9]
0 2 4 6 8
0.8
1
1.2
1.4
1.6
1.8
2
2.2
2.4
I Global optimality for linear ϕ, γ experimentally shown.
![Page 83: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/83.jpg)
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
0 50 100 15020
25
30
35
40
45
50
iterationsM
SE
I Global optimality for linear ϕ, γ experimentally shown.
![Page 84: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/84.jpg)
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
0 5 10 15 20
22
23
24
25
26
27
28
29
30
31
32
iterationsM
SE
I Global optimality for linear ϕ, γ experimentally shown.
![Page 85: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/85.jpg)
Simultaneous learning of motion and appearance
![Page 86: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/86.jpg)
Experiments - videos II
![Page 87: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/87.jpg)
Conclusions
I Learnable and very efficient tracking of objects with variableappearance.
I Accuracy, speed, robustness explicitly taken into account.
I Simultaneous learning motion and appearance.
Limitations
I Small, thin objects intractable.
Data, papers, various implemenations freely available athttp://cmp.felk.cvut.cz/demos/Tracking/linTrack/
![Page 88: Sequential Learned Linear Predictors for Object Tracking - Center For Machine ...cmp.felk.cvut.cz/demos/Tracking/linTrack/predictors_talk.pdf · 2009-06-23 · Learning of sequential](https://reader033.vdocument.in/reader033/viewer/2022050606/5fad9052556df83ff274acc6/html5/thumbnails/88.jpg)
Conclusions
I Learnable and very efficient tracking of objects with variableappearance.
I Accuracy, speed, robustness explicitly taken into account.
I Simultaneous learning motion and appearance.
Limitations
I Small, thin objects intractable.
Data, papers, various implemenations freely available athttp://cmp.felk.cvut.cz/demos/Tracking/linTrack/