Download - Visual Tracking Decomposition
![Page 1: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/1.jpg)
Junseok Kwon* and Kyoung Mu lee
Computer Vision Lab.Dept. of EECS Seoul National University, Korea
Homepage: http://cv.snu.ac.kr
![Page 2: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/2.jpg)
Goal of Visual Tracking
Robustly tracks target in real-world scenarios
Frame #1 Frame #60
![Page 3: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/3.jpg)
Real-World Scenarios
Occlusions
Illumination changes
Abrupt motions
Pose variationsMixed
![Page 4: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/4.jpg)
Previous Works
In the real-world scenarios, conventional tracking methods frequently fail.
OAL Tracker [2]MIL Tracker [1]Our method
[2] Ross et. al. Incremental learning for robust visual tracking. IJCV 2007.
[1] Babenko et. al. Visual tracking with online multiple instance learning. CVPR 2009.
![Page 5: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/5.jpg)
Bayesian Tracking Approach Maximum a Posteriori (MAP) estimate
)Y|X(pmaxarg t:1tXt
Position, scalecolor
edge
![Page 6: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/6.jpg)
Bayesian Tracking Approach
)Y|X(pmaxarg t:1tXt
)X|Y(p tt )X|X(p 1tt 1t1t:11t dX)Y|X(p
Observation model Motion model
Update rule
![Page 7: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/7.jpg)
Compound Model
Compound Observation Model
Pose variation
Occlusion
Illumination change
Clutters
Smooth
Abrupt
Compound Motion Model
Need for real-world scenarios But difficult to design
![Page 8: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/8.jpg)
Our Approach Observation Model
Decomposition
+ + +Basic
Observation Model
1
Basic Observation Model r
Basic Observation Model 2
Compound Observation Model
1w,)X|Y(pw)X|Y(pr
1ii
r
1ittiitt
![Page 9: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/9.jpg)
Our Approach
+ + +Basic
Motion Model 2
Basic Motion Model s
Basic Motion Model
1
Motion Model Decomposition
1w,)X|X(pw)X|X(ps
1jj
s
1j1ttjj1tt
Compound Motion Model
![Page 10: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/10.jpg)
Our Approach
Basic Tracker 1
Basic Tracker 2
Basic Tracker rs
Tracker DecompositionBasic
Observation Model 1
Basic Motion Model 1
Basic Motion Model 2
Basic Motion Model s
Basic Observation Model r
Basic Observation Model 2
Basic Observation Model 1
Basic Motion Model 1
Basic Observation Model 1
Basic Motion Model 2
Basic Observation Model r
Basic Motion Model s
![Page 11: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/11.jpg)
Our Approach
Basic Tracker 1
Basic Tracker 2
Basic Tracker rs
Tracker Decomposition Each tracker takes charge of a certain
change in the object.
![Page 12: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/12.jpg)
Our Approach
Sampling based Tracker Markov Chain Monte Carlo (MCMC)
Basic Tracker
Sampling…
Basic Observation Model i
Basic Motion Model j
Basic Observation Model i
Basic Motion Model j
)Y|X(p t:1tji
![Page 13: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/13.jpg)
Remaining Tasks
How to determine the basic models ?
How to estimate weights of the models ?
Basic Observation Model 1
Basic Observation Model r
Basic Motion Model 1
Basic Motion Model s
Basic Observation Model 1
Basic Observation
Model r
Basic Motion Model 1
Basic Motion Model s
![Page 14: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/14.jpg)
Remaining Tasks
How to determine the basic models ?
Sparse PCA [1]
How to estimate weights of the models ? Interactive MCMC [2]
[2] J. Corander et. al. Parallell interacting MCMC for learning of topologies of graphical models. Data Min. Knowl. Discov., 2005.
[1] A. d’Aspremont et. al., A direct formulation for sparse PCA using semidefinite programming. SIAM Review, 49(3), 2007.
![Page 15: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/15.jpg)
Design of Basic Observation Models
Template set
tS
Hue
Saturation
Value
Edge
4 recent frames
1 initial frame
1tM 2
tM rtM
Object models
A subset of the template set
)X|Y(p tt1 )X|Y(p tt2 )X|Y(p ttr
Basic observation models
Diffusion distance
![Page 16: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/16.jpg)
Object Model
Three conditions Representativeness
The model has to cover most appearance changes in an object over time.
Compactness
The formation of it should be as compact as possible.
Complementary relation The relations between models should be
complementary.
1tM 2
tM rtM
![Page 17: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/17.jpg)
Object Model
Sparse Principal Component Analysis (SPCA)
1c to
2
2
subject
ccAcMaximize tT PCA Sparsenes
s
tAc
: Gram matrix of the template set
: Principal component
1tM 2
tM rtM
![Page 18: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/18.jpg)
Object Model
1subject
ccAcMaximize
2
2
tT
c to
tA
Tem
pla
te
set
Template set
![Page 19: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/19.jpg)
Object Model
Sparse
PC 10 0 0 0 0 0 0 0
Object model 1
Representativeness
Sparse
PC 2 Object model 20 0 0 0 0 0 0 00 0
Compactness
Sparse
PC r Object model r0 0 0 0 0 00 0 0
Complementary relation
1subject
ccAcMaximize
2
2
tT
c to
![Page 20: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/20.jpg)
Basic Observation Model
Diffusion distance [3]
[3] H. Ling and K. Okada. Diffusion distance for histogram comparison. CVPR, 2006.
)M,λDD(Y ittexp
Edge
Object model
Diffusion distance [3]
Hue Saturation
Value
Edge
)X|(Yp tti
![Page 21: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/21.jpg)
Two conditions Exploitation ( for smooth motions )
Further simulating the seemingly good moves near the local minima
Exploration ( for abrupt motions ) Further simulating moves that have not
been explored much
ExplorationExploitation
Design of Basic Motion Models
),(| 211 jtttj XG)X(Xp
![Page 22: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/22.jpg)
Weights of Basic Models
Parallel Mode Interaction Mode
Basic Tracker 1
Basic Tracker 2
Basic Tracker rs
Basic Observation Model 1
Basic Motion Model 2
Basic Observation Model 1
Basic Motion Model 1
Basic Observation Model r
Basic Motion Model s
State
State
![Page 23: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/23.jpg)
Experimental Results
The number of models Basic observation models : #4 Basic motion models : #2 Basic tracker models: #8(=4X2)
Settings for comparison Standard MCMC (MC) : 800 samples Mean Shift (MS) On-line Appearance Learning (OAL) : 800
samples Multiple Instance Learning (MIL)
OAL : Ross et. al. Incremental learning for robust visual tracking. IJCV 2007.MIL : Babenko et. al. Visual tracking with online multiple instance learning. CVPR 2009.
![Page 24: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/24.jpg)
Experimental Results
![Page 25: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/25.jpg)
Abrupt Motions and Illumination Changes
![Page 26: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/26.jpg)
Illumination Changes and Pose Variations
![Page 27: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/27.jpg)
Occlusions and Pose Variations
![Page 28: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/28.jpg)
Background Clutters
![Page 29: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/29.jpg)
Quantitative ResultsMC MS OAL MIL VTD
tiger1 27 93 65 15 13
david 49 88 4 23 7
face 19 45 19 27 7
shaking
97 241 95 38 5
soccer 47 97 151 41 21
animal 32 207 23 30 11
skating1
111 141 174 85 7
- Average center location errors in pixels
OAL : Ross et. al. Incremental learning for robust visual tracking. IJCV 2007.MIL : Babenko et. al. Visual tracking with online multiple instance learning. CVPR 2009.
MS : Comaniciu et. al. Real-time tracking of nonrigid objects using mean shift. CVPR 2000.
![Page 30: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/30.jpg)
Summary
Visual tracking decomposition (VTD) Our method successfully tracks an object
whose motion and appearance change at the same time
Since VTD is easy to extend by adding new features or trackers, our method can be more improved.
![Page 31: Visual Tracking Decomposition](https://reader035.vdocument.in/reader035/viewer/2022062520/568157ad550346895dc53b45/html5/thumbnails/31.jpg)
http://cv.snu.ac.kr/paradiso