Download - Face/Head Tracking
![Page 1: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/1.jpg)
Face/Head Tracking
Ying [email protected]
Electrical and Computer EngineeringNorthwestern University, Evanston, IL
http://www.ece.northwestern.edu/~yingwu
![Page 2: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/2.jpg)
Tracking Heads?
The task:Localize faces and track them in image sequences
Challenges:Lighting, occlusion, rotation, etc.
Courtesy of Y. Wu, 2001
![Page 3: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/3.jpg)
Outline
Motivation What is tracking? One solution (Birchfield_CVPR98) Other methods and open issues
![Page 4: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/4.jpg)
Motivation
Why tracking?– The complexity of face detection
scan all the pixel positions and several scales
– The limitation of face detection hard to handle out-of-plane rotation
– Can we maintain the identity of the faces? although face recognition is the ultimate solution for this, we
may not need it, if not necessary
Objectives– fast (frame-rate) face/head localization– handle 360o out-of-plane rotation
![Page 5: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/5.jpg)
Visual Tracking
![Page 6: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/6.jpg)
Four Elements Infer target states in video sequences Target states vs. image observations Visual cues and modalities Four elements
– Target representation X– Observation representation Z– Hypotheses measurement p(Zt|Xt)– Hypotheses generating p(Xt|Xt-1)
![Page 7: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/7.jpg)
Visual Tracking
Ground TruthPrediction
HypothesisEstimation
]|[ 1tt ZXE
]|[ tt ZXE
tX
]|[]),|[( 11 ttttt ZXEZZXE
]|[ 11 tt ZXE
![Page 8: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/8.jpg)
Formulating Visual Tracking
ttttttt
tttttt
dXZXpXXpZXp
ZXpXZpZXp
)|()|()|(
)|()|()|(
11
11111
P(Xt|Xt-1)Dynm. Mdl
P(Zt|Xt)Obsrv. Mdl
![Page 9: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/9.jpg)
Tracking as Density Propagation
State space Xt
State space Xt+1
)|( tt ZXp
),|(
)|(
11
11
ttt
tt
ZZXp
ZXp
Posterior
Prob.
Posterior
Prob.
![Page 10: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/10.jpg)
One Solution(Birchfield_CVPR98)
Framework
Search strategy
Edge cue
Color cue
![Page 11: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/11.jpg)
Framework
s = (x,y,) Tracking is treated as a local search based
on the prediction
hypotheses Edge matching
score
color matching
score
![Page 12: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/12.jpg)
Search Strategy
Local exhaustive search
Do you have better ideas?
is the search step size
![Page 13: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/13.jpg)
Edge Cue Method I
Method II
Which is better?
The the magnitude of the gradient at perimeter pixel i of the ellipse s.
# of pixels on the perimeter of the ellipse
unit vector normal to the ellipse at pixel i.
![Page 14: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/14.jpg)
Normalization
Why do we need normalization? How good is it?
![Page 15: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/15.jpg)
Color Cue
Histogram intersection# of bins
Model histogram
![Page 16: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/16.jpg)
Color Cue Color space
– B-G– G-R– R+G+B (why do we need that)
8 bins for B-G and G-R, 4 for R+G+B Training the model histogram Normalization
![Page 17: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/17.jpg)
Comments
Can the rotation be handled? Can the scaling issue be handled? Is the search strategy good enough? Is the color module good? Is the motion prediction enough? Is the combination of the two cues good? Can it handle occlusion? Can it cope with multiple faces
– Coalesce – Switch ID
![Page 18: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/18.jpg)
Other Solutions
Condensation algorithm
3D head tracking
![Page 19: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/19.jpg)
Tracking as Density Propagation
State space Xt
State space Xt+1
)|( tt ZXp
),|(
)|(
11
11
ttt
tt
ZZXp
ZXp
Posterior
Prob.
Posterior
Prob.
![Page 20: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/20.jpg)
Sequential Monte Carlo
P(Xt|Zt) is represented by a set of weighted samples
Sample weights are determined by P(Zt(n)|Xt
(n))
Hypotheses generating is controlled by P(Xt|Xt-1)
![Page 21: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/21.jpg)
Challenge to Condensation
Curse of dimensionality– What to track?
Positions, orientations Shape deformation Color appearance changing
– The dimensionality of X– The number of hypotheses grows exponentially
![Page 22: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/22.jpg)
3D Face Tracking: The Problem
The goal:
Estimate and track 3D head poses
The challenges:
Side view
Back view
Poor illumination
Low resolution
Different users
![Page 23: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/23.jpg)
3D Face Tracking: A Solution
Predictor
Motion Model
Final Pose
Estimator
Cropped Input
Image
Prepro-cessing
Feature Extraction
Ellipsoid Model
Annotated Pose
Courtesy of Y. Wu and K. Toyama, 2000
![Page 24: Face/Head Tracking](https://reader035.vdocument.in/reader035/viewer/2022062808/568152d6550346895dc0f073/html5/thumbnails/24.jpg)
3D Face Tracking: some results