statistical models for face analysis · statistical models for face analysis 4th stochastic...
TRANSCRIPT
Statistical models for face analysis
4th Stochastic geometry days, August 2015
Pascal Bourdon
XLIM Laboratory (UMR CNRS 7252) / Université de Poitiers
2 8/27/2015
Summary
• Introduction
• Early works
• A deformable face model
• Model fitting and data representation
• Conclusion
Statistical models for face analysis
3 8/27/2015
Summary
• Introduction
• Early works
• A deformable face model
• Model fitting and data representation
• Conclusion
Statistical models for face analysis
4
Introduction
8/27/2015
Why face analysis?
5
Introduction
8/27/2015
Why face analysis?
6
Introduction
8/27/2015
Why face analysis?
7
Introduction
8/27/2015
Why face analysis?
• Identity recognition
• Morphing/VFX
• Aging simulation
• Fatigue monitoring (road safety)
• Behavior analysis (reactions, emotions…)
• Virtual mirror (online shopping: makeup, glasses…)
• Visio-conference, videos games…
8
Introduction
8/27/2015
A challenging task…
Pose
Illumination
Expression
Resolution
9 8/27/2015
Summary
• Introduction
• Early works
• A deformable face model
• Model fitting and data representation
• Conclusion
Statistical models for face analysis
Matching a reference image (template)
Object tracking associates image objects in consecutive video frames
10
Early works: object tracking/image alignment
8/27/2015
),(,,,',' yxtyxIttyxI
Matching a reference image (template)
Image alignment associates target objects from a reference
11
Early works: object tracking/image alignment
8/27/2015
),(,',' yxyxTyxI
Both problems are traditionally solved using optical flow estimation
• Common assumption: flow is constant within the image object I(x,y)
• Example: least mean squares method (Lucas-Kanade)
12
Early works: object tracking/image alignment
8/27/2015
ttVyyVxxItyxI ,,,,
Vy
Vxυ
υ),,(),,( tyxItyxI T
t
),(
2),,(),,(
yx
T
t tyxItyxI υυ
0ˆ
υ
υ
Lucas-Kanade tracker
13
Early works: object tracking/image alignment
8/27/2015
14
Early works: object tracking/image alignment
8/27/2015
A constant flow within the image object?
15
Early works: object tracking/image alignment
8/27/2015
A constant flow within the image object?
),(
2
,,yx
warptemplate
yxIyxT WW
yyz
xzx
tysxw
tywxsyx,W
Matthews-Baker ICIA tracker
16
Early works: object tracking/image alignment
8/27/2015
17
Early works: object tracking/image alignment
8/27/2015
A constant flow within the image object?
Rigid objects : 3D-to-2D projections
Non-rigid objects : a deformable model is needed!
18 8/27/2015
Summary
• Introduction
• Early works
• A deformable face model
• Model fitting and data representation
• Conclusion
Statistical models for face analysis
From rigid to deformable models
We want to enhance a previously static alignment model (i.e. template image) to
take into account changes of identity, expression or luminosity. In computer
vision, these changes are seen as local variations of shape and/or texture
19
A deformable face model
8/27/2015
Active Appearance Models (Cootes et al.)
are able to jointly synthesize an image (texture) and the shape it relates to. An
AAM model is built through principal component analysis (PCA) of aligned texture
and shape data from a (manually annotated) image database
20
A deformable face model
8/27/2015
21
A deformable face model
8/27/2015
Active Appearance Models (Cootes et al.)
are able to jointly synthesize an image (texture) and the shape it relates to. An
AAM model is built through principal component analysis (PCA) of aligned texture
and shape data from a (manually annotated) image database
22
A deformable face model
8/27/2015
Active Appearance Models (Cootes et al.)
are able to jointly synthesize an image (texture) and the shape it relates to. An
AAM model is built through principal component analysis (PCA) of aligned texture
and shape data from a (manually annotated) image database
23
A deformable face model
8/27/2015
Face synthesis from an AAM model
sP
p
pp
1
0 sss
gP
p
pp
1
0 AAAShape model: Texture model:
24 8/27/2015
Summary
• Introduction
• Early works
• A deformable face model
• Model fitting and data representation
• Conclusion
Statistical models for face analysis
Fitting a model to new data
• Original AAM (Cootes) : generating shape samples from the database
with random noise and learning a regression model which minimizes the
error between observed and synthesized textures
• Gradient-descent method (Matthews-Baker) : a least mean squares-
based approach inspired by Lucas-Kanade, where AAM parameters are
estimated together with global transform parameters
25
Model fitting and data representation
8/27/2015
),(
2
,;,;,,,yx
warptemplate
yxIyxA ΦpWΨΦpΨ
ΦΨ, : AAM parameters p : global (e.g. affine) transform parameters
W: image warping function, computed from all geometry-related parameters
Fitting a model to new data
26
Model fitting and data representation
8/27/2015
27
Model fitting and data representation
8/27/2015
Further data analysis from estimated AAM parameters
Motion capture and animation
28
Model fitting and data representation
8/27/2015
Further data analysis from estimated AAM parameters
Emotion analysis (e.g. online learning)
29 8/27/2015
Summary
• Introduction
• Early works
• A deformable face model
• Model fitting and data representation
• Conclusion
Statistical models for face analysis
30 8/27/2015
Conclusion
• A statistical face model learned from observed data
• Compacts relevant information into just a few parameters
• Applications in the field of information retrieval
• Current work : polynomial representation of texture model
and applications in animation and e-learning
• Next : force semantic information into PCA analysis i.e.
individual muscle activity parameters (local constraints)
Statistical models for face analysis
31 8/27/2015
Thank you !
Applications in alignment, tracking and landmarking