prediction of non-linear aging trajectories of faces
DESCRIPTION
Prediction of Non-Linear Aging Trajectories of Faces. K. Scherbaum, M. Sunkel, V. Blanz and H.-P. Seidel [ 2007/5/9, Eurographics 2007, Prague ]. Motivation / Goal. automated growth-prediction system applications photofit-pictures of missing children automated animation, art. 11 years. - PowerPoint PPT PresentationTRANSCRIPT
Prediction of Non-Linear Aging Trajectories of Faces
K. Scherbaum, M. Sunkel, V. Blanz and H.-P. Seidel
[ 2007/5/9, Eurographics 2007, Prague ]
Kristina Scherbaum [email protected]
Motivation / Goal
automated growth-prediction system applications
photofit-pictures of missing children automated animation, art
Kristina Scherbaum [email protected]
9 years
10 years 11 years
Age Progression – Optimal Case
face space
child 1
child 2
child 3
Kristina Scherbaum [email protected]
Real Case – Support Vector Regression
face space
9 years
10 years 11 years
only 1 sample per person no longitudinal study
find isosurfaces and gradients
Runge-Kutta Integration
9 years
10 years
11 years
Kristina Scherbaum [email protected]
Main Assumption - Curved Trajectories
use machine learning non-linear Support Vector Regression
integration of local age-gradient
growing faces transform along curved trajectories
Kristina Scherbaum [email protected]
Challenges
learn change over time of individual facesnon-linear dependency on time, curved trajectory
learn how the change depends on individual facenon-linear dependency in face space
sparse dataset, no longitudinal study
Kristina Scherbaum [email protected]
3D Morphable Facemodel
System is based on a Morphable 3D Facemodel [Blanz,Vetter‘99] Built from 200 3D-face-scans of adults
Kristina Scherbaum [email protected]
3D Morphable Facemodel
vector space of faces
vectors with point-to-point correspondence
Shape
Texture
Kristina Scherbaum [email protected]
Representation of Faces - Face Spaces
PCA to reduce dimensionality (yields coefficients)
Kristina Scherbaum [email protected]
Extended Morphable Model
Extension by … plus ~238 facemodels of teenagers 3 simultaneous laser scans per face
Correspondence by … top-down approach fitting Morphable Model to new 3D faces merging original data and best fit
Kristina Scherbaum [email protected]
Fitting the Morphable Model to 3D Scans
no optical flow because scans are often incomplete
best fit of the morphable model
merged result3D laser scans
Kristina Scherbaum [email protected]
learn function that maps any face x to a scalar age y
to learn this function we use …non-linear Support-Vector-Regression on training sets of l pairs
Age Progression Algorithm1
Kristina Scherbaum [email protected]
Fitting a Regression Curve
for a given set of samples find f(x) such that all samples are within an -tube preselect and tradeoff between smoothness and errors of outliers
2
x
y
Linear: f(x) = wx + b Non-linear: f is sum of Gaussian RBF kernels K(x-xi)
Kristina Scherbaum [email protected]
Gaussian RBF (Radial Basis Function) as kernel
we applied grid search using cross validation to optimize parameters such as (Kernelwidth) i and b are determined by SVM training
using LIBSVM for -Support Vector Regression
Non-Linear SVM Regression2
Kristina Scherbaum [email protected]
Isosurfaces are defined in PCA space Gradient gives shortest path to next isosurface
Along the gradient … many facial changes due to aging almost no other changes
(known technique, Blanz et al. 99)
Thus: Compute growth along the gradient!
Local Aging3
Kristina Scherbaum [email protected]
Gradient Example - Facial Attributes
gender manipulation
original
3
femalemale
Kristina Scherbaum [email protected]
Growth Simulation: New Approach3
growth curve with given face x0 at time t currently we compute the local gradient and walk along this gradient instead we should compute the curved trajectory
Kristina Scherbaum [email protected]
Solve differential equation … to compute curved trajectories integrate the differential equationusing Runge-Kutta algorithmperform small steps
Runge Kutta Integration4
Kristina Scherbaum [email protected]
Visualized Aging Trajectories4
Kristina Scherbaum [email protected]
Reducing Complexity
we did not train on all principle components speedup of SVM training we experimented with 20, 40 or 80 PCs
Justification …
growth leads to overall change of facial size significant changes are represented
by the first PCs [ large variance ] facial growth should happen in the first PCs
4
Kristina Scherbaum [email protected]
Growth Example
growth simulation for both, shape and texture
12 14 16 18 20 years
22 24 26 28 30 years
Kristina Scherbaum [email protected]
More Examples
10 years
15 years
20 years
30 years
12 13 12 10 14
3D laser scans,original
age
Kristina Scherbaum [email protected]
Background, Haircut
Face
Pose, Light
3D reconstruction and aging
Rendering the Result into Images [EG’04]
Composed Result
Kristina Scherbaum [email protected]
Input at the age of 11
Photofit Picture Example
Possible appearances at the age of 17
Kristina Scherbaum [email protected]
Aging in Images - Example
Picture (1999)
Ground truth pictures (2005)
Different prediction renderings
3D reconstruction and aging
Kristina Scherbaum [email protected]
Linear vs. Non-Linear
Linear age progression perform linear regression (yields a function)
[ straight-forward least squares fit ] transform faces also along the gradient
Disadvantages … the gradient is constant [ linear function ] each face moves along
the same straight trajectory
Kristina Scherbaum [email protected]
Linear vs. Non-Linear
comparison of age estimation error (in months) mean squared training and generalization errors
non-linear (RBF)
38.90 29.35
linear 67.66 62.87
non-linear (RBF)
32.68 18.12
linear 66.14 60.05
non-linear SVM regression behaves superior! generalization indicates: no overfitting
Kristina Scherbaum [email protected]
Remember the Challenges
Are growth trajectories curved?Mean angle between start- and target-tangent
the trajectories are curved, not linear
10.3º 30.0º
Have different faces distinct trajectories?Mean angle of trajectories of different faces
15.7º 33.5º
the trajectories are different
Kristina Scherbaum [email protected]
Conclusions
Results … aging involves non-linear components trajectories are distinct for different individuals linear systems are a reasonable approximation technique works without longitudinal data
But … more data would be helpful longitudinal data would allow for exact evaluation
Kristina Scherbaum [email protected]
Representation of Faces - Face Spaces
arbitrary faces by linear combinations of examples
PCA to reduce dimensionality (yields coefficients)
Kristina Scherbaum [email protected]
Main Idea … compute aging trajectories z(t) locally along gradient of the aging function f(x) and going through a start vector or face x0:
Aging Trajectories4
Kristina Scherbaum [email protected]
Aging Information
extracted from the database of 200 adult face scans and new database of 238 face scans of teenagers
teenager overview
Kristina Scherbaum [email protected]