realistic performance-driven facial animation
DESCRIPTION
Realistic Performance-driven Facial Animation. M. Sanchez, J. Edge and S. Maddock. Realism in Facial Animation. Traditional concept: Not necessarily true… The human face is a highly complicated system of which we have rather limited information to build a mechanical simulation - PowerPoint PPT PresentationTRANSCRIPT
Realistic Performance-driven
Facial Animation
M. Sanchez, J. Edge and S. Maddock
Realism in Facial Animation
Traditional concept:
Not necessarily true… The human face is a highly complicated system of which we have
rather limited information to build a mechanical simulation We mainly perceive the visual effects of such processes, not the
mechanisms underneath
Physically-based approaches have issues of their own: Stability is difficult to assess, convergence impossible Most numerical methods scale up worse than linearly wrt the
complexity of the model The problem is connaturally stiff (linear elasticity >> resistance to
bend and shear), making it even more numerically sensitive
Conclusion: it’s questionable whether this concept of realism can lead to more plausible results than mere anatomically-driven geometric systems (eg. Waters)
realism
simulating the biomechanical processes that drive facial motion
Capturing Realism
One sensible alternative:
Intrinsic problems: Only a limited domain of information of actual facial motion is
accessible through capture (spatially discrete / timewise discrete) The captured data is bound to the physiognomy of the actual subject
Geometric deformation techniques can be used to extrapolate spatially discrete motion (eg. markers) to the full extent of facial skin
Can be consistent with large-scale properties of skin deformation Oversee fine detail aspects of skin mechanics (furrows / creases)
Retargeting of facial motion is possible using purely geometric criteria (without an anatomic model)
Small skin deformation can be modelled as a parallel process to large-scale motion
Capturing from actual performances the visual clues that make facial motion perceptually
plausible
System overview
System overview
System overview
System overview
System overview
System overview
1. Large-scale deformation
1.3. Animating the skin
1.3. Animating the skin
Large-scale aspects of skin mechanics to be reproduced:
Motion is propagated across the connectivity of the skin
Underlying layers of tissue dampen the extents of deformation
Local smoothness is generally preserved
Conventional techniques for spatial interpolation of motion do not fully satisfy these criteria
Point-based deformations (Williams, Pelachaud, Kshirsagar)
Volumetric techniques (Kalra, Escher) RBFs (Fidaleo)
Similar approaches could work if interpolation was performed along geodesic curves on the surface, but that would only work for 2-manifolds, and the skin has openings
1.3. Animating the skin
Through surface-to-surface mappings we can consistently interpolate motion across approximations of the mesh to be deformed
BIDS does precisely that… Motion propagates naturally across a
piecewise surface whose patch topology is compatible with that of the face model
Smoothness and locality of deformation can be asserted through the construction and conditioning of such approximating surface
For further details: “Realistic Performance-driven Facial
Animation using HW Acceleration” (submitted to ToG)
Thesis (soon to come ;)
1.2. Adapting facial motion
1.2. Adapting facial motion
The relation between source and target motion is highly non-linear, both with relation to its point of application and its extent and direction
We model this by constructing an RBF-based mapping between the embedding of source and target control surfaces…
capable of representing the two kinds of non-linearities
lacking of boundary separation issues due to the application of BIDS
stable under the usual scale of facial motion
For further details: “Use and re-use of facial MoCap” VVG’03
Different physiognomies Different scale of motion
Key fact:
Conventional MoCap retargeting cannot be applied to soft body deformation
1.1. Labelling the target mesh
Basic idea:
Our approach to this model consists on deforming the ref. mask to fit the target mesh, and then using the resulting control surface
this is solved as an iterative process, minimizing the following energy function:
the elastic energy terms correspond to the classic definition by Terzopoulos, and they have analytic solutions for Bezier Triangles
non-analytic terms require using blind methods (eg simplex downhill) or forward differences for their spatial derivatives
1.1. Labelling the target mesh
Transplanting with minimal distortion the labelling of the
ref. mask onto the target mesh
RR S
bsaverts
Tv dSBkSkAkcvvwc1
1. Large-scale deformation
2. Fine tissue detail
2. Fine tissue detail
2.a. Fitting a model to photographs
Motivation: A model of the face is needed to act as
support of the normal fields captured in 2.b
We need to analyse the ratio of deformation for each of the expressions in 2.c
We proceed by retrieving the position of markers on the face of the subject with reverse stereometry from 3 viewpoints…
and then deform the reference mask using BIDS:
2.a. Fitting a model to photographs
2.b. Shape from shading
Consider the simplified BDRF equation:
without the specular component, it would be linear…
use complementary light polarizers on lights and camera lenses, or
apply translucent makeup on the subject (cheap option)
With a set of 3 exposures to distinct lighting conditions, N is fully determined for any given pose (Rushmeier)
The variation of the normal field (N) is then encoded in the tangent space of the fitted mesh, producing normal maps
2.b. Shape from shading
nsdinout kkII NHNL
2.cd. Deriving a model of wrinkling
2.cd. Deriving a model of wrinkling
The fitted model in every expression not only acts as reference for normal maps, but also allows measuring the ratio of deformation on the face by approximating its strain tensor with that on the control surface
represents both magnitude and direction of infinitesimal strain
compression (negative strain) can be isolated by chopping off negative eigenvalues
Compression is ultimately the cause of facial tissue wrinkling, however…
elastic response is not the same in every region, neither in every direction of strain
linear compression is much more relevant than shearing
We have a sampling representing these properties (normal maps), together with the corresponding compression tensors, so we can build a model relating the two:
it’s a polynomial approximation of N wrt the 3 components of the symmetric form of the tensor
experimentation reveals linear degree suffixes
For any displaced configuration of the control surface, we can evaluate the model and compute the corresponding normal map
Results
Results
Question time…