mmu faculty smposium_edmond_face_modeling_v5
DESCRIPTION
facial animationTRANSCRIPT
Facial Expressions - Modeling and Animation
Edmond Prakash
Department of Computing and MathematicsManchester Metropolitan University
United Kingdom
Overview
• Introduction• Physics Based Face Animation• Face Adaptation• Subtle Facial Expressions• Summary
IntroductionIntroduction
Virtual characters in film and videoModel-based low-bandwidth teleconferencing Advanced human-computer interactionFace and facial expression recognition
Virtual surgery planning
• Recent interest in facial modeling and animation is spurred by:
• The communicative power of the face makes the realistic modeling of facial expressions an important problem in computer animation and games.
Face Modeling for Movies
Toy Story Final Fantasy
Face Modeling in Games
Tomb RaiderReal Human Face
• Geometrically-based techniques– Key-frame interpolation animation
Parke (1972), Kouadio et al. (1998)– Parameterized models
Parke (1982), Thalmann et al. (1989), DiPaola (1991) – Free form deformation technique Kalra et. al. (1992), Tao and Huang (1998)
• Physically-based models– Models based on the particle system Platt (1981), Lee et al. (1995), Zhang et al. (2001)
– Finite element method Keeve et al. (1996), Koch et al. (1998)
Previous WorkPrevious Work(Facial animation)
Overview
• Introduction• Physics Based Face Animation• Face Adaptation• Subtle Facial Expressions• Summary
Facial Expression Recognition
• Prof. Eric Sung at NTU also came up with a request like this:
• Given several hours of video, how will you efficiently recognize the face of former US president Bill Clinton with a smiling expression?
• We developed a 3D Model based approach.• PhD Thesis of Zhang Yu
Individualized Face Modeling and AnimationIndividualized Face Modeling and Animation
Reconstructed Facial SurfaceReconstructed Facial Surface
Multi-layer MSD Deformable Skin Model
Multi-layer MSD Deformable Skin Model
Generic Skull Model
Physical Facial Muscle ModelsPhysical Facial Muscle Models
Muscle Construction
Muscle Construction
Anatomy-based Personalized Face Model
Anatomy-based Personalized Face Model
Facial Action Editor
Facial Action Editor
ODE SolverODE
Solver
Texture Mapping and
Rendering
Texture Mapping and
Rendering
Synthesized ExpressionsSynthesized Expressions
Skull FittingSkull
Fitting
Facial Data Acquisition & Preprocessing
Facial Data Acquisition & Preprocessing
Range Data
Range Data
Reflectance ImagesReflectance Images View-based Texture
Extraction
View-based Texture
Extraction
Simulation ResultsSimulation Results
Happiness Anger
How do we mathematically define this?
Simulation ResultsSimulation Results
Surprise Sadness Disgust
Expression Simulation on Real Person
Simulation ResultsSimulation Results
System Performance
Computer program can generate happiness easily ie 34.8 frames/sec.Sadness is difficult to synthesize. Needs more muscles and more time.
SO BE HAPPY ALWAYS!
• Examples of the simulated expressions on various personalized facial models.
• Real-time: Physics based Skin, Muscle, Skull
• Examples of the simulated expressions on various personalized facial models.
• Real-time: Physics based Skin, Muscle, Skull
Simulation ResultsSimulation Results
Overview
• Introduction• Physics Based Face Animation• Face Adaptation• Subtle Facial Expressions• Summary
Face model adaptation
• Scanned face not a good mesh.
• Adapt from a template mesh.
• Train the RBF with the corresponding feature points
Template head
Scanned head
RBF and template face adaptation
• What is RBF?
Sourcepoint
Target point
Weight coefficient
Basic function
Rotation, shearing, scaling matrix
Translation vector
Template model adaptation
• Train the RBF with the corresponding feature points
• Solve the linear equation AX=B by LU decomposition
• Apply all the vertex in the template model with the solved weights and R, t to the RBF
Radial basis function result
Adapted head
Overview
• Introduction• Physics Based Face Animation• Face Adaptation• Subtle Facial Expressions• Summary
Subtle facial expressions
• Summer of 2003 • Beckman Institute at UIUC• Prof. Yoshisa Shinagawa • Study the subtle facial expressions.
• Subtle Expressions – Experiments by Ramya Sridharan
How to Quantify Subtle Expressions?(Synthesized variations of happiness )
Laugh Devious smile
Dampened smile Duchenne simle
Observation
frame1 frame 15 Superimposed [(a) + (b)] /2
Dense Optical Flow along U & V directions
Uflow Vflow
Resultant
Experiment 2: Flow analysis for inside features (half face, eyes, one eye, cheek, mouth and lips)
reduce processing time
Optic Flow
U Matrix
V Matrix
UV Matrix: To Quantify Subtle Expressions
+ [U][V]
U1,V1
U2,V2
U3,V3
User text input
Input image
text input from the user and generate the corresponding output face according to the user’s input.
Human Skin Rendering Real vs Synthetic
Taxonomy of Face Animation
YoungFace
Speech to face animation
Speech stream Text stream
Text to face animation
Video to face animation
Face Modeling
Expressions
Expression to face animation
Video stream
Challenges
• Our goal is to use – Face in Games
• Practical Applications– Visual Synthesis of Subtle expressions– Visual Expression Analysis– Expression Cloning (2D images)– Expression Cloning on to 3D Heads– Partial Physical Disabilities & Plastic Surgery
Movie/Game Characters
What is happening elsewhere in 2008?
• SIGGRAPH 2008
• VR 2008
Conclusion• Physics Based Face Animation• Face Adaptation• Subtle Facial Expressions
Face processing still needs lot of research.
If you would like to know more:
Please send me email at: [email protected] visit:
http:/www.docm.mmu.ac.uk/STAFF/E.Prakash