towards human-realistic animation by observing real human dynamics darren cosker towards...
TRANSCRIPT
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Towards Human-Realistic Animation by Observing Real Human Dynamics
Towards Human-Realistic Animation by Observing Real Human Dynamics
Darren CoskerDepartment of Computer Science,University of Bath
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
MotivationMotivation
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
OverviewOverview
• Analysis and Re-Synthesis of Facial Dynamics– Dynamics > CG Puppet– Dynamics > us– Dynamics > Computer Vision
• Intuitive/Automatic methods for animation– Voice Driven Animation/Analysis– Reactive Avatars
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Analysing/Resynthesising DynamicsAnalysing/Resynthesising Dynamics
QuickTime™ and aMPEG-4 Video decompressor
are needed to see this picture.
Time
Par
amet
er M
agni
tude
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Analysing/Resynthesising DynamicsAnalysing/Resynthesising Dynamics
• Study effect of manipulating temporal characteristics of emotions, e.g. smiles
QuickTime™ and a decompressor
are needed to see this picture.
[Emotion 2007]
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Analysing/ResynthesisingDynamicsAnalysing/ResynthesisingDynamics
QuickTime™ and aMPEG-4 Video decompressor
are needed to see this picture.
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Analysing/ResynthesisingDynamicsAnalysing/ResynthesisingDynamics
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Analysing/ResynthesisingDynamicsAnalysing/ResynthesisingDynamics
• What did we learn?– ‘Genuine’ Dynamics = More
likeable/Trustworthy/Employable– ‘Fake’ Dynamics = Less
Likeable/Trustworthy/Employable
• Relationship between Attractiveness and Facial Dynamics
• To Do! - Mock Video Game Scenario
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
1. Define facial regions local
to sub-facial actions
2. Construct new training
set of artificial images
Analysing Multiple ActionsAnalysing Multiple Actions
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Analysing Multiple ActionsAnalysing Multiple Actions
3. Create a corresponding artificial training set of shape vectors
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Performance AnalysisPerformance Analysis
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Image Based TransferImage Based Transfer
QuickTime™ and aH.264 decompressor
are needed to see this picture.
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Performance AnalysisPerformance Analysis
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
3D Model Transfer3D Model Transfer
• Animation of a 3D facial model
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
3D Model Transfer3D Model Transfer
QuickTime™ and aMPEG-4 Video decompressor
are needed to see this picture.
Dynamic knowledge easily applied to existing pipelines
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
3D Facial Modelling3D Facial Modelling
• 4D Camera– Continuous 3D Scanning (Cardiff/Surrey)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
3D Facial Modelling3D Facial Modelling
(Link to 3D Example Expression)
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
3D Facial Modelling3D Facial Modelling
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Voice Driven AnimationVoice Driven AnimationVisual Speech Synthesis
‘Non-Verbal’ Action Synthesis
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aCinepak decompressor
are needed to see this picture.
QuickTime™ and aNone decompressor
are needed to see this picture.
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Perceptual EvaluationPerceptual Evaluation
• Motivation: How best to evaluate facial animation?– Perceptual test based on McGurk effect
QuickTime™ and aSorenson Video decompressorare needed to see this picture.
[ACM TAP 2005]
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Perceptual EvaluationPerceptual Evaluation
• Record participant responses
• Compare real and synthetic
• Subtle - participants blind to real motivation
• Isolate strengths and weaknesses
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Reactive Full Body Avatar AnimationReactive Full Body Avatar Animation
• Motivation:– Animate based on
an input performance
– Key avatar drives other avatar behaviour
QuickTime™ and aSorenson Video 3 decompressorare needed to see this picture.
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Conclusions and Future DirectionsConclusions and Future Directions• Humans are very sensitive to different dynamics
• Different interpretation depending on timing
• Human realism will depend on understanding dynamic behaviours
• Realism will also depend on the accurate modelling and resynthesis of dynamics - e.g. McGurk effect
• Technology now allows study of 3D dynamics - our work has only just started!
Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Thanks for listening!
Acknowledgements:• Eva Krumhuber, Antony Manstead, Lanthao Benedikt,
David Marshall, Paul Rosin, Alistair Calvert, Cathy Holt, Daniel Mason, Gemma Whatling, Simon Rushton, Susan Paddock
• RAEng, EPSRC.