animating (human) motion presented by: –yoram atir –simon adar

C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons”

Animating (human) motion

• Presented by:– Yoram Atir– Simon Adar


Applications of computer animation

• Movies

• Advertising

• Games

• Simulators

• …


General goals of the work presented

- New methods aimed to save time/money/skills needed.

- Study motion (texture).


Agenda

- Basic concepts

- Motion Synthesis/texture using motion capture- Physics/Biomechanics Motion Synthesis

- Cartoon Motion Retargeting.

Basic Concepts

Basic concepts

• Animation world (3D)

• Skeletal model representation

• Model positioning

• Keyframes

• Motion capture

• Frequency bands

• Correlations

Basic Concepts

3D animation world

- (Human) model is animated in Object space- Animated model projected into “global” space- Camera is placed and rotated- Perspective is set- Other…

Basic Concepts

Skeletal representation

- Each model has its own Default Pose

- DOF’s – joint angles/translations relative to Default Pose

- Hierarchical (tree) skeletal representation of model

Picture from Lecture in Computer Graphics course

Department of computer science

University of Washington

Basic Concepts

Creating motion

- Skeletal variations between frames- Overall rotation/Translation between frames- Correlate.

General Problem:

A LOT of work due to the large number of DOFS & high frame rate

Basic Concepts

Figure positioning

- Forward kinematics (simplified): Figure positioning by joint data specification.

Problem:- Tedious trial and error.

Basic Concepts

Figure positioning

Inverse kinematics (simplified)- Joint data is acquired by solving for the final position- In general, This is an optimization problem with a large system

of variables and constraints- Problems often are expressed as minimization problems, and

solved using standard algorithms (gradient decent etc).- Usually, infinite number of possible solutions.- A “good” solution has to be more than “feasible”- Often one is obtained by embedding specific knowledge as

additional constraints, and/or- Using Inverse kinematics as a part of a specific solution.

Basic Concepts

Basic methods for saving labor

Motion captureKeyFrames

Basic Concepts

Keyframes

– Specifying only part of DOFs and frames– Computer interpolation between them

Problem: “smooth” interpolation looks unreal

There are methods to apply “specific noise”

– Term has historical roots

Basic Concepts

Motion capture

– Acquired from “live action”– Copied onto animated character

• Problem: Hard to adapt.• “Motion Editing” – methods to adapt mocap

– Done in studios– Mocap libraries exist

Basic Concepts

Keyframing vs. MocapKeyframing vs. Mocap

Keyframing

Mocap

DisadvantagesAdvantages

•Control

Basic Concepts


Keyframing

Mocap


•Control•Intuitive

Basic Concepts


Keyframing

Mocap



•Detail hard

Basic Concepts


Keyframing

Mocap



•Detail hard•Many DOF

Basic Concepts


Keyframing

Mocap




•Detail easy

Basic Concepts


Keyframing

Mocap




•Detail easy•All DOF

Basic Concepts


Keyframing

Mocap



•No control



Basic Concepts


Keyframing

Mocap



•No control•Not intuitive



Keyframe Data vs. Motion Capture DataKeyframe Data vs.

Motion Capture Data

Basic Concepts

Frequency Bands

Right flat Right toe Left flat Left toe

Basic Concepts

Frequency Bands

• Simplifies the form of the data– Low frequency Variations:

Large scale motions.– Higher frequency variations:

individual “noise” / Jitter

Both are important to preserve in order to capture the essence of motion

Basic Concepts

Correlations

• Joints angle/translation data is related to each other

• Joint angles are correlated over time

• Correlation “plot” is– (somewhat) Specific to the type

of motion– Carries “personality” information

(style)


More information…

INTRODUCTION TO COMPUTER ANIMATION – Rick parent

http://www.cis.ohio-state.edu/~parent/book/outline.html

Splines

http://www.people.nnov.ru/fractal/splines/Intro.html

Hash Inc - Animation software (Movies, tutorials…)

http://www.hash.com

Google…















http://www.hash.com/


Agenda

- Basic concepts


- Cartoon Motion Retargeting

Goal: Motion Capture Assisted Animation


• Create a method that allows an artist low-level control of the motion

• Combine the strengths of keyframe animation with those of mocap

• Create a method that allows an artist low-level control of the motion

• Combine the strengths of keyframe animation with those of mocap

Motion Capture Assisted Animation – Pullen/Bregler



“Sketch” an animation by keyframing• Animate only a few degrees of freedom

• Set few keyframes

“Enhance” the result with mocap data• Synthesize missing degrees of freedom

• Texture keyframed degrees of freedom

“Sketch” an animation by keyframing• Animate only a few degrees of freedom

• Set few keyframes

“Enhance” the result with mocap data• Synthesize missing degrees of freedom

• Texture keyframed degrees of freedom



What is a Motion Texture?

• Every individual’s movement is unique

• Synthetic motion should capture the texture

• To “texture” means to add style to a pre-existing motion

• Technically, texturing is a special case of synthesis



Blue = Keyframed

Purple = Textured/Synthesized


How an Animator WorksHow an Animator Works

• A few degrees of freedom at first

• Not in detail

• Fill in detail with more keyframes later

• A few degrees of freedom at first

• Not in detail

• Fill in detail with more keyframes later


The Method in WordsThe Method in Words

• Choose degrees of freedom to drive the animation

• Compare these degrees of freedom from the keyframed data to mocap

• Find similar regions

• Look at what the rest of the body is doing in those regions

• Put that data onto the keyframed animation

• Choose degrees of freedom to drive the animation

• Compare these degrees of freedom from the keyframed data to mocap

• Find similar regions

• Look at what the rest of the body is doing in those regions

• Put that data onto the keyframed animation


Choices the Animator Must Make

1. Which DOF to use as matching angles

2. Which DOF to texture, which to synthesize

3. Which frequency band to use in matching

4. How many frequency bands to use in texturing

5. How many matches to keep

6. How many best paths to keep






6. How many best paths to keepMotion Capture Assisted Animation – Pullen/Bregler

Before Beginning:Choose Matching Angles

Before Beginning:Choose Matching Angles

Left Hip xLeft Hip yLeft Hip zLeft Knee xLeft Knee yLeft Knee zLeft Ankle xLeft Ankle yLeft Ankle zLeft Ball xLeft Ball yLeft Ball zRight Hip xRight Hip yRight Hip zRight Knee xRight Knee yRight Knee zRight Ankle xRight Ankle yRight Ankle zRight Ball xRight Ball yRight Ball z

Root x transRoot y transRoot z transRoot x rotRoot y rotRoot z rotSpine1 xSpine1 ySpine1 zSpine2 xSpine2 ySpine2 zSpine3 xSpine3 ySpine3 zNeck xNeck yNeck zHead xHead yHead zHead Aim xHead Aim yHead Aim z

Left Clavicle xLeft Clavicle yLeft Clavicle zLeft Shoulder xLeft Shoulder yLeft Shoulder zLeft Elbow xLeft Elbow yLeft Elbow zLeft Wrist xLeft Wrist yLeft Wrist zRight Clavicle xRight Clavicle yRight Clavicle zRight Shoulder xRight Shoulder yRight Shoulder zRight Elbow xRight Elbow yRight Elbow zRight Wrist xRight Wrist yRight Wrist z

Time TimeTime

Matching Angles Drive the SynthesisMatching Angles

Drive the Synthesis




Time TimeTime

Motion Capture DataMotion Capture Data




Time TimeTime

OverviewOverview

Steps in texture/synthesis method

• Frequency analysis

• Matching

• Path finding

• Joining

Steps in texture/synthesis method

• Frequency analysis

• Matching

• Path finding

• Joining


In the following series of slides:

Hip angle = matching angle

Spine angle = angle being synthesized

In the following series of slides:

Hip angle = matching angle

Spine angle = angle being synthesized

Example


Frequency Analysis:Break into Bands


Fre

quen

cy

Time

Band-pass decomposition of matching angles

Keyframed Data Motion Capture Data

Frequency Analysis


Fre

quen

cy

Time


Chosen low frequency band

Frequency Analysis



Hip angle data (a matching angle)

Chosen Low Frequency Band



Making Fragments

Break where first derivative changes sign



Making Fragments

Step through fragments one by one


Matching

KeyframedFragment


Matching

KeyframedFragment

Motion Capture Data


Matching

Compare to all motion capture fragmentsA

ngle

in d

egre

es

Time

KeyframedMocap

Matching

Resample mocap fragments to be same lengthA

ngle

in d

egre

es

Time

KeyframedMocap

MatchingUsing some metric on all matching anglesand on their first derivatives:

Keep the K closest matches

Ang

le in

deg

rees

Time

KeyframedMocap

Matching

KeyframedFragment

Motion Capture Data


Matching

KeyframedFragment

Motion Capture Data

CloseMatches


Matching

Hip Angle (Matching Angle)

Spine Angle (For Synthesis)


Matching and Synthesis

Low frequency hip angle data (a matching angle)

Spine angle data to be synthesized


Possible Synthetic Spine Angle Data

Ang

le in

deg

rees

Time

Path FindingA

ngle

in d

egre

es

Time

We would like to:

• Use as much consecutive fragments as possible

• Stay as close as possible to best fit

Path FindingA

ngle

in d

egre

es

Time

JoiningA

ngle

in d

egre

es

Time

Enhancing Animations:Texturing and SynthesisEnhancing Animations:Texturing and Synthesis

Keyframed

Textured Synthesized

Not keyframed

Texturing

Synthesize upper frequency bands


Texturing

Band-pass decomposition of keyframed dataF

requ

ency

Time

Texturing

Synthesize upper frequency bandsF

requ

ency

Time

Walking animationsTexturing and Synthesis

Keyframed Sketch


Motion Capture Data

Two different styles of walk


Enhanced Animation

Upper body is synthesized

Lower body is textured

Otter Animations: TexturingKeyframed data

Otter Animations: TexturingTextured animation

Dance Animations: Texturing and Synthesis

Lazy Keyframed Sketch


Motion Capture Data


Enhanced AnimationBlue = Keyframed

Purple = Textured/Synthesized

Dance Animations: Texturing

Keyframed Sketch With More Detail

Dance Animations: TexturingTextured Animation

Blue = Keyframed

Purple = Textured

Summary of the Method

Keyframed data

Mocap Data

Keyframed Data

Mocap Data Possible Synthetic Data

Matching Angles

Sketch + Mocap

Frequency Analysis Matching

Path Finding JoiningEnhanced Animation

Choices the Animator Must Make













Conclusions and Applications

• Appropriate for an artist interested in a very particular style of motion

• The artist may have a relatively small motion capture set of that style

• The artist may want precise control over parts of the motion

• Appropriate for an artist interested in a very particular style of motion

• The artist may have a relatively small motion capture set of that style

• The artist may want precise control over parts of the motion

Conclusions and Further Work

• Direct incorporation of hard constraints

• Fundamental units of motion

• Direct incorporation of hard constraints

• Fundamental units of motion

For more info . . .

http://graphics.stanford.edu/~pullen

Special Thanks to:Reardon Steele, Electronic Arts


Agenda

- Basic concepts

- Motion Synthesis/texture using motion capture- Physics, Biomechanics Motion Synthesis


C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations"

Motivation

• Generate rapid prototyping of realistic character motion

• Avoid simulated human models, that are very complex, and don’t always look realistic


Scope

• Highly dynamic movement such as jumping, kicking, running, and gymnastics.

• Less energetic motions such as walking or reaching will not work well in this framework


Overview of the process

Motion sketch

Character description

Motion DB

User interaction

Constraint & phase detection

Transition pose synthesis

Objective functions Optimization Animation

Momentum control



• The objective is to transforms simple animations into realistic character motion by applying laws of physics and the biomechanics domain

• The unknowns are: values of joint angles and parameters of angular and linear momentum