animating (human) motion presented by: –yoram atir –simon adar

C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons”

Animating (human) motion

• Presented by:– Yoram Atir– Simon Adar

Applications of computer animation

• Movies

• Advertising

• Games

• Simulators

• …

General goals of the work presented

- New methods aimed to save time/money/skills needed.

- Study motion (texture).

Agenda

- Basic concepts

- Motion Synthesis/texture using motion capture- Physics/Biomechanics Motion Synthesis

- Cartoon Motion Retargeting.

Basic Concepts

Basic concepts

• Animation world (3D)

• Skeletal model representation

• Model positioning

• Keyframes

• Motion capture

• Frequency bands

• Correlations

Basic Concepts

3D animation world

- (Human) model is animated in Object space- Animated model projected into “global” space- Camera is placed and rotated- Perspective is set- Other…

Basic Concepts

Skeletal representation

- Each model has its own Default Pose

- DOF’s – joint angles/translations relative to Default Pose

- Hierarchical (tree) skeletal representation of model

Picture from Lecture in Computer Graphics course

Department of computer science

University of Washington

Basic Concepts

Creating motion

- Skeletal variations between frames- Overall rotation/Translation between frames- Correlate.

General Problem:

A LOT of work due to the large number of DOFS & high frame rate

Basic Concepts

Figure positioning

- Forward kinematics (simplified): Figure positioning by joint data specification.

Problem:- Tedious trial and error.

Basic Concepts

Figure positioning

Inverse kinematics (simplified)- Joint data is acquired by solving for the final position- In general, This is an optimization problem with a large system

of variables and constraints- Problems often are expressed as minimization problems, and

solved using standard algorithms (gradient decent etc).- Usually, infinite number of possible solutions.- A “good” solution has to be more than “feasible”- Often one is obtained by embedding specific knowledge as

additional constraints, and/or- Using Inverse kinematics as a part of a specific solution.

Basic Concepts

Basic methods for saving labor

Motion captureKeyFrames

Basic Concepts

Keyframes

– Specifying only part of DOFs and frames– Computer interpolation between them

Problem: “smooth” interpolation looks unreal

There are methods to apply “specific noise”

– Term has historical roots

Basic Concepts

Motion capture

– Acquired from “live action”– Copied onto animated character

• Problem: Hard to adapt.• “Motion Editing” – methods to adapt mocap

– Done in studios– Mocap libraries exist

Basic Concepts

Keyframing vs. MocapKeyframing vs. Mocap

Keyframing

DisadvantagesAdvantages

•Control

Basic Concepts

Keyframing

•Control•Intuitive

Basic Concepts

Keyframing

•Detail hard

Basic Concepts

Keyframing

•Detail hard•Many DOF

Basic Concepts

Keyframing

•Detail easy

Basic Concepts

Keyframing

•Detail easy•All DOF

Basic Concepts

Keyframing

•No control

Basic Concepts

Keyframing

•No control•Not intuitive

Keyframe Data vs. Motion Capture DataKeyframe Data vs.

Motion Capture Data

Basic Concepts

Frequency Bands

Right flat Right toe Left flat Left toe

Basic Concepts

Frequency Bands

• Simplifies the form of the data– Low frequency Variations:

Large scale motions.– Higher frequency variations:

individual “noise” / Jitter

Both are important to preserve in order to capture the essence of motion

Basic Concepts

Correlations

• Joints angle/translation data is related to each other

• Joint angles are correlated over time

• Correlation “plot” is– (somewhat) Specific to the type

of motion– Carries “personality” information

(style)

More information…

INTRODUCTION TO COMPUTER ANIMATION – Rick parent

http://www.cis.ohio-state.edu/~parent/book/outline.html

Splines

http://www.people.nnov.ru/fractal/splines/Intro.html

Hash Inc - Animation software (Movies, tutorials…)

http://www.hash.com

Google…

Agenda

- Basic concepts

- Cartoon Motion Retargeting

Goal: Motion Capture Assisted Animation

• Create a method that allows an artist low-level control of the motion

• Combine the strengths of keyframe animation with those of mocap

• Create a method that allows an artist low-level control of the motion

• Combine the strengths of keyframe animation with those of mocap

Motion Capture Assisted Animation – Pullen/Bregler

“Sketch” an animation by keyframing• Animate only a few degrees of freedom

• Set few keyframes

“Enhance” the result with mocap data• Synthesize missing degrees of freedom

• Texture keyframed degrees of freedom

“Sketch” an animation by keyframing• Animate only a few degrees of freedom

• Set few keyframes

“Enhance” the result with mocap data• Synthesize missing degrees of freedom

• Texture keyframed degrees of freedom

What is a Motion Texture?

• Every individual’s movement is unique

• Synthetic motion should capture the texture

• To “texture” means to add style to a pre-existing motion

• Technically, texturing is a special case of synthesis

Blue = Keyframed

Purple = Textured/Synthesized

How an Animator WorksHow an Animator Works

• A few degrees of freedom at first

• Not in detail

• Fill in detail with more keyframes later

• A few degrees of freedom at first

• Not in detail

• Fill in detail with more keyframes later

The Method in WordsThe Method in Words

• Choose degrees of freedom to drive the animation

• Compare these degrees of freedom from the keyframed data to mocap

• Find similar regions

• Look at what the rest of the body is doing in those regions

• Put that data onto the keyframed animation

• Choose degrees of freedom to drive the animation

• Compare these degrees of freedom from the keyframed data to mocap

• Find similar regions

• Look at what the rest of the body is doing in those regions

• Put that data onto the keyframed animation

Choices the Animator Must Make

1. Which DOF to use as matching angles

2. Which DOF to texture, which to synthesize

3. Which frequency band to use in matching

4. How many frequency bands to use in texturing

5. How many matches to keep

6. How many best paths to keep

6. How many best paths to keepMotion Capture Assisted Animation – Pullen/Bregler

Before Beginning:Choose Matching Angles

Left Hip xLeft Hip yLeft Hip zLeft Knee xLeft Knee yLeft Knee zLeft Ankle xLeft Ankle yLeft Ankle zLeft Ball xLeft Ball yLeft Ball zRight Hip xRight Hip yRight Hip zRight Knee xRight Knee yRight Knee zRight Ankle xRight Ankle yRight Ankle zRight Ball xRight Ball yRight Ball z

Root x transRoot y transRoot z transRoot x rotRoot y rotRoot z rotSpine1 xSpine1 ySpine1 zSpine2 xSpine2 ySpine2 zSpine3 xSpine3 ySpine3 zNeck xNeck yNeck zHead xHead yHead zHead Aim xHead Aim yHead Aim z

Left Clavicle xLeft Clavicle yLeft Clavicle zLeft Shoulder xLeft Shoulder yLeft Shoulder zLeft Elbow xLeft Elbow yLeft Elbow zLeft Wrist xLeft Wrist yLeft Wrist zRight Clavicle xRight Clavicle yRight Clavicle zRight Shoulder xRight Shoulder yRight Shoulder zRight Elbow xRight Elbow yRight Elbow zRight Wrist xRight Wrist yRight Wrist z

Time TimeTime

Matching Angles Drive the SynthesisMatching Angles

Drive the Synthesis

Time TimeTime

Motion Capture DataMotion Capture Data

Time TimeTime

OverviewOverview

Steps in texture/synthesis method

• Frequency analysis

• Matching

• Path finding

• Joining

Steps in texture/synthesis method

• Frequency analysis

• Matching

• Path finding

• Joining

In the following series of slides:

Hip angle = matching angle

Spine angle = angle being synthesized

In the following series of slides:

Hip angle = matching angle

Spine angle = angle being synthesized

Example

Frequency Analysis:Break into Bands

Band-pass decomposition of matching angles

Keyframed Data Motion Capture Data

Frequency Analysis

Chosen low frequency band

Frequency Analysis

Hip angle data (a matching angle)

Chosen Low Frequency Band

Making Fragments

Break where first derivative changes sign

Making Fragments

Step through fragments one by one

Matching

KeyframedFragment

Matching

KeyframedFragment

Motion Capture Data

Matching

KeyframedFragment

Motion Capture Data

Matching

Compare to all motion capture fragmentsA

KeyframedMocap

Matching

Resample mocap fragments to be same lengthA

KeyframedMocap

MatchingUsing some metric on all matching anglesand on their first derivatives:

Keep the K closest matches

KeyframedMocap

Matching

KeyframedFragment

Motion Capture Data

Matching

KeyframedFragment

Motion Capture Data

CloseMatches

Matching

Hip Angle (Matching Angle)

Spine Angle (For Synthesis)

Matching and Synthesis

Low frequency hip angle data (a matching angle)

Spine angle data to be synthesized

Possible Synthetic Spine Angle Data

Path FindingA

We would like to:

• Use as much consecutive fragments as possible

• Stay as close as possible to best fit

Path FindingA

JoiningA

Enhancing Animations:Texturing and SynthesisEnhancing Animations:Texturing and Synthesis

Keyframed

Textured Synthesized

Not keyframed

Texturing

Synthesize upper frequency bands

Texturing

Band-pass decomposition of keyframed dataF

Texturing

Synthesize upper frequency bandsF

Walking animationsTexturing and Synthesis

Keyframed Sketch

Motion Capture Data

Two different styles of walk

Enhanced Animation

Upper body is synthesized

Lower body is textured

Otter Animations: TexturingKeyframed data

Otter Animations: TexturingTextured animation

Dance Animations: Texturing and Synthesis

Lazy Keyframed Sketch

Motion Capture Data

Enhanced AnimationBlue = Keyframed

Purple = Textured/Synthesized

Dance Animations: Texturing

Keyframed Sketch With More Detail

Dance Animations: TexturingTextured Animation

Blue = Keyframed

Purple = Textured

Summary of the Method

Keyframed data

Mocap Data

Keyframed Data

Mocap Data Possible Synthetic Data

Matching Angles

Sketch + Mocap

Frequency Analysis Matching

Path Finding JoiningEnhanced Animation

Choices the Animator Must Make

Conclusions and Applications

• Appropriate for an artist interested in a very particular style of motion

• The artist may have a relatively small motion capture set of that style

• The artist may want precise control over parts of the motion

• Appropriate for an artist interested in a very particular style of motion

• The artist may have a relatively small motion capture set of that style

• The artist may want precise control over parts of the motion

Conclusions and Further Work

• Direct incorporation of hard constraints

• Fundamental units of motion

• Direct incorporation of hard constraints

• Fundamental units of motion

For more info . . .

http://graphics.stanford.edu/~pullen

Special Thanks to:Reardon Steele, Electronic Arts

Agenda

- Basic concepts

- Motion Synthesis/texture using motion capture- Physics, Biomechanics Motion Synthesis

C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations"

Motivation

• Generate rapid prototyping of realistic character motion

• Avoid simulated human models, that are very complex, and don’t always look realistic

• Highly dynamic movement such as jumping, kicking, running, and gymnastics.

• Less energetic motions such as walking or reaching will not work well in this framework

Overview of the process

Motion sketch

Character description

Motion DB

User interaction

Constraint & phase detection

Transition pose synthesis

Objective functions Optimization Animation

Momentum control

• The objective is to transforms simple animations into realistic character motion by applying laws of physics and the biomechanics domain

• The unknowns are: values of joint angles and parameters of angular and linear momentum

Motion sketch

Motion DB

User interaction

Momentum control

Constraint and stage detection

• Each input sequence has two parts:– The part that needs to

be improved – The part that needs to

kept intacked

• Automatically extract the positional and sliding constrains

Positional constraint detection

• A positional constraint fixes a specific point on the character to a stationary location for a period of time

• We need to find if all these points lie on a line, plane

• In an articulated character we find the constraints on each body part

Positional constraint detection

• The algorithm looks for fixed points (point, line, plane)

iii xxT

0)( ii xIT

Sliding constraints

iilp lpWTDist ),(min ,

Motion sketch

Motion DB

User interaction

Momentum control

Transition pose generation

• A transition pose separates constrained and unconstrained stages.

• Two possibilities:– We ask the animator

to draw the transition poses

– We have an estimator to suggest a transition pose

Transition Pose Estimator• DB contains examples of different motions• The input of that DB are the motion

parameters like: flight distance, flight height, takeoff angle, landing angle, spin angle..

• The DB has a simplified representation of the transition poses by three COM’s

• We use IK to obtain the full character’s pose from those three COM’s

• The KNN - K nearest neighbor algorithm• The pose estimator predicts the candidate

pose by interpolating the KNN with the weights that describe the similarity to the input.

2AB )C(C

Motion sketch

Motion DB

User interaction

Momentum control

• Transition poses constrain the motion at few key points of the animation

• Dynamic constraints ensure realistic motion of each segment

• Linear and angular momentum give us these dynamic constraints

Momentum during unconstrained and constrained stages

• linear momentum - During “flight” the only force is gravity

• Angular momentum - During “flight” there is no change in Angular momentum

• During “ground” stage we avoid computing the momentums and use empirical characteristics

Motion sketch

Motion DB

User interaction

Momentum control

Objective functions

• There are three Objective functions, the basic idea behind them is power consumption– Minimum mass displacement– Minimal velocity of DOFs– Static balance

Motion sketch

Motion DB

User interaction

Momentum control

Putting it all together

• Environment constraints (Ce)• Transition pose constraints (Cp)• Momentum constraints (Cm)• Q are character’s DOFs

subject to )(min i

Motion sketch

Motion DB

User interaction

Momentum control

Some Results• Wide variety of figures: male, female, child• 51 DOFs • The body dimensions and mass distribution is taken from

biomechanics literature• In some of the cases the animator selects the body parts to be

constraints• The animator can change relative timing between each phase• The optimization was solved by using SNOPT a general

nonlinearly-constrained optimization package• The optimization time depends on the duration of the animation• All of the simple animation took less than five minutes to sketch • For all examples the synthesis process took less than five

minutes

Broad jump

• Only 3 keyframes at takeoff, peak and landing

Running

• The angular momentum constraint creates a counter-body movement by the shoulders and arms to counteract the angular momentum generated by the legs.

• Keyframing 7 DOFs

Hopscotch

• Each hop requires 3 keyframes and has fewer than 7 DOFs

Handspring

• There were no handstands within the DB so the user had to modify the result

High-bar

• Two constraints stages: the bar and ground

Karate kick

• A second synthesis add a keyframe in the peak

Twist jumps

Agenda

- Basic concepts

What is Cartoon Capture & Retargeting

• Cartoon Capture– Track the motion From

2D Animation– Represent the motion

& save

• Retargeting– Translate the motion

representation to another output media

Digitized video

Key shapes

Cartoon capture

Output corresponding

key shapes

Motion representation retargeting

Output video

Cartoon motion capture & retargeting scheme

Modeling Cartoon motion

Digitized video

Key shapes

Cartoon capture

key shapes

Output video

• Two types of deformations– Affine deformation

– Key shape deformation

Affine Deformation

• Affine parameters

daaSwarpV

],,,,,[)( 4321 yx ddaaaat

Tiii yxs 1

Key-Shape Deformation

• Sk are the key shapes

daaSwarpV

3w1w 2w

• In total there are 6+K variables that represent the motion

],....,,,,,,,[)( 14321 kyx wwddaaaat

Cartoon motion capture

Digitized video

Key shapes

Cartoon capture

key shapes

Output video

Cartoon motion capture

• contour capture: the input is a sequence of contours

• video capture: the input is the video sequence

contour capture

• Two step minimization:– Find Affine parameters

– Find Key-Shape weights

• Iterate

1 ),...,,( kSSwarpVErr

daaVErr

x SSSVdaa

21)( kk

daaVErr

Retargeting

Digitized video

Key shapes

Cartoon capture

key shapes

Output video

Retargeting

• For each Input key-shape an Output key-shape is drawn.

Retargeting Process

Key shapes Interpolation

Apply Affine transformation

From motion capture

Retargeted media

Retarget Motion capture

Examples

Additional constrains & post processing

• Undesirable effects may still appear

• Determine constraints that force the character go through certain position at certain time

• Apply ad-hoc global transformation that fulfill these constraints

Performance

• Quantative performance wasn’t mentioned

• The more complex the motion of the character is, the more key-shapes are needed

• Many of the animations contain jitter, but the overall exaggerated motion dominates

animating (human) motion presented by: –yoram atir –simon adar

basic concepts keyframing

basic concepts basic

basic concepts keyframes

basic concepts motion

hard slide

dof slide

easy slide

historical roots slide

Documents

a01-aliasstudio 2008 animating

animating mr. market

selectively de-animating video

styling & animating svgs

animating rotations and using quaternions. what we’ll talk...

animating with quaternions

animating the interactive experience

animating multiple turtles

atir site - weehawken-nj.us

animating attributes

yoram kanyuk on avner less

animating poststructuralism

biofloc yoram avnimelech 2

animating to music

yoram (jerry) wind

animating the salmon river

animating yourcoursebook

animating human athletes

animating spaceland

animating a postmodern ramayana