motion editing and retargetting

Motion Editing and Retargetting

Jinxiang Chai

Outline

• Motion editing [video, click here]

• Motion retargeting [video, click here]

Required Readings

• Comparing Constraint-Based Motion Editing Methods

• Retargeting Motion to New Characters

Kinematic Motion Editing

• Goal: Edit an input motion sequence to achieve new goals specified by the user

Kinematic Motion Editing

• Goal: Edit an input motion sequence to satisfy achieve new goals specified by the user

e.g., edit the motion to meet new constraints

Kinematic Motion Editing

• Goal: Edit an input motion sequence to satisfy achieve new goals specified by the user

e.g., edit the motion to meet new constraints- could be multiple constraints

Toy Examples

• Edit 1D function to match user constraints

Old motion M={m(t)| t=1,…,T}

t

m(t)

Toy Examples

• Edit 1D function to match user constraints

t

m(t)

Old motion M={m(t)| t=1,…,T}

Toy Examples

• Edit 1D function to match user constraints - could be multiple constraints

t

m(t)

Old motion M={m(t)| t=1,…,T}

Toy Examples

• Edit 1D function to match user constraints - could be multiple constraints

Old motion m(t), t=1,…,T

t

m(t)

So how can we generate a new motion M’ to satisfy new constraints c?

Toy Examples: Idea #1

• Modify the motion m(t), t=1,…,T using new constraints C

t

M(t)

Old motion m(t), t=1,…,T

Toy Examples: Idea #1

• Modify the motion m(t), t=1,…,T using new constraints C

t

m’(t)

Does this work?

Toy Examples: Idea #1

• Modify the motion m(t), t=1,…,T using new constraints C

t

m’(t)

Does this work? - probably not, the new motion is not smooth!

Toy Examples: Idea #1

• Modify the motion m(t), t=1,…,T using new constraints C

t

M’(t)

Does this work? - probably not, the new motion is not smooth! - filter the motion!

Toy Examples: Idea #1

• Modify the motion m(t), t=1,…,T using new constraints C

• Filter the edited motion M’

t

M’(t)

Does this work? - probably no, the new motion is not smooth! - filter the motion! - but the new motion now does not meet cons!

Toy Examples: Idea #1

• Modify the motion m(t), t=1,…,T using new constraints C

• Filter the edited motion M’

t

M’(t)

Does this work? - probably no, the new motion is not smooth! - filter the motion! - repeat modification and filtering process!

Idea #1: Modification & Filter

Another Example: Three Constraints

Another Example: Three Constraints

Idea #1: Summary

• Pros - easy to implement - computationally efficient• Cons - does not handle complex constraints, e.g., constraint

functions involved more than one frames

Old motion M={m(t)| t=1,…,T}

t

m(t)

t1 t2

e.g., m’(t1)-m’(t2)=5

Idea #1: Summary

• Pros - easy to implement - computationally efficient• Cons - does not handle complex constraints, e.g., constraint

functions involved more than one frames

How to address this limitation?

Idea #2: Constrained Optimization

• Modify the motion M to satisfy new constraints C

Old motion m(t), t=1,…,T

t

m(t)

Idea #2: Constrained Optimization

• Modify the motion M to satisfy new constraints C

Old motion m(t), t=1,…,T

t

m(t)

But this is an ill-posed problem - There are tons of solutions that satisfy the new constraints C

Idea #2: Constrained Optimization

• Modify the motion M to satisfy new constraints C

Old motion m(t), t=1,…,T

t

m(t)

But this is an ill-posed problem - There are tons of solutions that satisfy the new constraints C - Remove the ambiguity by picking the solution with a minimal change across an entire motion

Idea #2: Constrained Optimization

• Modify the motion M to satisfy new constraints C

Old motion m(t), t=1,…,T

t

m(t)

2

,...,1),(' )()('minarg tmtmTttm

CMf )(

Idea #2: Constrained Optimization

• Modify the motion M to satisfy new constraints C

Old motion m(t), t=1,…,T

t

m(t)

2

,...,1),(' )()('minarg tmtmTttm

CMf )(

Minimize the changes

Satisfy new constraints

Idea #2: Constrained Optimization

• We can also formulate this as an unconstrained optimization problem

Old motion m(t), t=1,…,T

t

m(t)

22

,...,1),(' )()()('minarg CMftmtmTttm

Idea #2: Constrained Optimization

• This is also called as spacetime optimization or trajectory optimization

• Simultaneously compute the entire motion

Old motion m(t), t=1,…,T

t

m(t)

22

,...,1),(' )()()('minarg CMftmtmTttm

Idea #2: Constrained Optimization

• This is also called as spacetime optimization or trajectory optimization

• Simultaneously compute the entire motion

Old motion m(t), t=1,…,T

t

m(t)

22

,...,1),(' )()()('minarg CMftmtmTttm

Now Back to Human Motion Editing!

A sequence of poses: q1,q2,…qT

Each pose is represented as a high-dimensional vector qt: Rn

Motion trajectoriesPose qt Motion q1,…qT

Human motion representation

Human Motion Editing

• Human motion is a high-dimensional function

• Constraints could be any kinematic constraints

User Constraints

Any kinematic constraints throughout the motion

Position cons.

Orientation cons.

Distance cons.

Joint angle cons.

Solution #1: Per-frame IK+Filtering

• Step 1: Per-frame inverse kinematics to modify poses in each frame with new constraints.

- this might be ill-posed - remove ambiguity by minimizing the difference between old

poses and new poses

• Step 2: motion filtering.

• Repeat step 1 and 2 until the solution converges

Solution #2: Spacetime Motion Editing

• Consider all constraints simultaneously

• Use optimization to solve the entire motion

22

,...,1),(' )()()('minarg CMftmtmTttm

Solution #2: Spacetime Motion Editing

• Consider all constraints simultaneously

• Use optimization to solve the entire motion

22

,...,1),(' )()()('minarg CMftmtmTttm

- Currently, minimizing joint angle poses across the entire sequence- A better criterion is possible!

Solution #2: Spacetime Editing

• Better Criterion? - Geometric constraints such as contact constraints are often

more important

Solution #2: Spacetime Editing

• Better Criterion? - Frequency Content or details

Solution #2: Spacetime Motion Editing

22

,...,1),(' )()()('minarg CMftmtmTttm

Implementation details:

- use Spline representation to reduce the solution space

- often computationally expensive

- optimization over sub windows when working on a long motion sequence

Motion Retargeting

• One motion, a cast of characters

• Focus on similar structure

• Formulated as motion editing problem

Motion Retargeting: Key Idea

22

,...,1),(' )()()('minarg CsMftmtm newTttm ；

Skeleton of new characters

• Step 1: Extract constraints/properties C from source motion

Motion Retargeting: Key Idea

22

,...,1),(' )()()('minarg CsMftmtm newTttm ；

Extracted constraints

• Step 1: Extract constraints/properties C from source motion

Motion Retargeting: Key Idea

22

,...,1),(' )()()('minarg CsMftmtm newTttm ；

Extracted constraintsRetargeted motion Source motion

• Step 1: Extract constraints/properties C from source motion

Motion Retargeting: Key Idea

22

,...,1),(' )()()('minarg CsMftmtm newTttm ；

Extracted constraintsRetargeted motion Source motion

Skeleton of new characters

• Step 2: Apply to new characters

Motion Retargeting: Key Idea

• Step 3: Approximate answer

Motion Retargeting: Key Idea

22

,...,1),(' )()()('minarg CsMftmtm newTttm ；

How to initialize m’(t)?

• Step 4: Solve constraints using spacetime optimization

Motion Retargeting: Key Idea

22

,...,1),(' )()()('minarg CsMftmtm newTttm ；

Motion Retargeting

• Youtube video (click here)