motion editing and retargetting
DESCRIPTION
Motion Editing and Retargetting. Jinxiang Chai. Outline. Motion editing [video, click here ] Motion retargeting [video, click here ]. Required Readings. Comparing Constraint-Based Motion Editing Methods Retargeting Motion to New Characters . Kinematic Motion Editing. - PowerPoint PPT PresentationTRANSCRIPT
Motion Editing and Retargetting
Jinxiang Chai
Outline
• Motion editing [video, click here]
• Motion retargeting [video, click here]
Required Readings
• Comparing Constraint-Based Motion Editing Methods
• Retargeting Motion to New Characters
Kinematic Motion Editing
• Goal: Edit an input motion sequence to achieve new goals specified by the user
Kinematic Motion Editing
• Goal: Edit an input motion sequence to satisfy achieve new goals specified by the user
e.g., edit the motion to meet new constraints
Kinematic Motion Editing
• Goal: Edit an input motion sequence to satisfy achieve new goals specified by the user
e.g., edit the motion to meet new constraints- could be multiple constraints
Toy Examples
• Edit 1D function to match user constraints
Old motion M={m(t)| t=1,…,T}
t
m(t)
Toy Examples
• Edit 1D function to match user constraints
t
m(t)
Old motion M={m(t)| t=1,…,T}
Toy Examples
• Edit 1D function to match user constraints - could be multiple constraints
t
m(t)
Old motion M={m(t)| t=1,…,T}
Toy Examples
• Edit 1D function to match user constraints - could be multiple constraints
Old motion m(t), t=1,…,T
t
m(t)
So how can we generate a new motion M’ to satisfy new constraints c?
Toy Examples: Idea #1
• Modify the motion m(t), t=1,…,T using new constraints C
t
M(t)
Old motion m(t), t=1,…,T
Toy Examples: Idea #1
• Modify the motion m(t), t=1,…,T using new constraints C
t
m’(t)
Does this work?
Toy Examples: Idea #1
• Modify the motion m(t), t=1,…,T using new constraints C
t
m’(t)
Does this work? - probably not, the new motion is not smooth!
Toy Examples: Idea #1
• Modify the motion m(t), t=1,…,T using new constraints C
t
M’(t)
Does this work? - probably not, the new motion is not smooth! - filter the motion!
Toy Examples: Idea #1
• Modify the motion m(t), t=1,…,T using new constraints C
• Filter the edited motion M’
t
M’(t)
Does this work? - probably no, the new motion is not smooth! - filter the motion! - but the new motion now does not meet cons!
Toy Examples: Idea #1
• Modify the motion m(t), t=1,…,T using new constraints C
• Filter the edited motion M’
t
M’(t)
Does this work? - probably no, the new motion is not smooth! - filter the motion! - repeat modification and filtering process!
Idea #1: Modification & Filter
Another Example: Three Constraints
Another Example: Three Constraints
Idea #1: Summary
• Pros - easy to implement - computationally efficient• Cons - does not handle complex constraints, e.g., constraint
functions involved more than one frames
Old motion M={m(t)| t=1,…,T}
t
m(t)
t1 t2
e.g., m’(t1)-m’(t2)=5
Idea #1: Summary
• Pros - easy to implement - computationally efficient• Cons - does not handle complex constraints, e.g., constraint
functions involved more than one frames
How to address this limitation?
Idea #2: Constrained Optimization
• Modify the motion M to satisfy new constraints C
Old motion m(t), t=1,…,T
t
m(t)
Idea #2: Constrained Optimization
• Modify the motion M to satisfy new constraints C
Old motion m(t), t=1,…,T
t
m(t)
But this is an ill-posed problem - There are tons of solutions that satisfy the new constraints C
Idea #2: Constrained Optimization
• Modify the motion M to satisfy new constraints C
Old motion m(t), t=1,…,T
t
m(t)
But this is an ill-posed problem - There are tons of solutions that satisfy the new constraints C - Remove the ambiguity by picking the solution with a minimal change across an entire motion
Idea #2: Constrained Optimization
• Modify the motion M to satisfy new constraints C
Old motion m(t), t=1,…,T
t
m(t)
2
,...,1),(' )()('minarg tmtmTttm
CMf )(
Idea #2: Constrained Optimization
• Modify the motion M to satisfy new constraints C
Old motion m(t), t=1,…,T
t
m(t)
2
,...,1),(' )()('minarg tmtmTttm
CMf )(
Minimize the changes
Satisfy new constraints
Idea #2: Constrained Optimization
• We can also formulate this as an unconstrained optimization problem
Old motion m(t), t=1,…,T
t
m(t)
22
,...,1),(' )()()('minarg CMftmtmTttm
Idea #2: Constrained Optimization
• This is also called as spacetime optimization or trajectory optimization
• Simultaneously compute the entire motion
Old motion m(t), t=1,…,T
t
m(t)
22
,...,1),(' )()()('minarg CMftmtmTttm
Idea #2: Constrained Optimization
• This is also called as spacetime optimization or trajectory optimization
• Simultaneously compute the entire motion
Old motion m(t), t=1,…,T
t
m(t)
22
,...,1),(' )()()('minarg CMftmtmTttm
Now Back to Human Motion Editing!
A sequence of poses: q1,q2,…qT
Each pose is represented as a high-dimensional vector qt: Rn
Motion trajectoriesPose qt Motion q1,…qT
Human motion representation
Human Motion Editing
• Human motion is a high-dimensional function
• Constraints could be any kinematic constraints
User Constraints
Any kinematic constraints throughout the motion
Position cons.
Orientation cons.
Distance cons.
Joint angle cons.
Solution #1: Per-frame IK+Filtering
• Step 1: Per-frame inverse kinematics to modify poses in each frame with new constraints.
- this might be ill-posed - remove ambiguity by minimizing the difference between old
poses and new poses
• Step 2: motion filtering.
• Repeat step 1 and 2 until the solution converges
Solution #2: Spacetime Motion Editing
• Consider all constraints simultaneously
• Use optimization to solve the entire motion
22
,...,1),(' )()()('minarg CMftmtmTttm
Solution #2: Spacetime Motion Editing
• Consider all constraints simultaneously
• Use optimization to solve the entire motion
22
,...,1),(' )()()('minarg CMftmtmTttm
- Currently, minimizing joint angle poses across the entire sequence- A better criterion is possible!
Solution #2: Spacetime Editing
• Better Criterion? - Geometric constraints such as contact constraints are often
more important
Solution #2: Spacetime Editing
• Better Criterion? - Frequency Content or details
Solution #2: Spacetime Motion Editing
22
,...,1),(' )()()('minarg CMftmtmTttm
Implementation details:
- use Spline representation to reduce the solution space
- often computationally expensive
- optimization over sub windows when working on a long motion sequence
Motion Retargeting
• One motion, a cast of characters
• Focus on similar structure
• Formulated as motion editing problem
Motion Retargeting: Key Idea
22
,...,1),(' )()()('minarg CsMftmtm newTttm ;
Skeleton of new characters
• Step 1: Extract constraints/properties C from source motion
Motion Retargeting: Key Idea
22
,...,1),(' )()()('minarg CsMftmtm newTttm ;
Extracted constraints
• Step 1: Extract constraints/properties C from source motion
Motion Retargeting: Key Idea
22
,...,1),(' )()()('minarg CsMftmtm newTttm ;
Extracted constraintsRetargeted motion Source motion
• Step 1: Extract constraints/properties C from source motion
Motion Retargeting: Key Idea
22
,...,1),(' )()()('minarg CsMftmtm newTttm ;
Extracted constraintsRetargeted motion Source motion
Skeleton of new characters
• Step 2: Apply to new characters
Motion Retargeting: Key Idea
• Step 3: Approximate answer
Motion Retargeting: Key Idea
22
,...,1),(' )()()('minarg CsMftmtm newTttm ;
How to initialize m’(t)?
• Step 4: Solve constraints using spacetime optimization
Motion Retargeting: Key Idea
22
,...,1),(' )()()('minarg CsMftmtm newTttm ;