motion editing (geometric and constraint-based methods) jehee lee
TRANSCRIPT
Motion Editing(Geometric and Constraint-Based Methods)
Jehee Lee
Animation from Motion Capture
• Motion capture provides a large collection of realistic motion data– What can we do with this data set ?
• Applications– Animation authoring: Film, music video, TV ad., …– Interactive characters: Video games, VR, …– Live performance: Computer puppetry
MotionDatabase
PreprocessOn-line
Controller
Motion EditingToolbox
MotionSensor
Data
CustomAnimation
ControllableCharacters
MappingLive
Performance
High-Level UserInterfaces
The Art ofAnimation
ComputerPuppetry
Animation Authoring
• Animators select motion segments from the database
• Motion segments will be modified, blended, and stitched– Animators need a variety of motion editing tools !
MotionDatabase
PreprocessMotion Editing
Toolbox
MotionSensor
Data
CustomAnimation
The Art ofAnimation
Motion Retargetting
[Gleicher 98]
Interactive Characters
• Users interact (or control) with computer-generated characters through high-level user interfaces
• System selects appropriate motions from the database– User interfaces and control algorithms are required
MotionDatabase
Preprocess On-lineController
MotionSensor
Data
ControllableCharacters
High-Level UserInterfaces
Sketch Interface
Computer Puppetry
• Motion sensor data is mapped to an animated character in realtime– No database is required
Motion Representation
• Configuration of an articulated figure– Linear components:– Angular components:
3)( Rp t3)( Sq ti
)(
)(
)(
)(
)( 1
0
t
t
t
t
t
nq
q
q
p
m
The position of the root segment
The orientations of body segmentsw.r.t. their parents (joint angles)
The orientation of the root segment
Simple Operations
• Translation
• Rotation
)(
)(
)(
)(
)(
)(
)(
)(
)()(' 2
1
2
1
t
t
t
t
t
t
t
t
tt
nn
t
q
q
q
vp
0
0
0
v
q
q
q
p
dmm
)(
)(
)exp()(
)(
)(
)(
)(
)(
)()(' 2
1
2
1
t
t
t
t
t
t
t
t
tt
nn
r
q
q
vq
p
0
0
v
0
q
q
q
p
dmm
Simple Operations
• Exercise joints
• Motion displacements– independent translation and rotation
)(
)exp()(
)(
)(
)(
)(
)(
)(
)()('
11
t
t
t
t
t
t
t
t
tt
n
i
n
i
j
q
vq
q
p
0
v
0
0
q
q
q
p
dmm
))(exp()(
))(exp()(
)()(
)(
)(
)(
)(
)(
)(
)()()(' 11
0
1
0
1
tt
tt
tt
t
t
t
t
t
t
ttt
nnnn vq
vq
vp
v
v
v
q
q
p
dmm
Motion Displacement
• Rigid Transformation at the root segment
)exp(
)exp(
)(
)(
)(
)(
)(
)(
)()()(' 11
1101
1
0
1
nnnn t
t
t
t
t
t
ttt
vq
vq
pqvq
v
v
v
q
q
p
dmm
Motion Displacement
• Rigid Transformation at the root segment
)'log(
)'log(
))()('(
)(
)(
)(
)('
)('
)('
)()(')(
1
11
1
11
1
11
nnnn
tt
t
t
t
t
t
t
ttt
qppq
q
q
p
q
q
p
mmd
Simple Operations
• Valid operations
• Invalid operations
)! careful Be()()()(
)()(
)()()(
)()()(
321
21
12
21
ttt
tt
ttt
ttt
ddd
dd
dmm
mdm
?)(
?)()( 21
t
tt
m
mm
Simple Operations
• Time warping
• Properties
))(()(' tst mm
)()()()()()(
)()()()(
2121 tttttt
tttt
ddmddm
mddm
Transition Graph
Stand
Right footforward
Start
Stop
TurnRTurnL
Loop
Transition Graph
Connecting Motion Segments
• Alignment and warping
First Motion
Second MotionRotate and translatethe second motion
to align two motions
Warp the motionsat the boundary
so that they can beconnected smoothly
Alignment
• The end of one motion A should be aligned to the beginning of the next motion B– The root location of the end of A:– The root location of the beginning of B:
– Apply to motion B, that is
),( An
An qp
),( 00BB qp
)(
)(
)exp()(
)()()(
)(
)(
)(
)(
)( 2
1
111
2
1
t
t
t
ttt
t
t
t
t
t
nn
B
q
q
vq
puqq
0
0
v
u
q
q
q
p
dm
B
B
An
An
0
0
q
p
q
p
v
u
Alignment Using In-Plane Transformation
• Rigid transformation restricted within a plane– Rotation about the vertical axis, followed by– Translation along two horizontal axes
• How can we find in-plane rotation from arbitrary rotation ?– Euler angles
• Discard rotation about x- and z-axes
zyx RRRR
Optimal In-Plane Transformation
• Closest point from a geodesic curve to a point– The geodesic curve represents a set of orientation that can be
reached by rotating about the vertical axis
Anp
B0p
BB yyG 00 )sinˆ,(cos),ˆ( pp
Optimal In-Plane Transformation
Point Cloud Matching
Warping
• Deform a motion smoothly so that it is seamlessly connected its previous motion
• A scalar transition function s(t)
First Motion Second Motiond
BAn 0mmd
dm )()( tstB
)(tS
t
1
Motion Database
• Reusability and flexibility– Motion capture is not cheap !– Motion capture data doesn’t always reflect our
intention• There is no live performer who can move like an ant or
Godzilla
Antz (1998) Godzilla (1998)
Motion Database
• Reusability and flexibility– Motion capture is not cheap !– Motion capture data doesn’t always reflect our
intention• We are not always able to construct an appropriate physical
environment in a mocap studio
Motion Database
• Reusability and flexibility– Motion capture is not cheap !– Motion capture data doesn’t always reflect our
intention• There is no live performer who can move like an ant or
Godzilla• We are not always able to construct an appropriate physical
environment in a mocap studio
– We sometimes want to reuse existing data rather than capturing new motion whenever needed
• Video games, interactive characters
Adaptation/Retargetting/Editing
• Reusing motion is challenging, because the motion was acquired– For a specific character– Within a specific environment– In a specific style and mood
Constraint-Based Approaches
• Constraints specify– Features to be retained– New features to be accomplished
• Find a new motion– Satisfy given constraints– While preserving the characteristics of the original
motion as much as possible
• The core of constraint-based approaches can be formulated as an optimization problem
Constraints
• Preserve important features of the original motion– Contact with environments (foothold positions)– Balance (dynamic constraints)– Subtle details (frequency bands)
• Accomplish new features for the target motion– Interactive manipulation– New characters/environment– New style/mood (How do you define?)
Constraint-Based Approaches
Inter- and Intra-Frame Coherency
Within each frame• Enforce constraints• By inverse kinematics
Inter-frame coherence• Avoid discontinuity• By using smooth splines
Motion and Constraints
Original Motion
How would we warp the motion to meet the constraints ?
Motion and Constraints
Original Motion
Bad warping
Displacement Mapping
• Motion warping [Popovic & Witkin 95]
• Motion displacement mapping [Bruderlin & Williams 95]
Displacement Map
Original Motion
Target Motion
Displacement Mapping
)exp(
)exp(
)()('
11
0
1
0
1
nnnn
tt
vq
vq
vp
v
v
v
q
q
p
dmm
Original Motion
Displacement map
Non-Uniform Spline Interpolation
• How would you create a smooth curve (displacement map) that interpolates given joint displacements ?– Spline interpolation– You already know how to do it !
• Constraints are distributed non-uniformly– Variational constraints (constraints that stand over a time period) are
discretized as a sequence of constraints
• Small details are added for constrained intervals– Details (high frequency) of the original motion could be
destroyed
• IK solution is not always coherent– Small change of start poses and constraints doesn’t necessarily
yeild small change of output poses
Least-Squares Curve Fitting
• We would like to discard outliers– Approximation rather than interpolation
• We don’t want to add unnecessary high-frequency– Uniform splines
• IK solving and curve fitting should be applied alternatively for several times
• The linear system could be partially over-specified and partially under-specified
Non-Linear Optimization
• Formulation [Gleicher 98]
– Curve approximation and IK can be combined into a single large non-linear optimization problem
– Objective function: minimize distortion
– Parameters• All joint angles at all time instances (It is a huge problem !)• Joint angles are often represented by B-splines
– in order to reduce the number of parameters– in order to maintain inter-frame coherency
dttdtttG )()()( 0 dmm
Adaptive Refinement
• Shape freedom vs. Smoothness
• Hard to determine the density of knots• Multi-level (hierarchical) B-splines
– Adaptive refinement of knot sequences– Sum of uniform B-splines with different knot spacings– Coarse-to-fine hierarchy
Hierarchical Displacement Mapping
0m
1d 2d
10 mm 210 mmm
)))((( hh dddmm 210
Algorithm
Examples
• Adaptation to New Environments
Examples
• Adaptation to New Characters
Examples
• Character Morphing
Adapting Motion for Humanoids
• Different structures and DOFs
[Pollard 2002]
Adapting Motion for Humanoids
• Different structures and DOFs– Adaptation algorithm is similar to IK with
multiple goals– Gimble lock avoidance– Teapot demo by Pollard (2002)– Jello demo by Choi (2004)
Adapting Motion for Humanoids
• Different structures and DOFs– Adaptation algorithm is similar to IK with
multiple goals– Gimble lock avoidance
• Joint limit– Inequality constraints– Similar to collision avoidance
Inequality Constraints
Upper bound
Lower bound
Inequality Constraints
Upper bound
Lower bound
Inequality Constraints
Upper bound
Lower bound
Hierarchical displacement Hierarchical displacement mapping can be usedmapping can be used
Inequality Constraints
Upper bound
Lower bound
Much better in terms of motion quality
Adapting Motion for Humanoids
• Different structures and DOFs
• Joint limit
• Dynamic limit– Velocity, torque, balance, …
Physically Based Motion Transformby Popovic (1999)
• Normal run: run-norm.mov• Stiff leg: run-stiff.mov• Low gravity: run-moon.mov• High gravity: run-neptune.mov
Motorized Marionetteby Yamane (2003)
Marionette• 6 motors for hands and 2 motors for legs
• Limited kinematic range
• Limited dynamic capability
• Avoid undesired swing
• Cockrobin
• Alaska
Summary
• Motion data processing– Reusability and flexibility– Efficient and correct– A Toolbox of motion editing tools
• Motion adaptation/retargeting/editing– Displacement mapping
• It is just spline interpolation/approximation• It can be formulated as a non-linear optimization problem
– Hierarchical displacement mapping• Adaptive refinement is desired