Automating Graph-Based Motion Automating Graph-Based Motion SynthesisSynthesis

Lucas KovarLucas Kovar

Michael GleicherMichael Gleicher

University of Wisconsin-MadisonUniversity of Wisconsin-Madison

Working with Large Data SetsWorking with Large Data Sets

Goal: convert motion data into a usable form.

Can we automate this?– Must preserve realism and provide control

Motion model User

specifications

MotionAutomated process

Motion data

Graph-Based SynthesisGraph-Based Synthesis

Motion data is static clips of fixed length. How can we make long sequences of motion?

walkEdge = clip

Node = choice point

Graph walk = motion

stand

stop

start walking

Typically one uses move trees, built by hand.

How can we automate this?

OutlineOutline

• Motion graphs

• Snap-together motion

OutlineOutline

• Motion graphs

– Building motion graphs

– Using motion graphs

• Snap-together motion

Motion GraphsMotion Graphs

Idea: automatically add transitions to a data set.

Quality: Only transition when motions are similar (but wherever they are similar).

Motion 1

Motion 2

Motion 1

Motion 2

Control: Search for “optimal” sequences of edges

Related WorkRelated Work

• Statistical models

– Brand and Hertzmann ’01, Li et al. ’02

– Looser quality guarantees, less focus on control

• Other motion representations

– Simulation (Lamouret and van de Panne ’96)

– Video (Shödl et al. ’01, ‘02)

• Concurrent work

– Arikan and Forsyth ’02, Lee et al. ‘02

Motion Graphs: An ExampleMotion Graphs: An Example

Adding TransitionsAdding Transitions

For arbitrary motions, transitions are hard.

Create transitions where motions are similar.

?

Finding Similar FramesFinding Similar Frames

• Need derivatives (velocity, acceleration, etc.)

• Joint angles are hard to compare directly

• Must account for coordinate invariance

– Effect of perturbation (e.g., rotate shoulder) depends on pose

– Different camera ≠ different motion!

Distance MetricDistance Metric

Derivative information

Align coordinate systems“Body shape”, not joint angles

Initial frames

Finding Transition PointsFinding Transition Points

Every pair of frames now has a distance.

Transitions are local minima below a threshold.

Motion 2 FramesM

otio

n 1

Fra

me

s

Finding Transition PointsFinding Transition Points

Transition thresholds control quality vs. flexibility tradeoff.

Threshold = 0 cm Threshold = 8 cm Threshold = 16 cm

Creating TransitionsCreating Transitions

Motion data is interpolated with C1 continuity.

Initial signals

Interpolation weights Result

Constraints are also “blended” and enforced as a post-process (Kovar et al. ’02).

OutlineOutline

• Motion graphs

Building motion graphs

– Using motion graphs

• Snap-together motion

Interacting With Motion GraphsInteracting With Motion Graphs

Any sequence of edges is a motion!

Control is harder – motion graphs are complex.

Find graph walks that minimize a cost function.

Searching for MotionSearching for Motion

Branch-and-bound speeds the search.

Incrementally search for the optimal motion.

Path SynthesisPath Synthesis

Goal: extract motion that follows a path.

Can also restrict motion style.

User’s path ( )

Motion’s path ( )

Minimize

2)()(

iii sPsP

P

P

ResultsResults

OutlineOutline

Motion graphs

• Snap-together motion

Motion Graphs: Advantages/DrawbacksMotion Graphs: Advantages/Drawbacks

But, their complicated structure is problematic.

– Must use costly search methods

– Synthesized motion is only a “best fit”

– Graph is hard to reason about

Motion graphs allow one to create lengthy, complicated motions with little effort.

Structure vs. Unstructured GraphsStructure vs. Unstructured Graphs

Snap-Together Motion (STM)Snap-Together Motion (STM)

Idea: help build graphs with simple structure.

– Control: small number of “hub” nodes

– Quality: smooth transitions, constraints enforced

– Automation: build hubs just by selecting a pose

STM: PreviewSTM: Preview

STM: OverviewSTM: Overview

Find groups of similar frames (match sets) and make multi-way transitions

1. Pick a “seed” frame or have the system suggest one

2. Find similar frames and add displacement maps so motions are identical

3. Ensure constraints remain enforced

STM: OverviewSTM: Overview

Snappable MotionSnappable Motion

Original MotionOriginal Motion

…

Synthesized MotionSynthesized Motion

Creating Match SetsCreating Match Sets

• Given a pose, find similar poses– Build distance grid (same metric as before)

– Pose = row; find 1D local minima below threshold

1 2 3 4

1 2 3 4

Making TransitionsMaking Transitions

Goal: make each frame in the match set identical.

Add displacement maps so each frame has the average pose and velocity.

Making Transitions(cont.)Making Transitions(cont.)

Original motion Posture fitting Velocity fitting

Use a two-level displacement map

– Coarse knots for pose

– Denser knots for velocity

Enforcing ConstraintsEnforcing Constraints

Two-step process to preserve pose at each hub1. Enforce constraints on frames in the match set.

2. Fix these frames, enforce in rest of each clip.

Problem: conflicting constraints

Hub 1 Hub 2

Solution: group hub nodes with conflicts and find single constraint position over group.

Foot positions don’t match!

constraint spans clip

ResultsResults

ConclusionConclusion

Simplify graph-based synthesis by automating the identification and tuning of transitions.

Different strategies different kinds of graphs

– Motion graphs: “greedy” strategy is general and highly automated, but synthesis requires search

– STM: clustering makes simpler graphs for improved control, but assumes hub poses exist.