interactive control of avatars animated with human motion data jehee lee carnegie mellon university...

44
Interactive Control of Avatars Interactive Control of Avatars Animated with Human Motion Data Animated with Human Motion Data Jehee Lee Jehee Lee Carnegie Mellon Carnegie Mellon University University Seoul National Seoul National University University Jinxiang Chai Jinxiang Chai Carnegie Mellon Carnegie Mellon University University Paul S. A. Paul S. A. Reitsma Reitsma Brown University Brown University Jessica K. Jessica K. Hodgins Hodgins Carnegie Mellon Carnegie Mellon University University Nancy S. Nancy S. Pollard Pollard Brown University Brown University

Post on 21-Dec-2015

220 views

Category:

Documents


0 download

TRANSCRIPT

Interactive Control of Avatars Interactive Control of Avatars Animated with Human Motion Animated with Human Motion

DataData

Interactive Control of Avatars Interactive Control of Avatars Animated with Human Motion Animated with Human Motion

DataData

Jehee LeeJehee LeeCarnegie Mellon Carnegie Mellon

UniversityUniversity

Seoul National UniversitySeoul National University

Jehee LeeJehee LeeCarnegie Mellon Carnegie Mellon

UniversityUniversity

Seoul National UniversitySeoul National University

Jinxiang ChaiJinxiang ChaiCarnegie Mellon Carnegie Mellon

UniversityUniversity

Paul S. A. Paul S. A. ReitsmaReitsma

Brown UniversityBrown UniversityJessica K. Jessica K. HodginsHodgins

Carnegie Mellon UniversityCarnegie Mellon University

Nancy S. Nancy S. PollardPollard

Brown UniversityBrown University

Avatars:Avatars: Controllable, Controllable, ResponsiveResponsive

Animated CharactersAnimated Characters

• Realistic behaviorRealistic behavior

• Non-trivial environmentNon-trivial environment

• Intuitive user interfaceIntuitive user interface

Interactive Avatar ControlInteractive Avatar ControlInteractive Avatar ControlInteractive Avatar Control

• How to create a rich set of behaviorsHow to create a rich set of behaviors ? ?

• How to direct avatars ?How to direct avatars ?

• How to animate avatar motion ?How to animate avatar motion ?

• How to create a rich set of behaviorsHow to create a rich set of behaviors ? ?

• How to direct avatars ?How to direct avatars ?

• How to animate avatar motion ?How to animate avatar motion ?

MotionDatabase

Preprocess SearchMotionMotionSensorSensorDataData

AnimateAnimateAvatarsAvatars

UserInterface

Motion DatabaseMotion DatabaseMotion DatabaseMotion Database

In video gamesIn video games • Many short, carefully planned, Many short, carefully planned,

labeled motion clipslabeled motion clips

• Manual processingManual processing

In video gamesIn video games • Many short, carefully planned, Many short, carefully planned,

labeled motion clipslabeled motion clips

• Manual processingManual processing

Walk CycleWalk Cycle StopStopStartStart

Left TurnLeft Turn Right TurnRight Turn

Motion DatabaseMotion DatabaseMotion DatabaseMotion Database

Our approachOur approach • Extended, unlabeled sequencesExtended, unlabeled sequences

• Automatic processingAutomatic processing

Our approachOur approach • Extended, unlabeled sequencesExtended, unlabeled sequences

• Automatic processingAutomatic processing

Motion Data AcquisitionMotion Data AcquisitionMotion Data AcquisitionMotion Data Acquisition

Maze - Sketch InterfaceMaze - Sketch InterfaceMaze - Sketch InterfaceMaze - Sketch Interface

Re-sequenceRe-sequenceRe-sequenceRe-sequence

Motion capture region Virtual environment

Obstacles

Sketched path

Re-sequenceRe-sequenceRe-sequenceRe-sequence

Motion capture region Virtual environment

Data AcquisitionData AcquisitionData AcquisitionData Acquisition

““PolesPoles and Holes” rough terrainand Holes” rough terrain““PolesPoles and Holes” rough terrainand Holes” rough terrain

Terrain NavigationTerrain NavigationTerrain NavigationTerrain Navigation

Unstructured Input DataUnstructured Input DataUnstructured Input DataUnstructured Input Data

A number of motion clipsA number of motion clips • Each clip contains many framesEach clip contains many frames

• Each frame represents a poseEach frame represents a pose

A number of motion clipsA number of motion clips • Each clip contains many framesEach clip contains many frames

• Each frame represents a poseEach frame represents a pose

Unstructured Input DataUnstructured Input DataUnstructured Input DataUnstructured Input Data

Connecting transitionConnecting transition • Between similar framesBetween similar frames

Connecting transitionConnecting transition • Between similar framesBetween similar frames

Graph ConstructionGraph ConstructionGraph ConstructionGraph Construction

Distance between FramesDistance between FramesDistance between FramesDistance between Frames

Weighted differencesWeighted differencesof joint anglesof joint angles

Weighted differencesWeighted differencesof joint velocitiesof joint velocities

Pruning TransitionsPruning Transitions

Reduce storage spaceReduce storage space

• O(n^2) will beO(n^2) will be prohibitiveprohibitive

Better qualityBetter quality

• Pruning “bad” transitionsPruning “bad” transitions

Efficient searchEfficient search

• Sparse graphSparse graph

i

j

Pruning TransitionsPruning Transitions

Reduce storage spaceReduce storage space

• O(n^2) will beO(n^2) will be prohibitiveprohibitive

Better qualityBetter quality

• Pruning “bad” transitionsPruning “bad” transitions

Efficient searchEfficient search

• Sparse graphSparse graph

i

j

Pruning TransitionPruning Transition

• Contact stateContact state:: Avoid transition to Avoid transition to dissimilar contact statedissimilar contact state

• LikelihoodLikelihood: User-specified threshold: User-specified threshold

• SimilaritySimilarity: Local maxima: Local maxima

• Avoid dead-endsAvoid dead-ends: Strongly : Strongly connected componentsconnected components

Graph SearchGraph Search

Best-first graph traversalBest-first graph traversal

• Path length is boundedPath length is bounded

• Fixed number of frames at each Fixed number of frames at each frameframe

Comparison to global searchComparison to global search

• Intended for interactive controlIntended for interactive control

• Not for accurate global planningNot for accurate global planning

Comparison to Real MotionComparison to Real MotionComparison to Real MotionComparison to Real Motion

Environment with physical obstaclesEnvironment with physical obstaclesEnvironment with physical obstaclesEnvironment with physical obstacles

Comparison to Real MotionComparison to Real MotionComparison to Real MotionComparison to Real Motion

Global vs. Local CoordinatesGlobal vs. Local Coordinates

Local, moving,Local, moving,

body-relativebody-relative

coordinatescoordinates

Global, fixed,Global, fixed,

object-relativeobject-relative

coordinatescoordinates

User InterfaceUser InterfaceUser InterfaceUser Interface

In maze and terrain In maze and terrain environmentsenvironments

• Sketch interface was effectiveSketch interface was effective

In maze and terrain In maze and terrain environmentsenvironments

• Sketch interface was effectiveSketch interface was effective

User InterfaceUser InterfaceUser InterfaceUser Interface

In playgroundIn playground • A broader variety of motions are availableA broader variety of motions are available

In playgroundIn playground • A broader variety of motions are availableA broader variety of motions are available

Choice InterfaceChoice InterfaceChoice InterfaceChoice Interface

What is available in database ?What is available in database ? • Provide with several optionsProvide with several options

• Select among available behaviorsSelect among available behaviors

What is available in database ?What is available in database ? • Provide with several optionsProvide with several options

• Select among available behaviorsSelect among available behaviors

Choice InterfaceChoice InterfaceChoice InterfaceChoice Interface

What to ShowWhat to ShowSpace and Time WindowsSpace and Time Windows

What to ShowWhat to ShowSpace and Time WindowsSpace and Time Windows

How to Create ChoicesHow to Create ChoicesHow to Create ChoicesHow to Create Choices

ClusteringClusteringClusteringClustering

A A

AB C

C

DD

EE

How to Capture TransitionsHow to Capture TransitionsHow to Capture TransitionsHow to Capture Transitions

A A

AB C

C

DD

EE

How to Capture TransitionsHow to Capture TransitionsHow to Capture TransitionsHow to Capture Transitions

A

AB C

C

DD

EE

AA

Cluster TreeCluster TreeCluster TreeCluster Tree

AB C

C

DD

EE

A B

A

D

C

AA

Three possible actions:Three possible actions: ABA, ABC, ABA, ABC, ABDABD

A

Cluster ForestCluster ForestCluster ForestCluster Forest

A A

AB C

C

DD

EE

D BA C

D

CA

E

B

CA B

A

D

C

Performance InterfacePerformance InterfacePerformance InterfacePerformance Interface

MotionDatabase

Search

Vision-basedInterface

Vision Interface – Single Vision Interface – Single CameraCamera

Vision Interface – Single Vision Interface – Single CameraCamera

SearchSearchSearchSearch3 sec

A A

AB C

C

DD

EE

Video buffer

SearchSearchSearchSearch3 sec

A BA

D

A A

AB C

C

DD

EE

SummarySummarySummarySummary

Graph representationGraph representation• Flexibility in motionFlexibility in motion

Cluster forestCluster forest• A map for avatar’s behaviorA map for avatar’s behavior

User interfacesUser interfaces

Graph representationGraph representation• Flexibility in motionFlexibility in motion

Cluster forestCluster forest• A map for avatar’s behaviorA map for avatar’s behavior

User interfacesUser interfaces

Similarity between FramesSimilarity between Frames

Our WorkArikan & Forsyth

Kovar & Gleicher & Pighin

Joint Angle/Position

Angle Position Position

Pose O O O

Velocity O O Implicitly

Acceleration X Translation Only

Implicitly

Pruning TransitionsPruning Transitions

Our WorkArikan & Forsyth

Kovar & Gleicher &

Pighin

Contact O X X

Likelihood O O O

Similarity O X O

Avoiddead ends

O X O

Related Work Related Work (Character (Character Animation)Animation)

Related Work Related Work (Character (Character Animation)Animation)

Rule-based Control system

Bruderlin & Calvert 96Perlin & Goldberg 96

Chi et al. 00Cassell et al. 01

Hodgins et al. 95Wooten and Hodgins 96

Laszlo et al. 96Faloutsos et al. 01

Example-basedProbabilistic/Statistical

Models

Popovic & Witkin 95Bruderlin & Willams 95

Unuma et al. 95Lamouret & van de Panne 96

Rose et al. 97Wiley & Hahn 97

Gleicher 97, 98, 01Sun & Mataxas 01

Bradley & Stuart 97Pullen & Bregler 00, 02

Tanco & Hilton 00Brand & Hertzmann 00

Galata & Johnson & Hogg 01Arikan & Forsyth 02

Kovar & Gleicher & Pighin 02Li & Wang & Shum 02

(THIS WORK)

Related Work Related Work (User (User Interfaces)Interfaces)

Related Work Related Work (User (User Interfaces)Interfaces)

Graphical User Graphical User InterfacesInterfaces

PerformancePerformance

(Motion (Motion capture capture devices)devices)

PerformancePerformance

(Vision-based)(Vision-based)

Bruderlin & Calvert Bruderlin & Calvert 9696

Laszlo et al. 96Laszlo et al. 96

Rose et al. 97Rose et al. 97

Chi et al. 00Chi et al. 00

Badler et al. 93Badler et al. 93

Semwal et al. 98Semwal et al. 98

Blumberg 98Blumberg 98

Molet et al. 99Molet et al. 99

““Mocap Boxing” Mocap Boxing” (Konami)(Konami)

Blumberg & Galyean Blumberg & Galyean 9595

Brand 99Brand 99

Rosales et al. 01Rosales et al. 01

Ben-Arie et al. 01Ben-Arie et al. 01