![Page 1: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/1.jpg)
Simulation (three lectures)How used in games?DynamicsCollisions—simple Collisions—harder
detectionbounding boxesresponse
-------------------------------ControllersMocap + simulationUser controlWhat is the future?
![Page 2: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/2.jpg)
Credits
Demos, slides, figures fromMichiel van de Panne (UBC)Michael Mandel’s talk at GDC (CMU alum)Victor Zordan (UC Riverside)Petros Faloutsos (UCLA)
![Page 3: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/3.jpg)
Harder
EasierFalling
DivingBalancing
Tai Chi
Difficult to design complex coordination of limbs
Results can look stiff and unrealistic
More ballistic: not so many DOFsto specify directly
Difficulties of Controller Design
Running
![Page 4: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/4.jpg)
ControlJoint-level Control
pose control—poses specified by artistcontinuous control—tracking mocap or programmer-
specified function
Hierarchical Control (layered)State machine picks low level controller based on sensors
or timingLow level controller controls joints
Combined approaches
![Page 5: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/5.jpg)
Proportional-Derivative (PD) Controller
and are spring and damper gainsis desired joint angle and is current angle
Actuate each joint towards desired target:
Acts like a damped spring attached to joint (rest position at desired angle)
Joint-level Control
![Page 6: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/6.jpg)
Control
What should k’s be?Where does come from?
![Page 7: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/7.jpg)
(see [Zordan ‘02] for more...)
Reduce tuned parameters to a single spring and damper: scale by effective MOI of the chain about the joint
shoulder
elbow
wrist
Perhaps more like natural dynamics of a behavior
Gains are often hand tuned (tedious for 15x2 or more!)
Choosing Controller Gains
![Page 8: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/8.jpg)
Pose Control
Artist selects key poses and dynamics interpolates between them
Very effective but requires patience and tuning
Diving (Wooten)Getting up (Faloutsos)
![Page 9: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/9.jpg)
Complex Behaviors from Simple Behaviors (Faloutsos 01)
Build basic pose controllers
Classify transitions between behaviors
Supervisory controller swaps between them when conditions met
![Page 10: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/10.jpg)
Goal: Keep the center of mass (COM) inside the support polygon
Pick a desired COM and minimize errors by making corrections in the desired angles for ankles and hips
Balance: Programmer specified function
![Page 11: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/11.jpg)
graphics
control
state
forces and torques
desired behavior
user model/machine
numerical integrator
Hierarchical Control
State machineControl actionsLow-level PD servos
![Page 12: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/12.jpg)
![Page 13: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/13.jpg)
![Page 14: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/14.jpg)
![Page 15: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/15.jpg)
![Page 16: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/16.jpg)
![Page 17: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/17.jpg)
![Page 18: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/18.jpg)
![Page 19: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/19.jpg)
![Page 20: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/20.jpg)
![Page 21: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/21.jpg)
![Page 22: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/22.jpg)
![Page 23: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/23.jpg)
Robotics
CMU and MIT 1987, with Marc Raibert
![Page 24: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/24.jpg)
Useful for video games?
• Working on a physical robot is impressive –but is that good enough for a video game?
• Needs to work every time for every input…Or have graceful failure modes.
• And how are we going to do more interesting things than just hopping???
![Page 25: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/25.jpg)
![Page 26: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/26.jpg)
Running Running [[HodginsHodgins ‘‘95]95]
FlightFlight
Heel ContactHeel Contact
LoadingLoading
Foot ContactFoot Contact
Toe ContactToe Contact
UnloadingUnloading
heel touches ground
Knee bend
ball of foot touches groundhip in front of heel
Knee extended
ball of foot leaves ground
QuickTime™ and aYUV420 codec decompressor
are needed to see this picture.
Control Systems for Humans
![Page 27: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/27.jpg)
Simulating Behaviors
SIGGRAPH 1995
![Page 28: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/28.jpg)
Combining Simulation and Mocap
Mocap for trajectory tracking Mocap for control system designMocap -> sim -> mocap
![Page 29: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/29.jpg)
Combining Approaches
Average between balance controller and dataVictor Zordan, PhD thesis
![Page 30: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/30.jpg)
Boxing (with opponent)
![Page 31: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/31.jpg)
Boxing (comparison)
![Page 32: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/32.jpg)
Receive Impact
Data-Driven ControlDynamics Control
Motion DatabaseGet Up
Idle
Run
Jump
Attack
Fall BalanceGrab Onto
Ledge
Simulated Behaviors
SearchMotion
Database
Settle Near
Motion Receive Impact
Data-Driven ControlDynamics Control
Motion DatabaseGet Up
Idle
Run
Jump
Attack
Fall BalanceGrab Onto
Ledge
Simulated Behaviors
SearchMotion
Database
Settle Near
Motion
Mocap -> Sim -> Mocap
![Page 33: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/33.jpg)
State space of data-driven technique:
State space of dynamics-based technique:Set of poses allowable by joint limit constraintsMUCH larger because it:
Any pose in the motion database
Clearly, some correspondence must be made to allow smooth transitions between the two
can produce motion difficult to animate or captureincludes unnatural poses
Executing Transitions
![Page 34: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/34.jpg)
SimulationMotion Data
Simulation Motion Data
Easy. Just initialize simulation with pose and velocities extracted from motion data.
Much harder. How to get near stored data?
Transitions between techniques
![Page 35: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/35.jpg)
1. Data Reduction/RepresentationSearch only some of the keyframes
2. Process into Spatial Data Structure
Problem: Find nearest matches in the motion database to the current simulated motion.
3. Search Structure at Runtime
kd-tree works well
Query pose comes from simulation
Data Representation: Joint positions
Nearest neighbor search problemChoose motion most relevant to in-game situation
![Page 36: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/36.jpg)
Walk Simulated Fall Get UpData-Driven Data-DrivenDynamics
Idle
QuickTime™ and a decompressor
are needed to see this picture.
What’s missing?
![Page 37: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/37.jpg)
Walk Simulated Fall Get UpData-Driven Data-DrivenDynamics
Idle
QuickTime™ and a decompressor
are needed to see this picture.
1. The fall lacks life
2. Transition has blending artifacts
What’s missing?
![Page 38: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/38.jpg)
At the time of the transition the simulation is NOT likely to be in a posture in the motion database
How can we get the simulation to settle near the best matching motion data?
Can we maintain physical constraints between the body and the environment?
(It IS likely, however, to be interacting closely with the environment)
Fixing the Transition
![Page 39: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/39.jpg)
Solution: Settle ControllerActuate joints using a special PD controller to settle the simulation near selected motion data
A physically grounded alternative to blending
Pose controller uses search result as target joint angles
Complex situations might be handled by more specialized controllersCan always finish it off with blending if necessary...
Avoids object interpenetrations and foot sliding...
Fixing the Transition
![Page 40: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/40.jpg)
Adding Life to the falling motion
One Possibility: A Simple Pose ControllerOne Possibility: A Simple Pose ControllerLook at initial conditions of an impact and choose initial desired reaction from a database of example poses
This can work well, but might not be as This can work well, but might not be as dynamic as wedynamic as we’’d like.d like.
May update desired pose as simulation evolves - still totally data-driven (and artist directed)
![Page 41: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/41.jpg)
Adding Life to the falling motion
highly effective motor control strategies hard to model
Reasonably approximate what humans do during a full loss of balance
Possible Approach: Possible Approach: Track predicted shoulder landing locations with armsDirection the body falls determines which arms track
![Page 42: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/42.jpg)
Idle Simulated Fall Get UpData-Driven Data-DrivenDynamics
QuickTime™ and a decompressor
are needed to see this picture.
Idle
Results
![Page 43: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/43.jpg)
Idle Simulated Fall and Roll
Get UpData-Driven Data-DrivenDynamics
Results: fall and roll
![Page 44: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/44.jpg)
[Zordan et al. ‘04]
Physically Based Transitions Following Impacts, With Motion Capture
Apply impact forces to sim
Search to find clip for after interaction
Actively track the motion clip as it transitions, to get the posture in place with joint torques
Add global positions using forces to position character
![Page 45: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/45.jpg)
Physically Based Transitions
Internal torques mimic human reactionExternal forces minimize error while not breaking the physical engine
This method uses mocap while the interaction forces are still active
Doesn’t guarantee a perfect match at the end, but hopefully we can cover this up with blending!
![Page 46: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/46.jpg)
QuickTime™ and aDivX 5.0 decompressor
are needed to see this picture.
![Page 47: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/47.jpg)
Games need to Games need to guaranteeguarantee robustnessrobustness
Start simple—pose controllers with artist predefined reactions
Fake things (like balance control)
Consider simulating only some of the body
Specify only the DOFs necessaryLet the natural dynamics of the system guide the behavior
Games can sacrifice physical realism for robustness/speed—know when using simulation is appropriate!
Make the ground “stickier”External balancing forces to keep the body upright
Making it Practical…
![Page 48: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/48.jpg)
User Control
• High-level control of characters– Velocity -> joystick—treat the character as a
cylinder and assume that there is code to make it follow instructions (run, walk, turn, climbing stairs)
– Button pushes -> discrete actions (kick, punch)
• A few exceptions– Olympic Decathlon on the Apple 2– Motionplayground (demo coming)
• And some failures– Trespasser
![Page 49: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/49.jpg)
Novel user interaction
• http://www.cs.ubc.ca/~van/sssjava/javademo.html
![Page 50: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/50.jpg)
User Control
Are game controls the ultimate 3d interface?
Maybe for gamers…
Stan Melax: http://www.gamasutra.com/features/20010324/melax_pfv.htm
![Page 51: Simulation (three lectures)jkh/video_games/simulation-part3.pdf · Useful for video games? ... Search to find clip for after interaction Actively track the motion clip as it transitions,](https://reader036.vdocument.in/reader036/viewer/2022071004/5fc18f6c8108a2621563429b/html5/thumbnails/51.jpg)
What is the future?
Why don’t we already have fully simulated characters?
Will we ever?What about a world that is “totally” live?