Agents and Avatars
Ruth Aylett
Overview
! Agents and Avatars ! Believability v naturalism ! Building the body ! H-anim ! Moving and sensing
IVAS
! Intelligent Virtual Agents (IVAs) – Also:
• Synthetic characters • Embodied conversational characters (ECAs) • Virtual humans • BUT do not have to be humanoid…
– Embodied and autonomous • Require a control architecture - or ‘agent mind’ • Insides v outsides: combining AI with graphics
– ‘Inhabit’ a virtual environment
Why IVAs?
! Adding life to a VE – Animals, birds, insects – Crowds
• E.g. students in the virtual campus – Increase sense of presence
! As a guide or teacher – Front-end to embedded knowledge in a VE
! As a character in a story – Computer games
! Interface agents – On web pages to make them more human – Personal representative – Sales rep
Why IVAs? - 2
! As virtual actors – Instead of extras – Immersive Education Media Stage
! As part of a simulation – Hostage release training – Battlefield medical training
! Scientific investigation – Buildings evacuation – Testing for disability friendliness – Ecology and animal behaviour
Avatars
! Hindu Sanskrit Term - “Representation of a Deity in visible form” – Snow Crash (Neil Stephenson 1992) - “A
graphical representation of yourself”
! VE representation of the user – Embodiment need not be humanoid
! Driven by the user – So NOT autonomous – A mapping rather than a control problem
Video avatars
! “magic video mirror” with back-projection
! video image mixed with computer generated overlay
Creating a video avatar (ALIVE, MIT)
• unencumbered interaction between human visitor and virtual character based on position, postures, and gestures
• main focus on “virtual presence” http://www.ai.univie.ac.at/oefai/agents/
Believability
! Term introduced by Joe Bates of the OZ group at CMU in the 1990s – Combined art and technology
! Very hard to define – A willing suspension of disbelief? – Seem like ‘real’ characters? – The ‘illusion of life’? – Willingness to attribute an internal state
• Ascribe intentionality
Believability and Naturalism
! Are they the same thing? ! Graphics people seem to think so
– Is Mickey a real mouse? – Is he a believable character?
The uncanny valley
! The ‘uncanny valley’ – Work by a Japanese
researcher, Mori – Acceptability v
naturalism
! Goes -ve as becomes ‘nearly human’
SEE: http://en.wikipedia.org/wiki/Uncanny_Valley
The problem of expectations
! Humans have hard-wired expectations – Used to interpret inter-personal behaviour – Fundamental social skill
! Need to invoke this very carefully – Acceptability of movement
• Lip sync a key problem
– Interactive responsiveness • Memory of interaction a key issue • Games create a real problem with instant rewind
Building a body
! Use of 3D modelling package – 3ds Character Studio; Poser etc
! Creation of skeleton – Required for animation later
! Polygonal body – But overall count must be low for real-time
interaction, low 000s or less if possible
! Texture to cover body – One body, many textures?
Texture Images
X 4
Silhouette Images
X 4
Stage 1: Image Capture
• Generic Avatar
• Polygonal Structure
• 1500 Polygons
Change Data: Deforms the Mesh
Stage 2: Deform the Mesh
Apply Textures to Deformed
Mesh
Stage 3: Apply the Textures
The touch up process
What sort of motion?
! Walking around – Jumping, running, swimming – Human and non-human
! Picking things up – Agent-agent interaction
! Gesture ! Talking heads
– Facial muscles, lip synch
! Group Movement – Crowds, flocks, herds
Moving the body
! Animation – Drawing on cartoon animation
! Motion capture – Drawing on gait analysis and then film
! PBM – Drawing on robotics
• Analytically calculated • Learned
Animation
! Time consuming for good results – Requires artistic skills
! The main character, Woody, in Toy Story had: – 700 degrees of freedom (200 for face and
fifty for the mouth) – 150 people (at Pixar) would generate 3
minutes of animation a week ! Can it be standardised?
Self-animation
! Poses new problems: – Parametrisable animation very desirable
• E.g a walk that could be reused with different stride length or foot height
– Melding and combining of animations ‘on the fly’ • Not just a morphing problem • ‘starting position’?
– Extra actions inserted by character
Standardising animation?
! Animation depends on the structure of skeleton that is being animated
! Can we standardise a humanoid skeleton? – H-anim just such an attempt – Originally in the context of VRML – See www.h-anim.org
H-anim
! A set of standard components – Humanoid: root of a figure – Joint: attached using transform specifying
current state of articulation plus geometry associated with attached body part
– Segment: specifies attributes of physical links between joints
– Site: where can add semantics – Displacer:range of movement allowed for object
in which embedded
H-anim
And in X3d..
<HAnimJoint DEF='hanim_l_hip' center='0.0961 0.9124 -0.0001' name='l_hip'>
<HAnimSegment DEF='hanim_l_thigh' name='l_thigh'/> <HAnimJoint DEF='hanim_l_knee' center='0.1040 0.4867 0.0308' name='l_knee'> <HAnimSegment DEF='hanim_l_calf' name='l_calf'/> <HAnimJoint DEF='hanim_l_ankle' center='0.1101 0.0656 - 0.0736' name='l_ankle'> <HAnimSegment DEF='hanim_l_hindfoot’ name='l_hindfoot'/> </HAnimJoint> </HAnimJoint>
</HAnimJoint>
Motion capture
! Electro-magnetic or by camera ! Produces the most ‘natural’ results
– Extensively used in film for graphical extras – Can be used on avatars very successfully to transmit user
movement to their graphical representation
! Even more problems for self-animation – Harder to parametrise – Worse combining/melding problems
Mocap process
! Calibration ! Capture ! 3D Position Reconstruction ! Fitting to the Skeleton ! Post Processing
Calibration
! Triangulate camera position and set origin
! Wand – Calibrate and triangulate
cameras and capture volume
! L-frame – Define ground plane, up
vector and origin
Multiple Hypothesis Tracking
! For small number of markers: Size ! Occlusions are a problem ! Multiple Hypothesis Tracking
Ringer, et al., 2002
C1
C2
Fitting to the Skeleton
! Utopian approach – 10 – 20% length
changes
! Markers on both sides
! Joint Displacement ! Use Rotation Angles
Only
Post Processing
! Motion Editing – Cut, Copy, Paste
! Motion Warping – Speed up or Slow Down – Rotate, Scale or Translate
! Motion Signal Processing – Smoother Motions
Physically-based modelling
! Extending robot motor control ! Forward kinematics
– If you move joints so much – Where does the end effector go?
! Inverse kinematics – If you want the end effector at x,y,z… – How much should which joints move?
! Computationally demanding and often not very naturalistic – But the most flexible option – Animation blending often added…
Ragdoll models
! Multiple rigid body simulation – Each tied to bone in skeleton – Constraints for joint movement
! Extended into procedural animation of whole body – Natural Motion: Euphoria
Fish example - spring-mass model
Learning to move
! Use of AI learning algorithms – Move – Evaluate and score – Keep movements which work well
! Karl Sims ‘Blockies’ – Early 1990s
Terzoplolous - Fish Learning
http://www.csri.utoronto.ca/~dt/
Virtual sensors
! Local v global interaction
– Global - read from world data-structures– Local - ‘sense’ the environment
! Global approach very common
– In most computer games– Efficient, easy to test
Advantages of local sensing
! Scales well– Not affected by global size of environment
! Agent has independence of environment
– Up to a point! Makes for believability
– Agent perceives what it should– Can’t see you round corners– Emergent complexity
Sensing by message passing
! Requires an architecture that distributes events as messages – Concept of locale
• What is local for local sensing?
– Semantically determined: e.g a room
! Scenegraphs not set up for this style of message passing – Think of routing in VRML for example
Credits
! Edward Tse, University of Calgary