1 mpml and scream: scripting the bodies and minds of life-like characters soft computing laboratory...
TRANSCRIPT
1
MPML andMPML and SCREAM: Scripting the Bodies SCREAM: Scripting the Bodies and Minds of Life-Like Charactersand Minds of Life-Like Characters
Soft computing LaboratoryYonsei University
October 27, 2004
2
AgendaAgenda
sub-title
• Introduction
• Related work– Scripting the Bodies of Life-Like Characters– Scripting the Minds of Life-Like Characters
• Multi-modal Presentation Markup Language (MPML)
• Scripting Emotion-based Agent Minds (SCREAM)
• Conclusion
3
IntroductionIntroduction• Communication of humans
– Language– Body gesture– Facial expressions
• Character-based interfaces– An effective tools to enhance the interaction between humans
and computers
• Two categories in designing life-like characters– To instruct the “body” of an animated character– To design the mental make-up of a character
4
Scripting the Bodies of Life-Like CharactersScripting the Bodies of Life-Like Characters• Scripting
– Low level Manipulating a character’s facial and body motion models
– High level The synchronization of a character’s synthetic speech with appropriate
gestures The expression of emotions The synchronization of body motion sequences of an ensemble of
characters
• Existing scripting languages– CML (Character Markup Language)– VHML (Virtual Human Markup Language)– BEAT (Behavior Expression Animation Toolkit)– APML (Affective Presentation Markup Language)– Hard to use for non-experts
• MPML (Multi-modal Presentation Markup Language)– Designed for non-experts
5
Scripting the Minds of Life-Like CharactersScripting the Minds of Life-Like Characters• A task complementary to scripting a character’s visual
appearance– To author a character’s mental state– The way the character perceives its surrounding world– How the character reacts emotionally
• SCREAM (Scripting Emotion-based Agent Minds)– Similar to Reilly’s EM architecture– But more flexible in the sense– Emotion-based synthetic characters can be run in a web browser
like the Extermpo and IMP systems
6
Multi-Modal Presentation Markup LanguageMulti-Modal Presentation Markup Language• What is MPML?
– A scripting language to instruct the body-related behavior of animated agents
• The Features of MPML– Ease of use– Intelligibility– Believability– Extensibility– Easy Distribution
7
The MPML FamilyThe MPML Family
• MPML 2.2a– MPML 2.2a supports
sequential and parallel behavior of multiple charactersan interface to Macromedia Flash
• DWML– DWML is concerned with scripting dynamic relations between
web-based media objects
• MPML-VR– MPML-VR enables presentations in 3D space
8
MPML System ArchitectureMPML System Architecture• Fig 3
9
MPML System Architecture (cont.)MPML System Architecture (cont.)• MPML Script text file
– Containing the tagging structures specifying character and environment control
• MPML 3.0 Visual Editor– Script Loader module
Load the text fileCheck the script for syntactical errors
– Graph moduleVisualize the script by generating a graphical representation of
the presentation– Script Saver module
Convert the graph to a textual MPML script– Converter module
Transforms the MPML script to JavaScript
10
MPML SpecificationMPML Specification• Using DTD (document type definition) for readability• Element
– Consisted the name and child elements– If no child elements exist for a tag, it is called “empty”
• Attribute– Name-value pairs separated by “=”– An attribute value may be required or fixed, denoted by
#REQUIRED and #FIXED• Entity declarations
– (e1, e2, …, eN) : An ordered list of child elements– (e1 | e2 | … | eN) : An unordered list of child elements– e? : zero or one time– e+ : one or more times– e* : zero or more times– e : exactly on time
11
Tagging Structures for Character ControlTagging Structures for Character Control• DTD for character-related elements
12
Tagging Structures for Presentation ControlTagging Structures for Presentation Control• DTD for presentation-related elements
13
MPML Visual EditorMPML Visual Editor
14
Using MPML to Script an Interactive PresentationUsing MPML to Script an Interactive Presentation
15
Scripting Emotion-based Agent MindsScripting Emotion-based Agent Minds• What is SCREAM?
– A tool to script the interactive behavior of affective agents
• The key properties of SCREAM– Ease of use– Intelligibility– Believability– Extensibility– Granularity
16
SCREAM System ArchitectureSCREAM System Architecture
17
Communicative ActsCommunicative Acts• Emotion-based agents receive input in the form of
communicative acts
• Written as com_act (S, H, Concept, Sit)– S: speaker– H: the addressee (interlocutor)– Concept: the information conveyed by S to H– Sit: situation
SCREAM System ArchitectureSCREAM System Architecture
18
Emotion GenerationEmotion Generation• Appraisal Module
– Emotion types22 classes: joy, distress, happy for, sorry for, resent, angry at,
and so on. Intensity: δi ∊{0, …, 5}
– PersonalityTwo dimensions: agreeableness and extraversion ∊{-5, …, 5}
• Emotion Resolution Module– Select the active emotions in Sit– from a set {<E1, δ1, Sit>, <E2, δ2, Sit>, …, <En, δn, Sit>}
• Emotion Maintenance– Decrease the intensity by the agent’s personality
SCREAM System ArchitectureSCREAM System Architecture
19
Emotion RegulationEmotion Regulation• Social Variables
– Social powersocial-power(L2,L1,θP,Sit)
agent L2 is θP ranks higher than L1 in Sit, as perceived by L1
– social distancesocial-distance((L1,L2,θD,Sit)
The closeness between two agents(L1,L2)
Lower values indicate more closeness, as perceived by L1
SCREAM System ArchitectureSCREAM System Architecture
20
Emotion Regulation (cont.)Emotion Regulation (cont.)• Control Variables
– PersonalityAn agent is more likely to express negative emotions if it is either
disagreeable or extrovert
– Other agent’s personality If the agent assumes that the interlocutor’s personality is
disagreeable, it will rater not express a negative emotion
– Reciprocal feedbackThere are “negative” feedback loops where agents adjust to
linguistic unfriendly behavior
• Emotion Filter Module– ε = δ – (θ + )
SCREAM System ArchitectureSCREAM System Architecture
21
Emotion ExpressionEmotion Expression• Expressive responses
– Somatic responses (flushing)– Behavioral responses (fist-clenching, throwing objects)– Two types of communicative responses, verbal and non-verbal
(for instance, frowning)
• Information-processing responses– The agent’s diversion of attention and evaluations
SCREAM System ArchitectureSCREAM System Architecture
22
Agent ModelAgent Model• Character profile
• Affect processing
–
– ∊ { + , - }, L : agent, I : interlocutor
– If (δ+ - δ-) > 0, agent L like agent I
– Otherwise agent L dislike agent I
SCREAM System ArchitectureSCREAM System Architecture
n
SitSit
n
i
ILnIL
n
1
,,
)()(
23
Using SCREAM to Script a Conversational CharacterUsing SCREAM to Script a Conversational Character
24
Using SCREAM to Script a Conversational CharacterUsing SCREAM to Script a Conversational Character
• Dialogue for five consecutive rounds where the user never follows Djinn’s advice. The figure does not show the user’s interaction with the dealer
25
ConclusionConclusion• MPML
– A powerful and easy-to-use character scripting tool for web content experts who are non-programmers
• SCREAM– A practical technology for scripting the mental processes
underlying a character’s affective behavior
• Future research– The design of an architecture that allows for a tighter integration
of character and environment scripting– The development of a plot manager that produces narratively
meaningful interactive stories