1 mpml and scream: scripting the bodies and minds of life-like characters soft computing laboratory...
TRANSCRIPT
![Page 1: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/1.jpg)
1
MPML andMPML and SCREAM: Scripting the Bodies SCREAM: Scripting the Bodies and Minds of Life-Like Charactersand Minds of Life-Like Characters
Soft computing LaboratoryYonsei University
October 27, 2004
![Page 2: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/2.jpg)
2
AgendaAgenda
sub-title
• Introduction
• Related work– Scripting the Bodies of Life-Like Characters– Scripting the Minds of Life-Like Characters
• Multi-modal Presentation Markup Language (MPML)
• Scripting Emotion-based Agent Minds (SCREAM)
• Conclusion
![Page 3: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/3.jpg)
3
IntroductionIntroduction• Communication of humans
– Language– Body gesture– Facial expressions
• Character-based interfaces– An effective tools to enhance the interaction between humans
and computers
• Two categories in designing life-like characters– To instruct the “body” of an animated character– To design the mental make-up of a character
![Page 4: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/4.jpg)
4
Scripting the Bodies of Life-Like CharactersScripting the Bodies of Life-Like Characters• Scripting
– Low level Manipulating a character’s facial and body motion models
– High level The synchronization of a character’s synthetic speech with appropriate
gestures The expression of emotions The synchronization of body motion sequences of an ensemble of
characters
• Existing scripting languages– CML (Character Markup Language)– VHML (Virtual Human Markup Language)– BEAT (Behavior Expression Animation Toolkit)– APML (Affective Presentation Markup Language)– Hard to use for non-experts
• MPML (Multi-modal Presentation Markup Language)– Designed for non-experts
![Page 5: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/5.jpg)
5
Scripting the Minds of Life-Like CharactersScripting the Minds of Life-Like Characters• A task complementary to scripting a character’s visual
appearance– To author a character’s mental state– The way the character perceives its surrounding world– How the character reacts emotionally
• SCREAM (Scripting Emotion-based Agent Minds)– Similar to Reilly’s EM architecture– But more flexible in the sense– Emotion-based synthetic characters can be run in a web browser
like the Extermpo and IMP systems
![Page 6: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/6.jpg)
6
Multi-Modal Presentation Markup LanguageMulti-Modal Presentation Markup Language• What is MPML?
– A scripting language to instruct the body-related behavior of animated agents
• The Features of MPML– Ease of use– Intelligibility– Believability– Extensibility– Easy Distribution
![Page 7: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/7.jpg)
7
The MPML FamilyThe MPML Family
• MPML 2.2a– MPML 2.2a supports
sequential and parallel behavior of multiple charactersan interface to Macromedia Flash
• DWML– DWML is concerned with scripting dynamic relations between
web-based media objects
• MPML-VR– MPML-VR enables presentations in 3D space
![Page 8: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/8.jpg)
8
MPML System ArchitectureMPML System Architecture• Fig 3
![Page 9: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/9.jpg)
9
MPML System Architecture (cont.)MPML System Architecture (cont.)• MPML Script text file
– Containing the tagging structures specifying character and environment control
• MPML 3.0 Visual Editor– Script Loader module
Load the text fileCheck the script for syntactical errors
– Graph moduleVisualize the script by generating a graphical representation of
the presentation– Script Saver module
Convert the graph to a textual MPML script– Converter module
Transforms the MPML script to JavaScript
![Page 10: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/10.jpg)
10
MPML SpecificationMPML Specification• Using DTD (document type definition) for readability• Element
– Consisted the name and child elements– If no child elements exist for a tag, it is called “empty”
• Attribute– Name-value pairs separated by “=”– An attribute value may be required or fixed, denoted by
#REQUIRED and #FIXED• Entity declarations
– (e1, e2, …, eN) : An ordered list of child elements– (e1 | e2 | … | eN) : An unordered list of child elements– e? : zero or one time– e+ : one or more times– e* : zero or more times– e : exactly on time
![Page 11: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/11.jpg)
11
Tagging Structures for Character ControlTagging Structures for Character Control• DTD for character-related elements
![Page 12: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/12.jpg)
12
Tagging Structures for Presentation ControlTagging Structures for Presentation Control• DTD for presentation-related elements
![Page 13: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/13.jpg)
13
MPML Visual EditorMPML Visual Editor
![Page 14: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/14.jpg)
14
Using MPML to Script an Interactive PresentationUsing MPML to Script an Interactive Presentation
![Page 15: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/15.jpg)
15
Scripting Emotion-based Agent MindsScripting Emotion-based Agent Minds• What is SCREAM?
– A tool to script the interactive behavior of affective agents
• The key properties of SCREAM– Ease of use– Intelligibility– Believability– Extensibility– Granularity
![Page 16: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/16.jpg)
16
SCREAM System ArchitectureSCREAM System Architecture
![Page 17: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/17.jpg)
17
Communicative ActsCommunicative Acts• Emotion-based agents receive input in the form of
communicative acts
• Written as com_act (S, H, Concept, Sit)– S: speaker– H: the addressee (interlocutor)– Concept: the information conveyed by S to H– Sit: situation
SCREAM System ArchitectureSCREAM System Architecture
![Page 18: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/18.jpg)
18
Emotion GenerationEmotion Generation• Appraisal Module
– Emotion types22 classes: joy, distress, happy for, sorry for, resent, angry at,
and so on. Intensity: δi ∊{0, …, 5}
– PersonalityTwo dimensions: agreeableness and extraversion ∊{-5, …, 5}
• Emotion Resolution Module– Select the active emotions in Sit– from a set {<E1, δ1, Sit>, <E2, δ2, Sit>, …, <En, δn, Sit>}
• Emotion Maintenance– Decrease the intensity by the agent’s personality
SCREAM System ArchitectureSCREAM System Architecture
![Page 19: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/19.jpg)
19
Emotion RegulationEmotion Regulation• Social Variables
– Social powersocial-power(L2,L1,θP,Sit)
agent L2 is θP ranks higher than L1 in Sit, as perceived by L1
– social distancesocial-distance((L1,L2,θD,Sit)
The closeness between two agents(L1,L2)
Lower values indicate more closeness, as perceived by L1
SCREAM System ArchitectureSCREAM System Architecture
![Page 20: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/20.jpg)
20
Emotion Regulation (cont.)Emotion Regulation (cont.)• Control Variables
– PersonalityAn agent is more likely to express negative emotions if it is either
disagreeable or extrovert
– Other agent’s personality If the agent assumes that the interlocutor’s personality is
disagreeable, it will rater not express a negative emotion
– Reciprocal feedbackThere are “negative” feedback loops where agents adjust to
linguistic unfriendly behavior
• Emotion Filter Module– ε = δ – (θ + )
SCREAM System ArchitectureSCREAM System Architecture
![Page 21: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/21.jpg)
21
Emotion ExpressionEmotion Expression• Expressive responses
– Somatic responses (flushing)– Behavioral responses (fist-clenching, throwing objects)– Two types of communicative responses, verbal and non-verbal
(for instance, frowning)
• Information-processing responses– The agent’s diversion of attention and evaluations
SCREAM System ArchitectureSCREAM System Architecture
![Page 22: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/22.jpg)
22
Agent ModelAgent Model• Character profile
• Affect processing
–
– ∊ { + , - }, L : agent, I : interlocutor
– If (δ+ - δ-) > 0, agent L like agent I
– Otherwise agent L dislike agent I
SCREAM System ArchitectureSCREAM System Architecture
n
SitSit
n
i
ILnIL
n
1
,,
)()(
![Page 23: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/23.jpg)
23
Using SCREAM to Script a Conversational CharacterUsing SCREAM to Script a Conversational Character
![Page 24: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/24.jpg)
24
Using SCREAM to Script a Conversational CharacterUsing SCREAM to Script a Conversational Character
• Dialogue for five consecutive rounds where the user never follows Djinn’s advice. The figure does not show the user’s interaction with the dealer
![Page 25: 1 MPML and SCREAM: Scripting the Bodies and Minds of Life-Like Characters Soft computing Laboratory Yonsei University October 27, 2004](https://reader035.vdocument.in/reader035/viewer/2022062720/56649f115503460f94c23d8f/html5/thumbnails/25.jpg)
25
ConclusionConclusion• MPML
– A powerful and easy-to-use character scripting tool for web content experts who are non-programmers
• SCREAM– A practical technology for scripting the mental processes
underlying a character’s affective behavior
• Future research– The design of an architecture that allows for a tighter integration
of character and environment scripting– The development of a plot manager that produces narratively
meaningful interactive stories