APML, a Markup Language for APML, a Markup Language for Believable Behavior GenerationBelievable Behavior Generation

Soft computing LaboratoryYonsei University

October 25, 2004

2

ContentsContents• Introduction• Expressing believable behaviors• MagiCster architecture• A markup language for behavior specification: APML

– Overview of existing markup languages for expressing human-like behavior

– Defining APML tags

• Facial description language• An example• Conclusion

3

IntroductionIntroduction• Humans communicate using verbal and non-verbal signals

– Body posture, gestures, facial expressions, gaze, intonation and prosody, and words and sentences

• Embodied conversational agents– Virtual body that interacts with another agent– Human-like manner– Believable way

Express emotionExhibit a given personality

• Two approaches on ‘Body’ and ‘Mind’– Strictly and necessarily interdependent– Mainly independent from each other

4

EU project MagiCsterEU project MagiCster• ‘Mind’ and ‘Body’ are interfaced by a language based on XML

• During the conversation– Mind decides what to communicate considering different

factors that trigger the goal of communicating and influence the contents to communicate

– Body reads what the Mind decides to communicate and interprets and renders it at the surface level according to the available communicative channels

• Define a set of language for specifying the format of dialogue moves at different abstraction levels– APML (Affective Presentation Markup Language)– Express the content of the dialogue move at the meaning

level

5

Chapter OverviewChapter Overview• Describe the main features of the underlying architecture

• Present the APML language and how it has been used in the context of the MagiCster project

• Show hot it has been interfaced with a 3D realistic face call Greta and a synthetic voice

• Use an example in the medical domain

6

Expressing Believable BehaviorsExpressing Believable Behaviors• Communication: A means to influence others

• Beliefs forming the content of a communicative act– Information about the world

Deictics, adjectival

– Information about the speaker’s identity– Information about the speaker’s mind

Speaker’s beliefs: Degree of certainty, metacognitive information

Goals: Performativity of the sentence, topic-comment or theme-rheme distinction, rhetorical relations, turn-taking and backchannel

• Emotions: Affective words, gestures, intonation, facial expression, gaze, and posture

7

MagiCster Architecture (1)MagiCster Architecture (1)• An example of advice-giving dialogue in the medical domain

– The agent (Gi): A doctor– The Interlocutor (Uj): A patient

• Coordination of the speech with various expressions– In move G1, she manifests her empathy with the User– In move G2, Greta indicates her chest while saying “a spasm

of chest”– In move G3, she looks at the User while saying “your

problem”

8

MagiCster Architecture (2)MagiCster Architecture (2)• The MagiCster system

– ‘Mind’ component: A content planner, a dialogue manager, and an affective agent modeling module

– ‘Body’ component : A 3D face/avatar with a speech synthesizer

– A plan enricher: An interface between the ‘Mind’ and ‘Body’ components

9

MagiCster Architecture (3)MagiCster Architecture (3)• The affective agent modeling module: Decide

– Whether a particular affective state should be activated– Whether the felt emotion should be displayed in a given

context

• The content planner– Generate the discourse plan appropriate to the context– DPML DTD: Achieve the goal in that piece of conversation

10

MagiCster Architecture (4)MagiCster Architecture (4)• The dialogue manager

– Top of the TRINDI architecture– Compute dialogue moves and a space in which information

relevant to the move selection

• The plan enricher– Translate the symbolic representation of a dialogue move

into an Agent’s behavior specification at the meaning level– Translate the DPML-based tree structure into APML

• The face and body animation– Interpret the APML-tagged dialogue move– Decide how to convey every meaning

11

A Markup Language for Behavior Specification: A Markup Language for Behavior Specification: APMLAPML

• High-level primitives for specifying behavior acts

• Express agent behavior at different levels of abstraction

• Control easily the behavior of ECAs independently of the body

• APML (the Affective Presentation Markup Language)– Specification of the agent behavior at the meaning level– Affective aspect of the communication between the agent

and the user

12

Overview of Existing Markup Languages for Overview of Existing Markup Languages for Expressing Human-Like BehaviorExpressing Human-Like Behavior

APML

• Human markup language (HML)– Provide a very abstract level language

Difficult for controlling specific agent bodies Require developing complex interpreters

– Enhance the fidelity of human communications– Allow the representation of physical, cultural, social, kinetic,

psychological, and intentional features

• VHML: Provide several languages for acting on different modalities

• MPML (Multimodal Presentation Markup Language): Enable authors of web pages to add agents for improving human-computer interaction (MS-Agent)

• BEAT (Behavior Expression Animation Toolkit): Generate embodied agent’s animation from textual input

• Avatar markup language (AML): Represent a new high-level language to describe avatar animation

13

Defining APML TagsDefining APML Tags

APML

• APML DTD: Communication function (A meaning-signal pair)

14

Facial Description Language (1)Facial Description Language (1)• Describe facial expressions as (meaning, signal) pairs

• Define expressions to capture a slight variation– At a high level: A facial expression is a combination of other

facial expressions already defined– At a low level: A facial expression is a combination of facial

parameters

• Combine facial expressions due to distinct co-occurring communicative acts using a Bayesian network– Facial basis (FB): a basic facial movement– Facial display (FD): a set of FBs

surprise = raised_eyebrow + raised_lid + open_mouth

15

Facial Description Language (2)Facial Description Language (2)

16

Facial Description Language (3)Facial Description Language (3)

17

An Example: Medical domain application (1)An Example: Medical domain application (1)

DPML

APML

18

An Example: Medical domain application (2)An Example: Medical domain application (2)

19

ConclusionConclusion• Describe the architecture of the behavior generator of a

believable conversational agent

• Focus on the importance of Mind-Body separation

• Define two XML-like markup languages to represent the Mints’ output and the Body’s input

• APML: Not available in public