victec and empathic interfaces

of 29 /29
VICTEC and Empathic Interfaces Ruth Aylett Ana Paiva

Upload: melvin-gray

Post on 02-Jan-2016




0 download

Embed Size (px)


VICTEC and Empathic Interfaces. Ruth Aylett Ana Paiva. What does VICTEC stand for?. V : Virtual ICT : Information Communication Technology E : Empathic C : Characters. Inspired by theatre. Use of performance to dramatise the problem Theatre in Education Linked workshops - PowerPoint PPT Presentation


Page 1: VICTEC and Empathic Interfaces

VICTEC and Empathic Interfaces

Ruth Aylett

Ana Paiva

Page 2: VICTEC and Empathic Interfaces

V: Virtual

ICT: Information Communication


E: Empathic

C: Characters

What does VICTEC stand for?

Page 3: VICTEC and Empathic Interfaces

Inspired by theatre

Use of performance to dramatise the problem Theatre in Education

Linked workshops But is not a cheap option

In every group some will be bullying others Interaction is limited in conventional performance

Page 4: VICTEC and Empathic Interfaces

Forum theatre (Boal)

An interactive form of theatre Sections of audience ‘own’ a character The actor meets them ‘in role’ and discusses

actionsCan only act ‘in character’

Episodic performanceActor can halt an episode to ask for more advice

Hard to manage in school environment

Page 5: VICTEC and Empathic Interfaces

Implementing a Virtual Forum Theatre

3D interactive graphics and virtual actors - synthetic characters

Individual interaction based on creating empathy with the characters

Computer game technology - but NOT a game! Provide a safe environment for children to explore

different perspectives on bullying behaviour.

Page 6: VICTEC and Empathic Interfaces

What are Intelligent SyntheticCharacters?

A synthetic character is a digital or physical entity with personality that can interact with people in real time. Computer games characters are not very intelligent Do not remember anything about interaction Hard to believe in them as personalities Thus hard to create empathy with them

Page 7: VICTEC and Empathic Interfaces

How realistic should they be?

We want them to be believable Is believability the same as realism? We believe in Mickey Mouse…

Characters that are NEARLY realistic may be ‘creepy’ We have high expectations And then they clash with them

Page 8: VICTEC and Empathic Interfaces

Empathic Characters

Already much experience in animation

Page 9: VICTEC and Empathic Interfaces

Mediating Empathy

(1) via the situation- the observer concludes the emotional state of the target from the situation the target is dealing with.

(2) via emotional expressions - the observer interprets the behaviour of the target: for example, when a target cries this means that he/she is probably sad.

Page 10: VICTEC and Empathic Interfaces

Empathic Synthetic Agents

Users experience Empathic emotions

Characters show empathy towards users and other


""empathic synthetic agent" as empathic synthetic agent" as "an agent that is able"an agent that is able

to perceive and have an to perceive and have an internal representation of the internal representation of the other agent's emotions and/or other agent's emotions and/or experiencing an appropriate experiencing an appropriate emotion as a consequenceemotion as a consequence

"empathy triggering synthetic agent" empathy triggering synthetic agent" as "an agent that is able to, by itsas "an agent that is able to, by itsbehaviour and features, allow the behaviour and features, allow the

users to build an empathicusers to build an empathicrelation with it"relation with it"

Page 11: VICTEC and Empathic Interfaces

Mediation by situation

Depends on narrative engagement Collection of narratives from children

Using Kar2ouche story-boarding tool Observation of dramatic performance Scripting of dramas by drama students

Performances by Uni Herts students

Narratives should be: believable, dramatic, somewhat unpredictable

Page 12: VICTEC and Empathic Interfaces

Mediation by emotional expression

Expressive behaviour linked to internal state modelling character’s affect

Facial Expression (expressive facial expressions that include emotions and communication signals)

Voice Body expression, posture and gestures

Page 13: VICTEC and Empathic Interfaces

Designing Characters in VICTEC: Mediating empathy through facial expression

Page 14: VICTEC and Empathic Interfaces

Designing Characters in VICTEC: Mediating empathy though body expression

Page 15: VICTEC and Empathic Interfaces

Designing Characters in VICTEC: Mediating empathy through exaggeration

Through body expression and animation

Through posture and facial expression


• Current Text-to-Speech technology is inadequate

•Real Voice must be used to emphasize emotional content

Page 16: VICTEC and Empathic Interfaces

Engaging the user

The character must be believable No rerun - what happens counts Different scenarios and characters

Emerging story as in improvisational drama, not pre-scripted

Acting as an ‘invisible friend’. Tamagotchis seemed to evoke empathy Give advice about dealing with the situation and

see what happens

Page 17: VICTEC and Empathic Interfaces

A Trailer

Not the finished software, but how we think it is going to look

Page 18: VICTEC and Empathic Interfaces

Capturing affective responses

Creating an affective loop FearNot! Attempts to evoke empathy in the user How could one produce an empathic response to

the user? Requires ability to detect affective state of the


Page 19: VICTEC and Empathic Interfaces

Interaction using Sentoy

Intelligent Physical InterfaceAims at making the bridge between the real and virtual worlds, extending and enhancing the child-system interactions

Page 20: VICTEC and Empathic Interfaces

Hardware of SenToy

Magnet (on off)

Pressure sensors(4 positions each)

Accelerometer(x, y, z angles)

Page 21: VICTEC and Empathic Interfaces



• Sensors size / number• Sensors capabilities• Battery vs Autonomy• Has to handled by children (not too heavy but also robust)

Page 22: VICTEC and Empathic Interfaces

Validation of Embodiment

A Teddy, a puppet and a Barbie were tested - They had to express emotions and actions with

every dolls: - The subjects preferred a neutral expression (Barbie and Teddy)

- The doll had to be malleable (Teddy)

The teddy was easier to handle, more flexible and allowed the free expression of emotions / actions

Page 23: VICTEC and Empathic Interfaces

Built-in patterns for emotion recognition

Gestures Emotions Reference

Put the toy's hands in front of it's eyes or moving the toy backwards


According to Lazarus fear is associated with avoidance.

Moving the toy slightly backwards (squeezing it slightly)

DISGUSTAccording to Lazarus action tendencies for disgust include “move away”, nausea and even vomiting.

Swinging the toy (putting it dancing) and/or playing with its arms

HAPPINESSJoy is portrayed with open arms, movements such as clapping or rhythmic movement according to Darwin.

Bend down its neck or bend down all the toy's trunk

SADNESSAccording to Scherer sadness is expressed through slow movement inwards and head down.

To place its arms crosswise or shake the toy vigorously ANGER

According to Lazarus anger is associated with “tendency to attack”

Open its arms backwards inclining its trunk slightly backwards too


According to Laban surprise is associated with attention and with a sudden event and inclination of trunk backwards. “A surprise person often raises his opened hands high above his head, or by bending his arms only to the level of his face”. 

Page 24: VICTEC and Empathic Interfaces

Built-in patterns

Gestures Actions

Swing the legs forward and backward alternately or moving the toy forward with

small jumpsWALK

A bouncing jump in the vertical JUMP

To bend down and move the arm like it was picking something or moving the toy

like it was divingPICK

Page 25: VICTEC and Empathic Interfaces


The Data from the sensors is passed through radio into a receiver connected with the computer.

Then some signal processing occurs that filters the information, detects patterns and send it to a specific application integration module that provides the semantic.

SenToy Hardware

Signal ProcessingProcedure

SenToy’s Integration


Page 26: VICTEC and Empathic Interfaces

Sentoy in Victec

Roles of SenToy in the Interaction: Out-of-scenario interaction

As an embodiment of the user (toy as best friend). Number of toys is big restriction for evaluation

In-scenario As a form of transparently enriching the experience.

As inspirational input to the character (eg. cheering) As a non-intrusive recording device

Page 27: VICTEC and Empathic Interfaces

Virtually friends

Event for 400 children in June 2004 at University of Hertfordshire: Virtually friends: Social role-play with robots and

agents Questionnaire and interaction with the software Activities with the robots used in research

Page 28: VICTEC and Empathic Interfaces

Challenges - 1

There IS no easy answer No strategy always works Tension between realism and a straightforward message Do we have to present a happy ending?

Producing believable characters and varying scenarios Pushing the technology here Especially language for unscripted interaction

Page 29: VICTEC and Empathic Interfaces

Challenges - 2 Cultural differences

There is no word exactly equivalent to ‘bullying’ in German or Portuguese

School uniform and other differences Where bullying takes place

No school dinners in GermanyChildren unaccompanied to school

Now extending work to China...