Transformed Social Interaction – TSI Theory (Bailenson et al. 2008)

• To describe the transformation of interaction in mediated communication environments of humans.

Applying TSI in QuizMaster and virtual worlds

• Mediation - TSI is a mediated communication theory. It is not specific to virtual worlds

• Transformation - the model of communication and effects of the interactants are changed, or transformed, during the interaction.

• Avatars - the mediation is the use of avatars, or avatar-mediated communication

Avatar mediated communication (AMC)

• AMC is the use of avatar for purpose of communication

• The use of avatars:– Rich and multi-modal communication• Multiple channels (visual, audio, gestures, facial

expressions)• Emotions

– Self-representation• You got to pick an image you want to show other

people

Avatar-mediated communication

Fidelity

Co-presence

Real time

A three-dimensional model of communication

Video-conferencing

Avatar-mediated

Face-to-face

Anonymous forums

Impact of TSI on our project

• If we can program the non-playing characters (NPCs) as close as possible to the way the human will use the avatars, we expect that the engagement and satisfaction, and eventually the very objective of the application (in our case, learning effectiveness) will increase.

Implementing TSI

• Three principles of TSI– Self-representation– Sensory abilities– Situational context

Principle 1: Self-Representation

• There is evidence in support of the advantage of self-representation.

• For example, Bailenson, et al. found that the morphing of faces of political candidates with potential voters will increase the affective bonding in low information context (Bailenson et al., 2006)

Principle 1: Self-Representation (Con’t)• the dramatic and

subtle changes to the – Appearances– Behaviors

• of the avatars chosen by the users, such that emotional bonding can be established.

user

Implementation of the TSI dimensions• Self-representation –

morphing of real faces into the avatars

=

User 1: Oscar’s face

Generic pedagogical agent:Stephen

User 2: Steve’s face

40:60In shape only

Transformed face of the pedagogical agent that Oscar will see in the virtual world

Transformed face of the pedagogical agent that Steve will see in the virtual world

eVolver (http/l//www.evolver.com)

Principle 2: Sensory-Abilities:• These transformations empower avatars to

complement human perceptual abilities. Agent-enhanced host avatar • Agent technology’s roles:

– the sensory ability to process the behaviors of contestants:• eye gazing or moving data• emotional status

– Detect students’ personal taint and preferences to provide personal greeting, etc.

– Learn about knowledge to make decision – Provide advice/reminders to contestants. – Express emotions to the contestants

Implementation of Sensory abilities • Applied by recording and

observing the student avatar behaviors, especially his/her gaze direction. – After all contestants enter the

studio, the Host Agent will greet each and every student, addressing his/her name with eye-gaze directed to that student.

User monitor --- tracking of viewing angles of participants Contestant Name: LinData: Lin's Look-Around behaviour during every question sessionDate: Fri Sep 23 22:01:48 MDT 2011Question Number 1 Session -->Lin Did Not Looked Around.Question Number 2 Session -->Lin Did Not Looked Around.Question Number 3 Session -->Lin Did Not Looked Around.Question Number 5 Session -->Lin Did Not Looked Around.

Principle 3: Situational Context

• Transformations – Altering the spatial or temporal

structure of a conversation. • For example, the communication

between agents and students can be optimally configured in terms of the geographical setup of a conference room. • For example, a class of 20

students can sit directly in front of the virtual instructor, and perceive the rest of the students as sitting farther away.

Principle 3: Situational Context (Con’t)

• Altering the flow of rendered time in the communication session, users can implement strategic uses of rewind and fast forward during a real-time interaction in an attempt to increase comprehension and efficiency.

• E.g. the point for providing hints …

Will it work?

• Some observations– Using SMS together (Smart Mob - Howard

Rheingold, 12 years ago)– Personal sharing (Coming of Age in Second Life –

Tom Boellstorff – 4 years ago)– Digital age (Personal Communication in the Digital

Age - Nancy Balm – last year)

Lisa (Moshi Monster)

Penny (Club Penguin)

Jessica (Pets Pets)

April (Bearville)

What did we learn (so far …)

• It is a multi-disciplinary work.• Implementation of a theory is technically

challenging.• It is worth the effort because of the change of

the social environment.