making pedagogical agents more socially intelligent

USC / Information Sciences Institute

Making Pedagogical AgentsMaking Pedagogical AgentsMore Socially IntelligentMore Socially Intelligent

Lewis JohnsonLewis JohnsonDirector, CARTEDirector, CARTE

USC / ISIUSC / ISIftp://ftp.isi.edu/isd/johnson/si/ftp://ftp.isi.edu/isd/johnson/si/


Background: Pedagogical Background: Pedagogical Agents (aka Guidebots)Agents (aka Guidebots)


Adele DemoAdele Demo


ClaimsClaims Such guidebots requireSuch guidebots require

Understanding of humans’ activitiesUnderstanding of humans’ activities Social interaction skills, i.e., Social interaction skills, i.e., social social

intelligenceintelligence Most tutoring systems Most tutoring systems

understand learner activities, understand learner activities, but lack social intelligencebut lack social intelligence

Challenge: to create guidebots Challenge: to create guidebots with SIwith SI

Without social intelligence:Without social intelligence:


Characteristics of Social Characteristics of Social AgentsAgents Cognizance of other agentsCognizance of other agents

Aware of their beliefs, attitudes, characteristicsAware of their beliefs, attitudes, characteristics Sensitivity to social relationship, rolesSensitivity to social relationship, roles Sensitivity to social context, exchangeSensitivity to social context, exchange Able to manage interactions, taking Able to manage interactions, taking

above into accountabove into account


Social Intelligence ProjectSocial Intelligence Project Develop models of social intelligence Develop models of social intelligence

for educational softwarefor educational software Track learner cognitive and affective states, Track learner cognitive and affective states,

personality and learning characteristicspersonality and learning characteristics Manage interaction to maximize communication Manage interaction to maximize communication

effectiveness, persuasivenesseffectiveness, persuasiveness Adapt interaction to the learnerAdapt interaction to the learner Track learner-agent interaction as a social Track learner-agent interaction as a social

relationshiprelationship


Architecture of SI SystemArchitecture of SI System


Experimental Basis Experimental Basis Videotaped sessions of computer-Videotaped sessions of computer-

based learning with human tutorsbased learning with human tutors Students read written tutorial on line, Students read written tutorial on line,

completed simulation-based exercisescompleted simulation-based exercises Tutors sat next to students, observed, Tutors sat next to students, observed,

engaged in dialog as appropriateengaged in dialog as appropriate Multiple sessions with each studentMultiple sessions with each student Intended to provide a model of Intended to provide a model of

appropriate guidebot interactionappropriate guidebot interaction


Conclusions from VideotapesConclusions from Videotapes Dialog consisted of a series of exchangesDialog consisted of a series of exchanges On student side:On student side:

Differing degrees of understanding, as well as confidenceDiffering degrees of understanding, as well as confidence Differing preferences for social interactionDiffering preferences for social interaction Differing preferred divisions of rolesDiffering preferred divisions of roles

On tutor side:On tutor side: Monitoring learner activityMonitoring learner activity Sensitivity to understanding and confidenceSensitivity to understanding and confidence

On both sides:On both sides: Use of interaction tacticsUse of interaction tactics


Interaction TacticsInteraction Tactics Intended to achieve a particular Intended to achieve a particular

primary goal (communicative, primary goal (communicative, persuasive)persuasive)

Often address additional subsidiary Often address additional subsidiary goalsgoals

Listener response monitored to assess Listener response monitored to assess primary goal achievementprimary goal achievement

Tactics revised in response to Tactics revised in response to achievement failureachievement failure


ExampleExample Tutor: So it’s asking for regression Student: Right, that wasn’t an option…

there’s no place… Tutor: You want to click on regression here…


Tutor Monitoring of Goal Tutor Monitoring of Goal AchievementAchievement Look for student’s verbal Look for student’s verbal

acknowledgement (or otherwise)acknowledgement (or otherwise) Look for student actions indicating Look for student actions indicating

understandingunderstanding Rely on expectations of actions both Rely on expectations of actions both

before and after before and after


Subsidiary Communicative Subsidiary Communicative GoalsGoals Tutor phrased comments in order to Tutor phrased comments in order to

reinforce learner control and joint reinforce learner control and joint activity. E.g.:activity. E.g.: ““Why don’t you go ahead and read your tutorial Why don’t you go ahead and read your tutorial

factory”factory” ““You want to save the factory”You want to save the factory” ““I’d skip this paragraph”I’d skip this paragraph” ““So why don’t we do that?”So why don’t we do that?”


Some Implications for Some Implications for GuidebotsGuidebots Need to reduce disruptiveness of Need to reduce disruptiveness of

human-guidebot communicationhuman-guidebot communication Communication should be goal and Communication should be goal and

tactic orientedtactic oriented Communication should be situated in Communication should be situated in

work contextwork context A tactic-oriented approach could also A tactic-oriented approach could also

help prevent and repair communication help prevent and repair communication breakdownsbreakdowns


A Tactic-Oriented Learner-A Tactic-Oriented Learner-Guidebot InterfaceGuidebot Interface Both tutorial view and simulation interface Both tutorial view and simulation interface

are instrumentedare instrumented Learner communicates with guidebotLearner communicates with guidebot

Directly using selected questions, typed commentsDirectly using selected questions, typed comments Encoded as dialog moves using DISCOUNT schemeEncoded as dialog moves using DISCOUNT scheme Utilizes eDrama Learning’s NL parsing techniqueUtilizes eDrama Learning’s NL parsing technique

Indirectly via actions, focus of attentionIndirectly via actions, focus of attention To be added soon:To be added soon:

Vision tracking -> focus of attention monitoringVision tracking -> focus of attention monitoring Dialogs to assess learner confidence, update learner Dialogs to assess learner confidence, update learner

characteristics, assess progress in assessing social characteristics, assess progress in assessing social rolesroles


Next Step: Wizard-of-Oz Next Step: Wizard-of-Oz ExperimentExperiment Student interacts with agent enhanced Student interacts with agent enhanced

interfaceinterface Controlled by remote tutorControlled by remote tutor

Questions:Questions: Does tactic model permit appropriate tutorial Does tactic model permit appropriate tutorial

interaction? interaction? Will subjects interact with the agent the way they Will subjects interact with the agent the way they

interact face to face with tutors?interact face to face with tutors?


AcknowledgmentsAcknowledgments Faculty:Faculty:

Maged Dessouky, Chistoph v. d. Malsburg, Jeff Rickel Maged Dessouky, Chistoph v. d. Malsburg, Jeff Rickel (USC)(USC)

Richard Mayer (UCSB)Richard Mayer (UCSB) Helen Pain (U. of Edinburgh)Helen Pain (U. of Edinburgh)

Research staff:Research staff: Erin Shaw, Kate LaBore, Larry Kite, Kazunori Okada Erin Shaw, Kate LaBore, Larry Kite, Kazunori Okada

(USC)(USC) Students:Students:

Lei Qu, Ning Wang (USC)Lei Qu, Ning Wang (USC) Wauter Bosma, Sander Kole (U. of Twente)Wauter Bosma, Sander Kole (U. of Twente) Jason Finley (UCLA)Jason Finley (UCLA) Heather Collins (UCSB)Heather Collins (UCSB)

making pedagogical agents more socially intelligent

Documents