sam spaulding - emotion ai developer day 2016

Socially Assistive Robots, Educational Tutoring, and Affective Computing

Sam Spaulding MIT Media Lab

Social Robots Inventing our Future while Learning about Ourselves

Robotic Engineering

Artificial Intelligence

Studies of Human Behavior

Human-Robot Interaction

Educational Companion

Aging-in-Place HelperAssembly-mate

Socially Assistive Robots (SAR)

Socially Assistive Robots are designed to leverage their social and affective attributes to provide social support to people in order to sustain engagement, motivate, coach, monitor, educate, or facilitate communication & teamwork for improved outcomes.

Embodiment matters!

Robots produce higher learning

gains

Robots are more able to form long-term

bonds

Robots produce greater

compliance

Leyzberg et al. (2012)

Bainbridge et al. (2012)

Kidd & Breazeal (2008)

Compared to screen-based representations...

Intelligent Tutoring Systems

Use domain-general inference and modeling algorithms

Have been extensively tested in real-world environments over long periods of time

Intelligent Tutoring SystemsRobotic Tutors

The best of both...

Social PresenceOnboard Sensing

Agent-based Interaction

Adaptive PersonalizationData-driven Student Models

Physically embodied robot tutors that: sense and understand emotions

build models of students based on affective data act intelligently as a result of the model info

Affect-aware Student Models for Robot Tutors

- Child and Robot interact through shared “Storymaker” game context

- Robot framed as peer, periodically asks child to demonstrate reading ability

(to be presented at AAMAS ’16)

Knowledge State Estimation

- A key challenge for adaptive, computational tutors is - “how to personalize experience?”

- In order to provide personalized curriculum, the tutor must first determine students’ initial knowledge state

Task Difficulty

Student Ability

Boredom

Overwhelmed

Flow: Optimal Challenge

“Flow”

Bayesian Knowledge Tracing (BKT)

Bayesian Knowledge Tracing (BKT)

P(Lit)

Correctt

P(Lit+1) P(Li

t+2)... ...

Correct t+1 Correct t+2

Background Model Domain Evaluation Contributions

Sparse channel for knowledge, but widely studied

Each ‘traced’ skill modeled by an HMM

Affective Bayesian Knowledge Tracing

Affective data drawn from 5s before question asked to 5s after question answered

Affect-aware Student Models for Robot Tutors

- 25 children came and played with Dragonbot

- 13 children did same interaction with Tablet only

- Experiment conducted in Summer 2014, affective analysis completed 6mo. later

(to be presented at AAMAS ’16)

Are Children More Emotionally Expressive When Interacting with A Robot?

... ... ... ...} {Avg Smile: 18Avg BrowFurrow: 7Avg BrowRaise: 62Avg LipDepress: 4Avg Valence: -24Avg Engage: 67

} {Median Filter } {Mean

Metric Value

Session FootageRaw Affdex MeasurementsMedian-smoothed Affdex Data

Subject Bag

Average Metric Value over Interaction

Are Children More Emotionally Expressive When Interacting with A Robot?

*

**

* p < .05

n=25 “Robot” conditionn=13 “Tablet condition

Training and Evaluating Skill Models

BKT and Aff-BKT models trained for 4 Skills via Expectation Maximization

Model classes evaluated via log-likelihood comparison, with Leave-one-out cross-validation

Does Including Affective Data in Training Yield Better Models?

... }Affective + Right/Wrong Training Data

Subject Bag

Expectation Maximization {

P(Lit)

CorrecttSmilet

P(Lit+1)

...

Engagedt Correctt+1Smilet+1 Engagedt+1

...

P(Lit)

CorrecttSmilet

P(Lit+1)

...

Engagedt Correctt+1Smilet+1 Engagedt+1

...}Training Data Trained Model Trained Model

Subset

{Model Subset

P(Lit)

CorrecttSmilet

P(Lit+1)

...

Engagedt Correctt+1Smilet+1 Engagedt+1

...

P(Lit)

Correctt

P(Lit+1)

...

Correctt+1

...

Held-out Right/Wrong Test Data, D'

Likelihood of Model, Given Test Data D’

}D’ P(D’| θaff)

θaff = maxθ P(Daff|θ)θaff, a subset of θaff, containing only

BKT model parameters

θaff

^

^

^

Daff

Does Including Affective Data in Training Yield Better Models?

[ [ [ [

Exact-Correct

BKT Aff-BKT BKT Aff-BKT BKT Aff-BKT BKT Aff-BKT

First-Letter Length Last-Letter

Does Including Affective Data in Training Yield Better Models?

p < 1.0 x 10-4

*

*

***

***

**p < 1.0 x 10-5

p < 1.0 x 10-6

*

***

Are children more emotionally expressive when interacting with robots?

Can we leverage emotional expression data to create better student models?

Two main questions:

Are children more emotionally expressive when interacting with robots?

Can we leverage emotional expression data to create better student models?

Two main questions:

Intelligent Tutoring SystemsRobotic Tutors

The best of both...

Social PresenceOnboard Sensing

Agent-based Interaction

Adaptive PersonalizationData-driven Student Models

Physically embodied robot tutors that: sense and understand emotions

build models of students based on affective data act intelligently as a result of the model info

Meet Tega!

Tega: a “real-world ready” social robot!

Student models allow us to personalize curricular content. How do we personalize affective support?

Affective Personalization

Affective Personalization of a Social Robot Tutor for SSL

(presented at AAAI ’16)

•What’s it all about?•Children learning Spanish as a second language with a robot companion

•An Integrated System•Tega Robot •Custom Educational Game• Affdex affective sensor •All synchronized and coordinated through a ROS-based cognitive architecture

•The Study•Long-term, in-the-wild, fully autonomous interaction•Personalization of affective response

Integrated System: Software

Game:- Unity-based sprite game- 8 sessions of content + review- Fully autonomous play- Virtual “instructor” character, robot as peer

Affdex phone:- Real-time detection of facial

expressions- Valence / Engagement used as

reward to RL algorithm

Educational Context

Robot again framed as peer, with ‘bilingual’ Toucan as teacher

As the student plays through the game, the robot provides affective support through verbal + nonverbal actions

Reinforcement Learning on Affective Data

SARSA Algorithm

Reward = .4(Engagement) +.6( (Valence+100) )2

State Space = 3 x 2 x 2 x 2 = 24 states total

Neg./Neut./Pos. Valence

Hi/LoEngagement

On/OffTask

Right/Wrong Last Question

Action Space = 3 x 2 + No-action = 7 action classes total

ɛ-greedy algorithm, with ɛ decreasing across sessions

Sample Actions

Timeline• 8 Weeks of In-Class deployment• 1 Pre-test Session• 6 Study Sessions (part review, part new content)• 1 Post-test Session

Affective Personalization

***

***

*

***

Affective Personalization

*

Appropriate affective responses are critical to avoiding “novelty effect”

Did they actually learn?

*

Contributions

• Novelty of current study:• Long-term interaction (8 sessions)• Fully autonomous social robot (Tega)• In-the-wild experiment (inside a classroom)• Affective personalization (Affdex)• Age of participants (3-6 yrs)

Social robot personalized its affective response, thus increasing children’s valence during long-term

interaction.

Looking ahead…

Old challenges are becoming tractable: sensing, deployment, robust systems.

New challenges are conceptual and computational - i.e. how to fully integrate emotions into an agent’s cognition

Collaborators and Supporters

This research was supported by the National Science Foundation(NSF) under Grant CCF-1138986 and Graduate ResearchFellowship Grant No 1122374.

Luke Plummer JinJoo Lee

Goren Gordon Jacqueline KoryProf. Cynthia Breazeal