Robots that Work in Collaboration with PeopleRobots that Work in Collaboration with People

Guy Hoffman and Cynthia Breazeal

Robotic Life GroupMIT Media Laboratory

Cambridge, MA, U.S.A

Robots learning from us and working with us

Robots learning from us and working with us

To be intuitive, efficient and enjoyable

these robots must understand and exhibit social skills

To be intuitive, efficient and enjoyable

these robots must understand and exhibit social skills

The goal: working shoulder-to-shoulder with a robot

The goal: working shoulder-to-shoulder with a robot

Most HRI work:robots as toolsMost HRI work:robots as tools

Joint Intention TheoryJoint Intention Theory

• Collaborative Plans Joint Intention

Grosz (1996), Searle (1990)

• Shared Cooperative Activity, Teamwork

Bratman (1992), Cohen & Levesque (1991)

Shared Cooperative ActivityShared Cooperative Activity

• Meshing Subplans

• Responsiveness

• Commitment

• Support

• Communication

Goals and CollaborationGoals and Collaboration

Leonardo DemoLeonardo Demo

So what will it take to collaborate with a robot?

So what will it take to collaborate with a robot?

Q & AQ & A

Additional SlidesAdditional Slides

Recent Related Work in HRIRecent Related Work in HRI

• Robonaut (2003)Collaboration with Teleoperated Robots

• Perzanowsky et al. (2001)Natural, Multi-modal Interface for Robot Control

• Fong et al. (2001)“Collaborative Control” - Using Humans as Source of Information

• Kimura et al. (1999)Vision-Based Teamwork for a Robotic Arm, Without Social Cues

• Traum et al. (2003)Virtual Teams Within a Command Structure

Social SkillsSocial Skills

Human Robot Social Function

“Let’s do task 5” Looks at the Buttons Leo acknowledges that he understands the task, and visibly establishes mutual belief on the task’s initial conditions

Points to Himself He can do the first (and only) part of the task and suggests doing so

“OK, you go!” Presses the red button, looking at it

Looking away from the partner while operating establishes turn taking boundaries

Social SkillsSocial Skills

Human Robot Social Function

Looks back at hispartner

Gaze shift is used to signal endof turn

Nods shortly Acknowledgement signals robot’sperceived end of task.

“I’ll go”

“Let’s do task 5” Looks at the buttons,points to himself

As before

Presses the redbutton

Looks at the redbutton

Acknowledges partner’s action,maintaining mutual belief withregard to the task state

Social SkillsSocial Skills

Human Robot Social Function

Nods shortly Communicates robot’s perceivedend of task.

“Let’s do task 5” Looks at the buttons As before

Assessing his capabilities, therobot requests support in thetask execution

Moves the red button out of the robot’s reach

Looks at the redbutton

Acknowledges partner’s action,maintaining common ground.

Looks back at the human partner; extends his arms in “Help me” gesture.

Social SkillsSocial Skills

Human Robot Social Function

“Let’s do task 1” Looks at the buttons,points to himself

As before

Presses the redbutton

Looks at the redbutton; looks back at the human partner; nods shortly.

Glance acknowledges partner’s action; nod creates mutual belief as to the task’s completion.

“OK, you go!” Presses the red button, looking at it

Looking away from the partner while operating establishes turn taking boundaries

Social SkillsSocial Skills

Human Robot Social Function

At the same time, presses the green button

Looks at the greenbutton; looks back at the human partner; nods shortly.

Glance acknowledges partner’s simultaneous action; nod creates mutual belief as to the task’s completion.

“Let’s do task 7”(unclear)

Leans forward, perking one ear

Communicates misunderstanding,cues human to repeat the request.

“Let’s do task 7” Cocks head, shrugs Communicates verbal understanding, but inability to completethe request. Cues human to adjustthe request (in this case, Leo does notknow task 7)