Introduction to Human-Robot

Interaction (HRI)By: Anqi Xu

COMP-417

Friday November 8th, 2013

What is Human-Robot Interaction?• Field of study dedicated to understanding, designing, and evaluating

robotic systems for use by, or with, humans [Goodrich 2007]

Osaka U.

MITMedia Lab

Rethink Robotics

NASA

Brief History (of HRI as a Field)

• 1992: 1st IEEE International Symposium on Robots and Human Interactive Communications (RO-MAN)

• Late 1990’s: workshops and conference tracks on HRI at International Robotics Conferences (e.g. AAAI Symposia Series, ICRA, IROS, RSS, Human Factors, etc.)

• 2000: 1st IEEE/RAS International Conference on Humanoid Robots (HUMANOIDS)

• 2004: IEEE/RAS & IFRR summer school on Human-Robot Interaction

• 2006: 1st ACM International Conference on Human-Robot Interaction (HRI)

• 2012: inaugural volume of Journal of Human-Robot Interaction (jHRI)

Related Fields

Telerobotics & Haptics Human Factors Air Traffic Control

Human-ComputerInteraction

Operations Research Machine Learning

Unique Features of HRI

• Physical interaction with embodied intelligence

• Rich social relations between humans and robots

• Complex, dynamic & unpredictable environment

NASA Robonaut

USAR Robot@ NIST / Texas U&M

© Fox

Taxonomies in HRI• Physical vs. cognitive

• Co-located vs. remote

• Team configurations [Yanco 2002]

H

R

H

R R

H

R R R

H H

R

H H H H

R R

H H

R R

H H

R R

Constituents of HRI

• Objective: understand and shape interactions between human(s) and robot(s)• Multidisciplinary: social sciences, natural sciences, engineering, …

• 5 Main Constituents of HRI• Level and behavior of autonomy

• Nature of information exchange

• Structure of human-robot team

• Training of people and robots

• Design and shaping of task for human-robot collaborations

Autonomy

• Mapping of inputs from the environment into actuator movements, representational schemas, etc. [Goodrich 2007]

• Neglect Time [Goodrich 2003]

• Turing test

• Means to support productive HRI

Laundry-Folding Robot (UC Berkley)

Autonomy (cont.)• Levels of Autonomy (LOA) [Sheridan 1978]

1. Computer offers no assistance; human does it all

2. Computer offers a complete set of action alternatives

3. Computer narrows the selection down to a few choices

4. Computer suggests a single action

5. Computer executes that action if human approves

6. Computer allows the human limited time to veto before automatic execution

7. Computer executes automatically then necessarily informs the human

8. Computer informs human after automatic execution only if human asks

9. Computer informs human after execution only if it decides to

10. Computer decides everything and acts autonomously, ignoring the human

direct control dynamic autonomy

Information Exchange

• Intelligent interaction requires deliberate communication

• Interaction Time [Goodrich 2003]• Switch attention to current task

• Establish context

• Plan actions

• Communicate plan to robot

• Workload

• Situational Awareness

• Shared Mental Model

Information Exchange (cont.)

Medium Robot-Initiated Human-Initiated

VisualVisual displays

(lights, GUI, VR, AR)Gestures (hand,

facial, body)

AudioNatural language, non-speech audio

Natural language

Tactile HapticsKeyboard, mouse, gamepad, haptics

• Methods of communication:

Paro Theraputic RobotTag-based Comm.

with Aqua Robot (McGIll U.)HRP-4C (NIST)

Teams

• Motivations & Purposes• Task division

• Task efficiency

• Redundancy

• Multi-robot management• Cognitive overload

• Relates to level of autonomy, nature of task, and mode of communication

• Fan-Out [Goodrich 2003]

Kestrel Autopilot v3.0 (Lookheed Martin)

Teams (cont.)

• Team hierarchy

• Conflict resolution• Especially for peer-based relationships (co-X)

• Active roles• Supervisor

• Operator

• Mechanic / Assistant

• Peer

• Slave

• Passive roles• Patients

• Visitors

• BystandersLovotics Robot (N. Taipei U.)

RIBA: Robot for InteractiveBody Assistance (RTC)

Training

• Training robots (learning)• Learning from Demonstration (LfD): transfer of task-domain knowledge

from human teacher to robot student

• Training operators• Learning curve

• Intuitive design: facilitates interaction with challenged users

• Necessity of training: streamline interaction, reduce risk

• Training designers• Human-centric interface design

• Humans and Automation: Use, Misuse, Disuse, Abuse [Parasuraman 1997]

Task Shaping

• Fundamental changes to nature of tasks• Optimize human/robot workload distribution

• Encourage synergy between capabilities of humans and robots

• Additional support to human-robot team• Design tools to facilitate interaction for both human and robot

• Employ technology / other robots for greater situational awareness

Heterogeneous Multi-Robot Coral Reef Inspection (McGill U.)

HRI Constituents: Summary

• Framing the main constituents and objectives of HRI research• Level and behavior of autonomy

• Nature of information exchange

• Structure of human-robot team

• Training of people and robots

• Design and shaping of task for human-robot collaborations

• Dynamic interaction: time-varying, task-varying, context-specific changes to HRI constituents

Sample HRI Problems / Solutions

• Establishing design principles for efficient human-robot interaction[Goodrich 2003]

• Characterizing human’s situational awareness [Endsley 1988]

• Teaching robot helicopter acrobatic moves [Ng 2004]

Seven Principles of Efficient Human Robot Interaction [Goodrich 2003]1. Implicitly switch interface and autonomy modes

2. Let robot use natural human cues

3. Manipulate the world instead of the robot

4. Manipulate relationship between the robotand the world

5. Let people manipulate presented information

6. Externalize memory

7. Help people manage attention

© Raizlabs

Virtual Cockpit v2(Procerus / Lookheed Martin)

Situational Awareness Global Assessment Technique (SAGAT) [Endsley 1988]

• Evaluation procedure:• Assign task scenario to user and robot (in simulation)

• At some random point in time, halt simulation and blank relevant displays

• Administer random subset of questions about SA requirements

• SA levels: immediate, intermediate, long-range

• Compare real and perceived situation post-hoc, and report true/false %

• Repeat for different users to obtain measures of statistical significance

Autonomous Inverted Helicopter Flight via Reinforcement Learning [Ng 2004]

Further Resources

• Conferences: HRI, RO-MAN, ICRA, IROS, RSS

• Journals: jHF, jHRI

• Website: www.humanrobotinteraction.org

• My Contact: anqixu@cim.mcgill.ca / www.cim.mcgill.ca/~anqixu

References

• M. Goodrich and A. Schultz. Human-Robot Interaction: A Survey. Foundations and Trends in Human-Computer Interaction, 1(3), 2007, pp. 203-275.

• M. Goodrich and D. Olsen. Seven Principles of Efficient Interaction. in Proceedings of IEEE International Conference on Systems, Man, and Cybernetics, 2003, pp. 3943-3948.

• T. Sheridan and W. Verplank, Human and Computer Control for Undersea Teleoperators. MIT Man-Machine Systems Laboratory, 1978.

• H. Yanco and J. Drury. A Taxonomy of Human-Robot Interaction. in Proceedings of the AAAI Fall Symposium on Human-Robot Interaction, 2002, pp. 111-119.

• R. Parasuraman and V. Riley. Humans and Automation: Use, Misuse, Disuse, Abuse. Human Factors: The Journal of Human Factors and Ergonomics, 39(2), 1997, pp. 230-253.

• K. Lakshmanan, A. Sachdev, Z. Xie, D. Berenson, K. Goldberg, P. Abbeel. A Constraint-Aware Motion Planning Algorithm for Robotic Folding of Clothes. in proceedings of the 13th International Symposium on Experimental Robotics (ISER), 2012.

• F. Shkurti, A. Xu, M. Meghjani, J. Gamboa, Y. Girdhar, P. Giguere, B. Dey, J. Li, A. Kalmbach, C. Prahacs, K. Turgeon, I. Rekleitis, and G. Dudek. Multi-Domain Monitoring of Marine Environments using a Heterogeneous Robot Team, in Proceedings of the 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '12), Portugal, October 2012.

• M. Endsley. Situation Awareness Global Assessment Technique (SAGAT). In Proceedings of the National Aerospace and Electronics Conference (NAECON), 1988, pp. 789–795.

• A. Ng, A. Coates, M. Diel, V. Ganapathi, J. Schulte, B. Tse, E. Berger, and E. Liang. Inverted Autonomous Helicopter Flight via Reinforcement Learning. in International Symposium on Experimental Robotics, 2004.