sliding autonomy for peer-to-peer human-robot teams m.b. dias, b. kannan, b. browning, e. jones, b....

Sliding Autonomy for Peer-To-Peer Human-Robot Teams

M.B. Dias, B. Kannan, B. Browning, E. Jones, B. Argall, M.F. Dias, M.B. Zinck, M. Veloso, and A. Stentz

IAS 2008

Research Sponsored by The Boeing Company

2ResultsSliding Autonomy Summary

ApproachMotivation

Motivation

Human-Robot pickup teams Un-known team composition Highly heterogeneous teams Dynamic operating environment Complex nature of tasks Need to adapt to changing

situations


ApproachMotivation

Sliding Autonomy

Existing Work Initial set of 3 characteristics proposed by Sellner et al. [1] Alternate set of 3 characteristics proposed by Bruemmer

and Walton [2]

Our Work Define Sliding Autonomy for the Peer-to-Peer domain Describe a comprehensive set of characteristics for

incorporating sliding autonomy in Peer-to-Peer teams Define a general approach for implementation Implement it on an example application Collect results for comparison

Allowing team to adjust its level of autonomy as necessary


ApproachMotivation

Sliding Autonomy in Peer-to-Peer teams

Sub-team 1 Sub-team 2

Human and multi-robot team members

Sub-team 3

Robot team members

Human and robot team members

De-centralized situational awarenessVarying prioritization among team member

High priority

Low priority

High priority

Low priority

Varying levels of decision making

Higher decision making

Lower decision making

Higher decision making

Lower decision making

Team members, humans or robots, are actively involved in deciding when to temporarily relinquish control to another member or to an entity outside the sub-team

Team

Dynamic Sub teams


ApproachMotivation

Characteristics of Sliding Autonomy in Peer-to-Peer teams

Granularity of interaction

Maintaining coordination during interventions

Gaining and maintaining situation awareness

Prioritization of team members

Request help

Learning from interaction


ApproachMotivation

System ComponentsDistributed market-based planner

Robots

Human-Interface tools

Tightly-coordinated multi-agent plan

Plan selectionRole assignments

High-level task status

Low-level statusErrors

Low-level commands

Low-level statusErrors, Maps,

Location

Low-level commands


High-level tasks

Cost data capabilities

- Overview- Sliding Autonomy

Granularity of interactionCoordinationSituation AwarenessPrioritization of team members - currently fixedRequesting Help

ResultsSliding Autonomy Summary

ApproachMotivation 3

Experimental Domain - Treasure Hunt

Human-Robot teams coordinate to explore an unknown environment and locate items of interest

Let’s form a sub-team

search sector I

Robot X retrieve Treasure A

Treasure A

Robot X


ApproachMotivation

Example WalkthroughDistributed market-based planner

Robots

Human-Interface tools

Tightly-coordinated multi-agent plan

Plan selectionRole assignments


Low-level statusErrors

Low-level commands

Low-level statusErrors, Maps,

Location

Low-level commands


High-level tasks

Cost data capabilities


ApproachMotivation

Implementation without Sliding Autonomy - Error Detection and Recovery

Laser error:• Autonomous detection• Autonomous identification• Autonomous/Human-assisted recovery

No-Arcs error:• Autonomous detection• Autonomous identification• Human-assisted recovery

Pose error:• Autonomous detection• Autonomous identification• No current easy recovery

Where am I??

10


ApproachMotivation

Implementation - Error Handling


ApproachMotivation

Implementation with Sliding Autonomy - Error Detection and Recovery

Laser error:• Autonomous detection• Autonomous identification• Autonomous/Human-assisted recovery

No-Arcs error:• Autonomous detection• Autonomous identification• Human-assisted recovery

Pose error:• Autonomous detection• Autonomous identification• No current easy recovery

Where am I??

We expect the performance of system with sliding autonomy to better than the one without sliding autonomy

12


ApproachMotivation

Experimental Setup

Indoor testing environment - Robotics Institute, CMU 3 robot team-members and 2 human-team

members Pioneers - SICK LiDar and Fiber optic gyros

Planning and localization Segways - Camera

Following, relative localization and locating treasures

ER1 - Camera Tele-operation

3 different treasure configurations Each run is for a fixed time period of 15 minutes Total of 7 scattered “treasure”

13


ApproachMotivation

Experimental Setup - Test Environment

dfdfdfdfd

Highbay in the Robotics Institute, Carnegie Mellon University

14


ApproachMotivation

Results - Expt 1: Team composition = 2 Humans, 1 Pioneer, 1 Segway

Run Treasure seen

(recovered)

Error Types Error Source

Error per robot

T_1 2(2) Total: 1[L, 6.5 minutes]

G(1) R1(1)

T_1 1(1) Total: 2[P (2 min), L(5 min)]

G(2) R1(1), R2(1)

T_1 0(0) Total: 1[P(7.5 min)] G(1) R1(1)

Sliding Autonomy disabled

Run Treasure seen

(recovered)


Error per robot

T_1 4(2) Total: 5[L(1), A(2), P(2)]

N(5) R1(2), R2(3)

T_1 4(2) Total: 5[L(1), A(2), P(2)]

N(5) R1(2), R2(3)

T_1 4(2) Total: 5[L(1), A(2), P(2)]

N(5) R1(2), R2(3)

Sliding Autonomy enabled

# System is able to recover from multiple error instances

15


ApproachMotivation

Results - Expt 2: Team composition = 3 Humans, 1 Pioneer, 1 tele-operated ER1

Run Treasure seen

(recovered)


Error per robot

T_1 3(2) N Total: 1[L1] G(1) R1(1)T_1 0(0) N Total: 1[A(1)] N(1) R1(1)T_1 4(2) N Total: 1[L(1)] G(1) R1(1)

Sliding Autonomy disabled Skill level E - Expert, N - Novice

Run Treasure seen

(recovered)


Error per robot

T_1 4(4) N Total: 2[L(1), P(1)] G(2) R1(1), R2(1)T_1 6(3) E Total: 5[L(1), A(3),

P(1)]G(2),N(3) R1(3), R2(2)

T_1 4(2) E Total: 3[L(1), A(1), P(1)]

G(2),N(1) R1(2), R2(1)

Sliding Autonomy enabled

# Sliding Autonomy can improve team performance# Flexibility in accommodating different team configurations

16


ApproachMotivation

Conclusion and Future Work

Conclusion Extend Sliding Autonomy to Peer-to-Peer human-robot

teams Outline an approach for implementing SA Implement on an example human-robot team

application Ability to dynamically adjust the level of autonomy

can enhance system performance

Future Work Enhancing situational awareness via human

interaction and state information Dynamic prioritization among team members Incorporate learning into the system

17


ApproachMotivation

Questions

References [1] B. Sellner, F. W. Heger, L. M. Hiatt, R. Simmons, and

S. Singh, “Coordinated Multiagent Teams and Sliding Autonomy for Large-Scale Assembly,” Proceedings of the IEEE, Vol. 94, No. 7, 2006

[2] D. J. Bruemmer and M. Walton, “Collaborative tools

for mixed teams of humans and robots,” Proceedings

of the Workshop on Multi-Robot Systems, 2003

18


ApproachMotivation

Outline

Motivation

Sliding Autonomy for peer-to-peer teams

Approach

Results

Summary and future work

19


ApproachMotivation

Motivation

Humans and robots working together to accomplish complex team tasks

Pickup teams - un-known team composition with members of varying capabilities, expertise, and knowledge

Robots - In-sufficient capabilities to handle complex situation

Human roles Predominantly - supervisory or end-user Alternate approach - peer-to-peer relationship

Effective use of the complimentary capabilities of humans and robots

Allow humans the flexibility to handle situations that the robots cantKey feature - Allowing team to adjust its level of autonomy as necessary

20


ApproachMotivation

Fluid teamsTeam: time = t1

Team: time = t2

Team: time = t3

21


ApproachMotivation

Trading system

Robots are organized as an economy

Autonomous task-allocation based on cost and capability

Team mission is to maximize production and minimize costs

Instantaneous allocation Tiered-auctioning approach

Individual agents generate plans and auction them

Humans are yet not part of the auctions

$

$

$

$

$

22


ApproachMotivation

Play Manager

Play selection from playbook

Dynamic role assignment

Coordinates execution of action by sub-teams

Low-level commands to team members

Handles status messages to and from team-members

Play 1Play 2

Play 3Role 1

Search Retrieve

Selection

ExecutionMonitoring,Adaptation

Robot 1TacticRobot 1

TacticRobot NTactic

23


ApproachMotivation

System Components (cntd.)

Operator Tools High-level and low-level commands State information feedback Process status messages

24


ApproachMotivation

System components (cntd.)

Play manager coordinates execution of action by sub-teams low-level commands to team members handles status messages to and from team-members Error Handling via help requests

GUI Text-based Help Intervention

physical interaction direct low-granularity commands

Robot Software Abstract information - capabilities, actions and sensors Fluid participation

25


ApproachMotivation

Our Approach granularity

level 1 - High-level task objective level 2 - Low-level robot commands

Coordination via simple communication protocol Help requested for error-handling modes Fixed prioritization technique -

low-level commands over-rule human instructions

Situational awareness is handled via a customizable GUI reflecting state information

sliding autonomy for peer-to-peer human-robot teams m.b. dias, b. kannan, b. browning, e. jones, b....

Documents

level of autonomy

peer teams subteam

robot team membershuman

peer humanrobot teamsm

peer teamsdefine

peer domaindescribe

necessarysliding autonomy

arcs error