advanced decision architectures collaborative technology alliance five lessons learned in...
TRANSCRIPT
Advanced Decision Architectures Collaborative Technology Alliance
Five Lessons Learned in Human-Robot Interaction
Patricia McDermottAlion Science and Technology
Jennifer Riley, Ph.D.SA Technologies
Douglas Gillan, Ph.D.North Carolina State University
Laurel Allender, Ph.D.,Army Research Laboratory
Advanced Decision Architectures Collaborative Technology Alliance
RESEARCH OBJECTIVES Investigate the impact of key technological, human, and task
factors on human robot interaction in a range of robotic tasks. • Bandwidth
reduction • Span of control • Unmanned vehicle characteristics • Level of automation • Interface designs • Collaborative technologies • Teamwork configurations
Sense
Process
UnderstandPresent
Communicate
Helping the soldier…
Advanced Decision Architectures Collaborative Technology Alliance
Research Overview
Integrated HRIResearch Findings
Candidate Robot Interfaces
Collaborative Research Experiments
HRI Design Guides/ Implications
Identified HRIResearch Gaps
Research Road Map
SA Technologies
AlionNMSUNCSU
ARL
Advanced Decision Architectures Collaborative Technology Alliance
Top 5 Signs You have Made an HRI Error
5. The operator keeps covering the same ground over and over.
Advanced Decision Architectures Collaborative Technology Alliance
Perception, Navigation, and Operating Robots
• Field of View studies– Widening the view
is most beneficial– If can’t widen
view, 3rd person view can aid navigation
Screenshots of conditions in FOV/Camera Perspective study
Wide FOV/3rd Person Perspective
Wide FOV/1st Person Perspective
Narrow FOV/3rd Person Perspective
Narrow FOV/1st Person Perspective
Advanced Decision Architectures Collaborative Technology Alliance
• Limited Bandwidth• Spatial vs. Temporal
Resolution– Participants selected
higher spatial resolution, were faster to identify friend/foe
– If high spatial resolution not available, temporal resolution may be just as useful
Perception, Navigation, and Operating Robots
High Spatial Resolution
High Temporal Resolution
Advanced Decision Architectures Collaborative Technology Alliance
Perception, Navigation, and Operating Robots
• Lesson 5: Removing the human from the environment to be navigated disrupts the perception of visual information about the environment and the motion. The HRI interface can be designed to reduce this disruption and enhance the operator’s ability to navigate a robot through its distant environment and to perform the desired tasks.
– Widen the field of view (FOV) if possible. Providing a third-person perspective also benefits navigation, the benefits are additive.
– Temporal resolution may be sacrificed if the resolution is high and the robot is traveling at relatively low speeds. A high spatial resolution aids in target detection and identification.
Advanced Decision Architectures Collaborative Technology Alliance
Top 5 Signs You have Made an HRI Error
4. The operator keeps turning the wrong direction.
Advanced Decision Architectures Collaborative Technology Alliance
Team Factors and Processes
• Collaboration Experiments– Location of teammates– Collaborative technologies– Communication tools
View from UAV
View from UGV
Map with player locations
Advanced Decision Architectures Collaborative Technology Alliance
Team Factors and Processes
Lesson 4: When robotic operators are distributed from Soldiers who need the information from robotic assets to complete a mission, the interfaces and collaborative technologies must support the communication of high quality spatial information in relation to the Soldier in the field.
Advanced Decision Architectures Collaborative Technology Alliance
Team Factors and Processes
• The following solutions can be implemented to aid collaboration among humans with robotic tasks:– When robotics team members cannot be co-located, it is
advantageous to incorporate aspects of co-location such as the ability to share imagery.
– When sharing imagery, be sure to include enough reference information so that the recipient understands where the picture was taken in reference to his or her location.
– Allow both team members to have full-time access to a map of the area without having to search between displays.
– Allowing team members to communicate visually on a shared map can enhance the communication of highly spatial information.
Advanced Decision Architectures Collaborative Technology Alliance
Top 5 Signs You have Made an HRI Error
3. The operator keeps misidentifying targets.
Advanced Decision Architectures Collaborative Technology Alliance
Multitasking
• High workload task - integrating and interpreting visual information from the different sources – Unique shape and color
cues had a huge impact on performance
Advanced Decision Architectures Collaborative Technology Alliance
Multitasking and the Role of Automation
Lesson 3: Operating a robot inherently involves multitasking and task switching. When designing automation to aid multitasking, designers must consider how social and human factors in HRI can interact to negatively counter the benefits of automated functions.
Advanced Decision Architectures Collaborative Technology Alliance
Multitasking and the Role of Automation
• When implementing automation in robotic tasks, designers should keep in mind the following:– At natural break points in tasks or during down time, the
system could prompt participants to switch tasks or robots. This would facilitate appropriate task switching strategies that produce better performance without sacrificing SA
– Provide critical cues to help operators reconcile imagery from different perspectives such as aerial and ground or from two different ground perspectives. Shape and color were the most salient cues followed by texture, markings, and shadows.
– Evaluate the coordination costs of two operators sharing one automated asset. The costs may outweigh the benefits, especially in an ad-hoc team.
Advanced Decision Architectures Collaborative Technology Alliance
Top 5 Signs You have Made an HRI Error
2. The operator does not notice a decrement in the robot status
Advanced Decision Architectures Collaborative Technology Alliance
SA Design Needs
Lesson 2: Factors associated with human control of robotics assets can differentially impact SA. It is critical to determine the information needs of Soldiers within a context and to support the multiple facets of SA through system and interface design.
In order to provide information at the right level, start with a goal-directed (cognitive) task analysis
and behavioral and communications analysis.
Advanced Decision Architectures Collaborative Technology Alliance
SA-Related Design Needs
• Support for visually demanding tasks. Examples include object recognition support, strategies or interface support for depth perception and judging size and distance, and support for parallel processing and divided attention.
• Support for understanding robot localization and situated-ness. Examples include improved GPS for robots, mapping tools at the interface, interface designs that present data on orientation and robot configuration, and mission recording and playback for quick SA update.
• Improved interface design for developing higher level SA. For example, conduct SA-oriented design of interfaces to facilitate integration of data at the interface and design for understanding of robot automation states and modes).
Advanced Decision Architectures Collaborative Technology Alliance
Top 5 Signs You have Made an HRI Error
1. Your paper only includes performance data.
Advanced Decision Architectures Collaborative Technology Alliance
HRI Measures
• Performance Data• SA Data• Latency experiment example
– Better control performance with no latency– Decreased SA errors with latency
• Impacts interpretation in the context of complex tasks and goals
Advanced Decision Architectures Collaborative Technology Alliance
HRI Measures
Lesson 1: Performance measures and SA measures should be examined together to get the “whole” story on operational effects in HRI.
Implication: There are multiple methods to assess SA:Direct assessmentSubjective assessmentProcess assessment
Result: Better understanding of interrelationships between decision making performance and SA in HRI
Advanced Decision Architectures Collaborative Technology Alliance
Implications for Future Army
• Ways in which humans and robots can work seamlessly together in naturalistic environments
• Implications for display design, interaction design, team configuration, collaborative technologies, and HRI metrics
• Future research – Development of system design requirements– Examination of interrelationships in HRI, i.e., spatial
awareness and SA, spatial awareness and mental map development
– More studies should be done on the interrelationships among human performance measures such as performance time and accuracy, SA, teamwork, and collaboration
Advanced Decision Architectures Collaborative Technology Alliance
SPECIAL ISSUE OF JOURNAL OF COGNITIVE ENGINEERING AND DECISION
MAKING
Improving Human-Robot Interaction in Complex Operational Environments: Translating Theory into
Practice
Topic Areas• Sensor Interpretation & Integration• Manipulation• Navigation• Planning• Multiple Robot Operations• Team Performance• Trust and Acceptance• Technological Issues• Research Issues
Key Dates15 October: notification of plan to submit15 November: paper submission
Special Issue Guest EditorsJennifer M. RileySA [email protected]
Patricia L. McDermottAlion Science & [email protected]
Douglas J. GillanNorth Carolina State [email protected]