intelligent autonomy for reducing operator workload v2 · 2020-04-04 · intelligent systems •...
TRANSCRIPT
ARLPenn State
Intelligent Autonomy for Reducing Operator Workload
Mark RothgebIntelligent Control Systems Department
Autonomous Control and Intelligent Systems Division
April 10, 2007
ARLPenn State
Applied Research Laboratory background
Autonomous unmanned vehicles (ARL / DoD)
Issues in automation and levels of autonomy
Two examples of reducing operator workload via increasing levelsof system autonomy
Overview
ARLPenn State
Navy UARC’s established in mid-1940’s to continue University centered R&D effective during WWII
We offer a diverse portfolio of systems expertise and technologies applicable to Distributed Systems
UARC Universities maintain a long-term strategic relationship with the Navy
Characteristics:– Can address evolving needs with enabling technologies– Understanding of operational problems and environment– Objectivity and independence– Corporate knowledge and memory– From concept to prototype (integration and test facilities)– Freedom from conflict of interest
Applied Research LaboratoryNavy UARC Background
ARLPenn State
Core Technologies
• Fluid Dynamics• Hydro Acoustics• Computational Mechanics• Composite Materials• Information Fusion and Visualization• Energy and Power Systems• System Simulation• Autonomous Control and Intelligent
Systems
ARLPenn State
ARL Full-Time Equivalent Years
Systems Engineering OrientationBasic Research thru Demonstration to Full-Scale Implementation Project Management of Cross-disciplinary, Multi-performer Teams
Characteristics and Size
ARL Part of Penn State Research
ARLPenn State ARL Locations
APPLIED RESEARCH LABORATORY BUILDINGAPPLIED RESEARCH LABORATORY BUILDING
APPLIED SCIENCE BUILDINGAPPLIED SCIENCE BUILDING
NAVIGATION RESEARCH & DEVELOPMENT CENTERNAVIGATION RESEARCH & DEVELOPMENT CENTER ARL CATO PARKARL CATO PARK
GARFIELD THOMAS WATER TUNNELGARFIELD THOMAS WATER TUNNEL
ELECTRO-OPTICS SCIENCE & TECHNOLOGY CENTERELECTRO-OPTICS SCIENCE & TECHNOLOGY CENTER
Keyport Naval FacilityKeyport, Wa.
Distributed Engineering CenterPenn State Fayette Campus
Washington Office
Washington, DC
ARL HawaiiPearl Harbor, Hi.
Electro-Optics CenterKittanning, Pa. ARL Penn State
State College, Pa.
Navigation Research & Development CenterWarminster, Pa.
ARLPenn State
PSU/ARL Experimental UGV
• Embedded Health Monitoring• Autonomous Navigation & Control• Intelligent Self-Situational Awareness• COTS OCU Development• JAUS Development & Testing
ARLPenn State
UAV
PSU Aero/ARL UAV Base Aircraft
4 channels,5 servosRadio
.40-.46 2 strokeor .91 4-strokeEngine
6 to 6 1/2 poundsWeight
64 3/4 inchesLength
1180 sq. inchesWing Area
80 inchesWingspan
Specifications
TRAINER
ARLPenn State
OBJECTIVEDeveloped a rapid prototype AUV for use in collection of oceanographic environmental data
VEHICLE FEATURESLong-range capabilities (>300 nm @ 4 kts)Fully autonomous vehicle operationsLaunch/recovery from TAGS 60 platformSensors: Sidescan Sonar, Acoustic Doppler Current Profilers (ADCP), Conductivity, Temperature, and Depth (CTD)Simple maintenance & turnaround at sea
OCEANOGRAPHIC DATA GATHERING
Diameter:38 in.
Length:27 ft.-10 in.
Weight:9,900 lbs.
Endurance:300 nautical miles
Payload:dual side-scan sonars; other oceanographic instruments
Navigational Accuracy: better than 150 meters
PSU/ARL Autonomous Undersea Vehicle
ARLPenn State
DoD Autonomous Vehicles
• Predator• Firescout
• Battlespace Preparation AUV
ARLPenn State
NASA Dart Autonomous Operation
• Even basic automation concepts … not so simple
…On April 15, more than 450 miles above Earth, an experimental NASA spacecraft called DART (Demonstration of Autonomous Rendezvous Technology) fired its thrusters and closed in on a deactivated U.S. military communications satellite—and then gently bumped into it. (Popular Science 2005)
• Rendevous and Inspection• Proximity Operations
ARLPenn State
Automation Perspectives
• Underwater Vehicles– Communication issues– Load ‘n Go– Automation Manual
• Ground (UGV), Air (UAV), Surface (SUV) Vehicles– Remote control / Tele-operation (fly-by-wire)– Human in control with bits of automation (waypoints)– Manual Automation
• Spacecraft Vehicles– Ground-Control driven– Backoff and “Safe” the system (valuable assets)– Solve the problem on the ground through analysis
ARLPenn State
Operator Overload Forcing Automation
• Operator overload comes in different ways– Increase in number of tasks for the same number of
people• Can’t add crew, but now have more sensors
– Reduce head-count for same number of tasks• Littoral Combat Ship (LCS)
– Increased complexity of tasks forces automation• Surface vehicle on open ocean, surface vehicle in harbor
– Increase in amount of data to process• Need to react quickly also forces automation
– Systems that automatically respond because of timing requirements
– Advisory systems that call the operator to attention
ARLPenn State
Acceptance of Automation
What is required for acceptance of automation…Its all about gaining trust….
• Don’t do something fundamentally wrong (run into the wall)• Don’t do something non-intuitive (go right around wall versus left)• Do tell the operator when the autonomy doesn’t know what to do
– Ambiguous circumstances– Able to solve the 95% case but not the 5%
• Do give insight into decision-making• Do have automation assist the operator, not vice versa
– Microsoft word helps you?– Mapquest fixes for example (beltway anomoly)– Employee Reimbursement System (cure worse than ailment?)
• Do let the operator dynamically alter the level of autonomy– Full manual Full autonomy
ARLPenn State
Levels of Autonomy
• Various groups have defined levels of autonomy– National Institute of Standards (NIST)– Future Combat Systems (FCS)– Air Force Research Laboratory (AFRL)– Uninhabited Combat Air Vehicle– ASTM Committee on Unmanned Undersea Vehicle
Systems (UUV) – NASA FLOOAT (Function-specific Level of
Autonomy and Automation Tool)– Sheridan’s Levels of Autonomy
ARLPenn State
NIST Definitions
Autonomous - Operations of an unmanned system (UMS) wherein the UMS receives its mission from the human <1> and accomplishes that mission with or without further human-robot interaction (HRI). The level of HRI, along with other factors such as mission complexity, and environmental difficulty, determine the level of autonomy for the UMS [2]. Finer-grained autonomy level designations can also be applied to the tasks, lower in scope than mission.
Autonomy - The condition or quality of being self-governing
[NIST Special Publication 1011 - Autonomy Levels for Unmanned Systems (ALFUS) Framework]
ARLPenn State
Sheridan’s Scale for Degrees of Automation
1. The computer offers no assistance, human must do it all2. The computer offers a complete set of action alternatives, and3. narrows the selection down to a few, or4. suggests one, and5. executes that suggestion if the human approves, or6. allows the human a restricted time to veto before automatic
execution, or7. executes automatically, then necessarily informs the human, or8. informs him after execution only if he asks, or9. informs him after execution if it, the computer, decides to.10. The computer decides everything and acts autonomously,
ignoring the human.
R. Parasuraman, T. B. Sheridan, and C. D. Wickens, "A Model for Types and Levels of Human Interaction withAutomation Transactions on Systems, Man, and Cybernetics -Part A, vol. 30, pp. 286-297, 2000
ARLPenn State
Future Combat Systems Levels of Autonomy
1. Remote control / teleoperation2. Remote control with vehicle state knowledge3. External preplanned mission 4. Knowledge of local and planned path5. Hazard avoidance or negotiation 6. Object detection, recognition, avoidance or
negotiation 7. Fusion of local sensors and data 8. Cooperative operations 9. Collaborative operations 10.Full autonomy
– SOURCE: LTC Warren O’Donell, USA, Office of the Assistant Secretary of the Navy (Acquisition, Logistics, and Technology), “Future Combat Systems Review,”April 25, 2003.
ARLPenn State
Levels of Autonomy as Defined by theUninhabited Combat Air Vehicle Program
• Level 1 (Manual Operation)– The human operator directs and controls all mission functions.– The vehicle still flies autonomously.
• Level 2 (Management by Consent)– The system automatically recommends actions for selected functions.– The system prompts the operator at key points for information or decisions.– Today’s autonomous vehicles operate at this level.
• Level 3 (Management by Exception)– The system automatically executes mission-related functions when response
times are too short for operator intervention.– The operator is alerted to function progress.– The operator may override or alter parameters and cancel or redirect actions
within defined time lines.– Exceptions are brought to the operator’s attention for decisions.
• Level 4 (Fully Autonomous)– The system automatically executes mission-related functions when response
times are too short for operator intervention.– The operator is alerted to function progress.
ARLPenn State
NIST Levels of Autonomy
We make a distinction between the terms of “degrees of autonomy” and “levels of autonomy.”Total autonomy in low-level creatures does not correspond to high levels of autonomy. Examples include the movements of earthworms and bacteria that are 100% autonomous but considered low.
[NIST Special Publication 1011 - Autonomy Levels for Unmanned Systems (ALFUS) Framework]
ARLPenn State
Air Force Research Laboratory (AFRL) Levels of Autonomy (Clough)
[ Met
rics,
Sch
met
rics!
How
The
Hec
k D
o Y
ou D
eter
min
e A
UA
V’s
Aut
onom
y A
nyw
ay?
Bru
ce T
. Clo
ugh,
Air
Forc
e R
esea
rch
Labo
rato
ry ]
ARLPenn State
NASA FLOAAT (Function-specific Level of Autonomy and Automation Tool)
[FLO
AA
T, A
Too
l for
Det
erm
inin
g Le
vels
of A
uton
omy
and
Aut
omat
ion,
A
pplie
d to
Hum
an-R
ated
Spa
ce S
yste
ms,
Rya
n W
. Pro
ud*
and
Jere
my
J.
Har
t†]
ARLPenn State
Automation Approach in the “Real” World
• Don’t over-commit on capability of automation• Begin by automating the mundane
– Bid and proposal database (Excel…)– Periscope key-in’s– Surface ship heading recommendations
• Extend by making some mildly intelligent inferences regarding decision-making– Go the right way around the wall– Not always simple: Cul-de-sac
• Extend to more complex “intelligent” systems…– Neural Nets– Fuzzy Systems– Rule-based Systems– Other techniques– Cognition?
• But… What is intelligence?
ARLPenn State
Intelligent Systems
• AIAA Intelligent Systems Technical Committee (JACIC, Dec., 2004), they stated:
"The question of what is an intelligent system (IS) has been thesubject of much discussion and debate. Regardless of how one defines intelligence, characteristics of intelligent systems commonly agreed on include:
1) Learning - capability to acquire new behaviors based on past experience; 2) Adaptability - capability to adjust responses to changing environmental or internal conditions; 3) Robustness - consistency and effectiveness of responses across a broad set of circumstances; 4) Information Compression - capability to turn data into information and then into actionable knowledge; and 5) Extrapolation - capability to act reasonably when faced with a set of new (not previously experienced) circumstances."
[courtesy: Lyle Long]
ARLPenn State
Some System Architectures
• Many options– NASA: CLARAty– MIT: MOOS (Framework for Modeling)– MIT: CSAIL (Robotic Reactive Planning)– CMU: SOAR (Cognitive Architecture)– CMU: CORAL (Cooperative Robots)
• Has won Robocup several times• Robocup Goal: “By the year 2050, develop a team of fully
autonomous humanoid robots that can win against the human world soccer champion team.”
– USC: STEAM (Agent Teamwork Model)– PSU/ARL: PIC (Behavior-based Framework)– PSU/IST: R-CAST (RPD Model for Agent Teamwork)– …
ARLPenn State INTELLIGENT CONTROL ARCHITECTUREINTELLIGENT CONTROL ARCHITECTURE
DATA INPUTS
Sensor 1
Sensor N
Messages
.
.
.
INTELLIGENT CONTROLLER
Perception Response
• Sensor Data Fusion• Information Integration• Inferencing and Interpretation • Situational Awareness
• Operational Assessment
• Dynamic Planning and Replanning
• Plan Execution
Messages
ConventionalControlSystems
Human Collaborator
Other AutonomousControllers
• Human-in-the-loop Operations (Collaborates / Commands)
ARLPenn State
System for Operator Workload Reduction
• Talked mostly regarding unmanned systems• Applicability versus a wide range of operational
systems– Let the operator have ultimate control (allow him to
control levels of autonomy)– Gain his confidence by …
• Helping him make better decisions• Not misleading to bad decisions
– Allow him to understand what the system is doing– Don’t provide him more of a burden to operate
• An example of a simple system…
ARLPenn State
Target Anesthesia/Analgesia Example
• Advisory System• Human-In-The-Loop• Information Overload• Subtle Combinatorial
Changes• Reduce 5-to-1
?
An example of a more complex system…
ARLPenn State Contact Awareness Example
• Reduces workload when making tactical maneuvering decisions– Reduce manual integration of information
• Reduce time to make maneuvering decision– Improve situational awareness holistic view
• Improve quality of tactical decision– Better situation understanding leads to better decision– Traceability to “truth” data
• Provide help for less experienced operator – Queue operator to predicted loss of tactical control– Incorporate SME expertise in automated recommendations with
ability to interrogate recommendation
ARLPenn State Contact Collision Threat Level
• CPA Concept
• Collision Threats • Orange to Red
• Level 0-1
• Violation Threats• Yellow to Orange
• Level 0-1
• No Threat Level• Green
2 Kyd
.500 yd
5 Kyd2 Kyd
.500 yd
5 Kyd
Speed in the line of sight (range rate)
Speed of Advance (SOA)