single operator control of mul2ple uas: a supervisory
TRANSCRIPT
![Page 1: Single Operator Control of Mul2ple UAS: A Supervisory](https://reader031.vdocument.in/reader031/viewer/2022012418/61729c70b97a6a3c1c5c6fc5/html5/thumbnails/1.jpg)
SingleOperatorControlofMul2pleUAS:A
SupervisoryDelega2onApproach
PresentedtoUASEXCOMScienceandResearchPanel(SARP)WorkshoponSingleOperatorControlofMul2pleUAS
JayShivelyNASA-AmesResearchCenter
UAS INTEGRATION IN THE NAS
JayShivelyDAASub-ProjectManager
1
![Page 2: Single Operator Control of Mul2ple UAS: A Supervisory](https://reader031.vdocument.in/reader031/viewer/2022012418/61729c70b97a6a3c1c5c6fc5/html5/thumbnails/2.jpg)
Levelsofautoma2onofDecisionandAc2onSelec2on(Sheridan&
Verplanck,1978)
1Thecomputeroffersnoassistance,humanmusttakealldecisionsandac2ons2Thecomputeroffersacompletesetofdecision/ac2onalterna2ves,or3Narrowstheselec2ondowntoafew,or4Suggestsonealterna2ve,and5Executesthatsugges2onifthehumanapproves,or6Allowsthehumanarestrictedveto2mebeforeautoma2cexecu2on7Executesautoma2cally,thennecessarilyinformsthehuman,and8Informsthehumanonlyifasked,or9Informsthehumanonlyifit,thecomputer,decidesto10Thecomputerdecideseverything,actsautonomously,ignorestheHuman
![Page 3: Single Operator Control of Mul2ple UAS: A Supervisory](https://reader031.vdocument.in/reader031/viewer/2022012418/61729c70b97a6a3c1c5c6fc5/html5/thumbnails/3.jpg)
SupervisoryControlSheridan(2002)definedsupervisorycontrolasanarrangementinwhich“oneormorehumanoperatorsareintermiaentlyprogrammingandcon2nuallyreceivinginforma2onfromacomputerthatitselfclosesanautonomouscontrolloop,”buthealsoaccentuatedthehumansystemrela2onshipunderlyingthedefini2on:“Supervisorycontrolderivesfromthecloseanalogybetweenasupervisor’sinterac2onwithsubordinatepeopleinahumanorganiza2onandaperson’sinterac2onwithintelligentautomatedsubsystems”Supervisorycontrolisageneraltermforcontrolofmanyindividualcontrollersorcontrolloops,suchaswithindistributedcontrolsystem.Itreferstoahighlevelofoverallmonitoringofindividualprocesscontrollers,whichisnotnecessaryfortheopera2onofeachcontroller,butgivestheoperatoranoverallplantprocessview,andallowsintegra2onofopera2onbetweencontrollers.
![Page 4: Single Operator Control of Mul2ple UAS: A Supervisory](https://reader031.vdocument.in/reader031/viewer/2022012418/61729c70b97a6a3c1c5c6fc5/html5/thumbnails/4.jpg)
Delega2onControl:Playbook®• Delega2on:onewayhumansmanagesupervisory
controlwithheterogeneous,intelligentassets
• Playbook®:onesmeansofdelega2on
• Plays:analogoustofootball– Quickcommands–complex
ac2ons
• APlayprovidesaframework– Referencesanacceptablerange
ofplan/behavioralterna2ves– Requiressharedknowledgeof
domainGoals,TasksandAc2ons– Supervisorcanfurtherconstrain/
s2pulate• Poten2allyfacilitatesintui2vecoopera2vecontrolof
UnmannedSystems
• Drill-downandmodifyasrequiredbycontext
ApagefromAlonzoStagg’s1927Playbook
![Page 5: Single Operator Control of Mul2ple UAS: A Supervisory](https://reader031.vdocument.in/reader031/viewer/2022012418/61729c70b97a6a3c1c5c6fc5/html5/thumbnails/5.jpg)
TANGO
Predator Provides Overwatch and Hellfire strike capability
Shadow designates target
Firescout Does Quick Med Drop
Example:TroopsinContactTango
![Page 6: Single Operator Control of Mul2ple UAS: A Supervisory](https://reader031.vdocument.in/reader031/viewer/2022012418/61729c70b97a6a3c1c5c6fc5/html5/thumbnails/6.jpg)
Example:ProsecuteTarget
Tools:Armlaser➔Lasetarget➔SendcoordinatestoweaponizedUAV➔ToggleUAVs➔Armmissile➔Fire
Scripts:Select‘Lase’script➔ToggleUAVs➔Armweapons➔Fire
Plays:Select‘ProsecuteTarget’play➔Fire
0%
10%
20%
30%
40%
50%
60%
70%
80%
Plays Scripts Tools
% C
orre
ct
Control Mode
Secondary Task Performance (% Correct Hits)
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
5.0
Plays Scripts Tools
Ove
rall
Rat
ing
Control Mode
NASA-TLX Ratings
0
10
20
30
40
50
60
Plays Scripts Tools
Rea
ctio
n Ti
me
Control Mode
Primary Task Performance (RT)
Target Acquisition
Target Prosecution
ShorterReac2onTimeforPlays
PlayshadlowerworkloadHigherAccuracyforPlays
LevelsofAutomaConSimulaCon
![Page 7: Single Operator Control of Mul2ple UAS: A Supervisory](https://reader031.vdocument.in/reader031/viewer/2022012418/61729c70b97a6a3c1c5c6fc5/html5/thumbnails/7.jpg)
Manned-UnmannedTeaming:MUM
Level IV Control: Control of Payload and Vehicle Excluding Take-off and Landing
Extend to simultaneous control of multiple heterogeneous UAS
![Page 8: Single Operator Control of Mul2ple UAS: A Supervisory](https://reader031.vdocument.in/reader031/viewer/2022012418/61729c70b97a6a3c1c5c6fc5/html5/thumbnails/8.jpg)
Manned-UnmannedTeaming:MUM
Goals:
• ApplyPlaybook®methodologyandDelConlessonslearnedtohelicoptercockpit;Testinsimula2on
• IncreasecapabilityandefficiencyofUAScontrolbyhelicopterpilots
• Supervisorycontrolofmul2ple,heterogeneousUAS
• Developinfrastructureandlayfounda2onforlaterefforts
![Page 9: Single Operator Control of Mul2ple UAS: A Supervisory](https://reader031.vdocument.in/reader031/viewer/2022012418/61729c70b97a6a3c1c5c6fc5/html5/thumbnails/9.jpg)
Propor2onofTargetsMarkedbyControlMode(OutofTotalPossible)
0
0.2
0.4
0.6
0.8
1
NoUAV Manual Playbook
Prop
orCo
n
ControlMode
p < .05
0
5
10
15
20
25
Manual Playbook
Time(s)
ControlMode
UASRoutePlanningTimebyControlMode
Results
0
2
4
6
8
10
Temporal Frustra2on Performance Overall
AverageRa
Cng
WorkloadDimension
NoUAV Manual Playbook
NASA–TLXRa2ngs
p < .05 p < .05 p < .05
p < .05
HigherAccuracyPlaybook
LowerRoutePlanningTimeforPlaybook
LowerworkloadforPlaybookonseveraldimensions
![Page 10: Single Operator Control of Mul2ple UAS: A Supervisory](https://reader031.vdocument.in/reader031/viewer/2022012418/61729c70b97a6a3c1c5c6fc5/html5/thumbnails/10.jpg)
FlightDemonstra2on2009Ft.OrdCA,23APR2009
Goal:• Demonstratesini2alproofofconceptof
Delega2onControl(Playbook)inflight–supervisorycontrolofmul2pleair/groundassetsinMOUTScenario
Method:• Live/VirtualDemo–ControllingRMAX,CMU
MAXRoverand2virtualUASwithDelega2onControl
• VoiceRGNControl(USAF)
Features:• Delega2oncontrolhuman-machineinterface
supportscontrolandmonitoring4payloads• Automa2onTransparency• LiveUGV-UAVcoordina2onforslungload
drop• Reducedoperatorworkload/highsitua2on
awareness
Live RMAX Virtual Shadow
Virtual Sky Warrior
Live CMU
MAX Rover
![Page 11: Single Operator Control of Mul2ple UAS: A Supervisory](https://reader031.vdocument.in/reader031/viewer/2022012418/61729c70b97a6a3c1c5c6fc5/html5/thumbnails/11.jpg)
• Troopsincontact
• RouteRecon
• AreaRecon
• Convoysupport
• QuickMeds
TopPlays
![Page 12: Single Operator Control of Mul2ple UAS: A Supervisory](https://reader031.vdocument.in/reader031/viewer/2022012418/61729c70b97a6a3c1c5c6fc5/html5/thumbnails/12.jpg)
FlightDemonstra2on2011Ft.Hunter-Ligge9CA,19May2011Purpose:• Buildonprevioussimula2onsandflighttest
examiningsingleoperatorcontrolofmul2pleheterogeneousground/airunmannedsystemsthroughdelega2oncontrolemployment– Operatorperformancedatacollec2on/workload
assessments– Heterogeneousflightassets:BoeingScanEagle
andYamahaRMAX;twovirtualUAS– Tes2nginopera2onallyrelevantmissionscenarios– Mul2-sensorcross-cueinsupportofboth
targe2ngandconvoysupport• ArmyAFDD/BoeingCRADAKeyObjec2ve:• DevelopandtestDelConTopPriorityPlays;
routerecon,convoysupport,troopsincontact
![Page 13: Single Operator Control of Mul2ple UAS: A Supervisory](https://reader031.vdocument.in/reader031/viewer/2022012418/61729c70b97a6a3c1c5c6fc5/html5/thumbnails/13.jpg)
Demonstratedinnumeroussimula2onsandflighttests(evenNOPEsimula2ons)
• AFRL–Basesecurity,UASgroundsta2on
• RCO–Dispatch,cockpit
• HumanAutoma2onTeaming(HAT)
SupervisoryControlSummary
13
![Page 14: Single Operator Control of Mul2ple UAS: A Supervisory](https://reader031.vdocument.in/reader031/viewer/2022012418/61729c70b97a6a3c1c5c6fc5/html5/thumbnails/14.jpg)
CivilUASPlays
• MonitorBorder– Flydesignatedborder– Alertany“signsoflife”
• UAS1–flywaypointatob• UAS2–flyWPbtoC• UAS3–follow-upwithanyalerts
• Evaluatepowerlines• Transitairspace
![Page 15: Single Operator Control of Mul2ple UAS: A Supervisory](https://reader031.vdocument.in/reader031/viewer/2022012418/61729c70b97a6a3c1c5c6fc5/html5/thumbnails/15.jpg)
CivilPlays
• SearchandRescue– Flydesignatedareasofsearchzone–lawnmowerpaaern,alertshapes,colors,etc.
– Survivaldrop–assoonasWPisdesignated• Meds• Radio• Food/water• Shelter
![Page 16: Single Operator Control of Mul2ple UAS: A Supervisory](https://reader031.vdocument.in/reader031/viewer/2022012418/61729c70b97a6a3c1c5c6fc5/html5/thumbnails/16.jpg)
HATAgent
HAT Agent
Operator
Interface
DisplayAudioVisual
Context - Time Pressure- User Info- more
Queries/Requests - A v. B- Why?- What If?
Automation
Plays- Goals- Risks to
achieving goals- Mitigations
AlertsContextResponses to Queries- Alternatives- Transparency info - Predicted Outcomes - Reasoning - Confidence level
Transparency Info
RequestsPolling for Risks
Etiquette Rules/Contextual Sensitivity
Authority Info
Scratch Pad
16
![Page 17: Single Operator Control of Mul2ple UAS: A Supervisory](https://reader031.vdocument.in/reader031/viewer/2022012418/61729c70b97a6a3c1c5c6fc5/html5/thumbnails/17.jpg)
• Pilotdirectedinterface
– Nointentinferencing– Directedbypilotac2ons– Nosetrolesandresponsibli2es– Playbook
• Bi-direc2onalCommunica2on
– Why?– Howconfident?– Whatif?– Addinforma2on
• Transparency
– Calibratedtrust– Granularity– Timepressure
HATAaributes
17
![Page 18: Single Operator Control of Mul2ple UAS: A Supervisory](https://reader031.vdocument.in/reader031/viewer/2022012418/61729c70b97a6a3c1c5c6fc5/html5/thumbnails/18.jpg)
ProblemswithAutoma2on
• Briale– Automa2onotenoperateswellforarangeofsitua2onsbutrequireshuman
interven2ontohandleboundarycondi2ons(Woods&Cook,2006)
• Opaque– Automa2oninterfacesotendonotfacilitateunderstandingortrackingofthe
system(Lyons,2013)
• MiscalibratedTrust– Disuseandmisuseofautoma2onhaveleadtoreal-worldmishapsand
tragedies(Lee&See,2004;Lyons&Stokes,2012)
• Out–of-the-LoopLossofSitua2onAwareness– Trade-off:automa2onhelpsmanualperformanceandworkloadbut
recoveringfromautoma2onfailureisotenworse(Endsley,2016;Onnasch,Wickens,Li,Manzey,2014)
18
![Page 19: Single Operator Control of Mul2ple UAS: A Supervisory](https://reader031.vdocument.in/reader031/viewer/2022012418/61729c70b97a6a3c1c5c6fc5/html5/thumbnails/19.jpg)
HATSolu2onstoProblemswithAutoma2on
• Briale– NegoCateddecisionsputsalayerofhumanflexibilityintosystembehavior
• Opaque– Requiresthatsystemsbedesignedtobetransparent,presentraConaleand
confidence– Communica2onshouldbeintermstheoperatorcaneasilyunderstand
(sharedlanguage)
• MiscalibratedTrust– Automa2ondisplayofraConalehelpshumanoperatorknowwhentotrustit
• Out–of-the-LoopLossofSitua2onAwareness– Userdirectedinterface;adaptable,notadap2veautoma2on– Greaterinterac2on(e.g.,negoCaCon)withautoma2onreduceslikelihoodof
beingoutoftheloop
19
![Page 20: Single Operator Control of Mul2ple UAS: A Supervisory](https://reader031.vdocument.in/reader031/viewer/2022012418/61729c70b97a6a3c1c5c6fc5/html5/thumbnails/20.jpg)
WorkingAgreements
• Pre-determinedauthoritysharingagreementswithautoma2on– Ifthewatercoolingleveldropsbelowacertainvalue,openvalvestoemergencycooling
![Page 21: Single Operator Control of Mul2ple UAS: A Supervisory](https://reader031.vdocument.in/reader031/viewer/2022012418/61729c70b97a6a3c1c5c6fc5/html5/thumbnails/21.jpg)
• Autonomy– Notmuchintoday’s“approved”UAS– WordsMaaer
• ICAO
• Businesscaseforsingleoperatorsupervisorycontrolofmul2pleUAS– Playbookdelega2onisonesuccessfulmethod
• HAT– Coopera2veagentwithknowledgeofworkdomain– Sharedworldknowledge– Canbeextendedtonetworksupervision
Summary
21