run-time ethics checking for autonomous unmanned vehicles developing a practical approach 2014...
TRANSCRIPT
Run-Time Ethics Checking for Autonomous Unmanned
Vehicles Developing a Practical Approach
2014 CRUSER TechCon
9 April 2014
Duane Davis, Don Brutzman, George Lucas Jr., Bob McGhee
Cyber Academic Group (CAG); Modeling, Virtual Environments, Simulation (MOVES), Graduate School of Business and Public Policy (GSBPP)
Naval Postgraduate School (NPS), Monterey California USA
Military Ethics & Autonomous Vehicles
Schools of thought on ethics and autonomy:1. Any use of autonomy is inherently unethical2. Non-use of autonomy when capability exceeds
human operators is inherently unethical3. Ethical use is system dependent and situational4. Ethical autonomy requires human oversight
(and ultimately human accountability)
Our position:• Increasing autonomy is inevitable—#1 is unrealistic• Artificial Intelligence (AI) is insufficient (so far?) for
#2• The “wild, wild, west” of #3 amounts to no ethics at
all• Human processes are the key to ethical
operation
Broad ObservationsMilitary ethics is not about religion or morality• Law of Armed Conflict• International law (both treaty and convention)• Rules of Engagement (ROE)• Guidance and Doctrine (CDR Intent, orders, TTPs,
etc.)
Human operators employ systems accordingly• Professionally and legally accountable
Autonomous systems are just that—systems!• Employment in accordance with same ethical
standard• Robots, like other systems, require trust within their
intended operational regime• Human operators accountable for their appropriate
use
Additional Observations
Robot mission sequencing must be deterministic• Must be fully vetted by operators• Must not be susceptible to “programming errors”
Ethical tests must be determinable• By a human supervisor or critic,• By a virtual environment running a simulation, or• By on-board robot sensors in operating environment
Constraint tests can match common guidelines such as Rules of the Road, water-space management, Rules of Engagement (ROE), OPORDs, etc.• Cannot be vague! must result in True or False• Combining multiple logical constraints is OK
Developing a Practical Approach
Ethical autonomy cannot rely on complex AI algorithms or obscure abstractions for appropriate behavior• No embedded homunculus or abstract ethics
engine
Robot missions must be understandable to human operators• Tractable (and deterministic) mission flow• Informed by well articulated ethical constraints
Robot mission design should be adaptable to a variety of disparate robot paradigms• Generally adaptable to tasking of diverse systems• Can be built on patterns that work well for human
groups
Rational Behavior Model (RBM)
Strategic• Declarative planning of goals while avoiding
obstacles and observing constraints.
Tactical• Operational control of navigation, tactical and
mission tasks. Sensor employment.
Execution• Low-level control tasks. Open-loop, closed-loop
commands for propulsors and effectors.
Twenty+ years of well-documented, progressive work
Background
Turing machine
Turing machine (TM) • Consists of a finite state machine (FSM) • Augmented by an external agent in form of a
potentially infinite memory • Realized as tape of an “incremental tape
recorder”
Often referenced but seldom used• Extensive theoretical development has shown
that Universal TM has greatest computational power
• Clumsy to program, infrequently used• Nevertheless, appealing basis for theoretical
design since it maps to general theory of any computation
Background
Prior work: Mission Execution Automaton
(MEA)Generalization of a Universal Turing Machine (UTM)• RBM Strategic level implements a UTM transition
engine• FSM encodes an arbitrary autonomous vehicle mission• RBM Tactical level implements the UTM external agent
Strengths• Missions (FSM) easily read and understood by
operators• Determinism allows exhaustive mission flow testing• Potential for “executable” specifications (no coding
required!)Upshot (or why any of this matters from an ethical standpoint)• Mathematical rigor provides operators the ability to fully
understand what the vehicle is going to do over the course of the mission
• The available Tactical-level behaviors inherently describe vehicle limits
Background
Strategic-level mission
control
Success/Failure mission logicfor each task
simply loop thru MEA goal tasks
Issue Tactical-
Level Order
Evaluate Respons
e
Success Follow-
on
Fail Follow-
on
Load Succeess Follow-on
Load Succeess Follow-on
Load Fail Follow-on
Stop
Start
Succee
dFail
Yes
Yes
NoNo
Strategic Level Mission
Execution Engine (MEE)
Background
Strategic Level Mission Flow Example
Mission flow is easily readable by human
operators
Decision trees are easily followed by
many different robots
Robot mission conductcan be independent of
software implementation
Phase 1Search Area A
Phase 3Search Area C
Phase 2Sample
Environment in Area B
Phase 4Rendezvous with UUV-2 in Area D
Phase 5Return to Base
Mission Complete
Recover
Mission AbortSurface/Scuttle
Fail Succeed Succee
dSuccee
d
Succeed
Succeed
Fail
Fail
Fail
Fail
Start
Background
Ethical constraints “inform” rather than “define” the mission
Constraints can be applied to individual rules or throughout
the entire mission
Pending ethical violations can be
treated as a goal “fail” or a third type of
transition condition
Eth
ics
Ru
les
Eth
ics
Ru
les
Eth
ics
Ru
les
Eth
ics
Ru
les
Eth
ics
Ru
les
Adding Ethics to an RBM Mission
Phase 1Search Area A
Phase 3Search Area C
Phase 2Sample
Environment in Area B
Phase 4Rendezvous with UUV-2 in Area D
Phase 5Return to Base
Mission Complete
Recover
Mission AbortSurface/Scuttle
Fail Succeed Succee
dSuccee
d
Succeed
Succeed
Fail
Fail
Fail
Fail
Start
Ethical Rules
MEE Update MEE for Ethical
ControlOriginal paradigm calls
for a Boolean response• Might consider a
pending ethical breach a “fail”
• But this is NOT and MEE requirement
Ternary response option allows for different sequence options
Issue Tactical-Level Order
Evaluate Response
Success Follow-on
Fail Follow-on
Load Success Follow-on
Load Success Follow-on
Load Fail Follow-on
Stop
Start
SucceedFa
il
Yes
Yes
NoNoN
oEth
ics
Ethics Follow-on
Yes
Load Fail Follow-on
Autonomous Vehicle Command Language (AVCL)
Ongoing objectives• Vehicle independent mission definition, mission
execution, and post-mission analysis• Realistic mission review, rehearsal and replay
(high-level mission flow, physically-based simulation)
• Increased autonomy & interoperability
Current products and efforts• XML vocabulary for goal and constraint definition
• Schema-defined terminology and structure• Fixed (growing) set of goal, behavior and constraint
types• Simulation environment (AUV Workbench)
Exemplar Mission in Simulation (launch)
Exemplar Mission in Simulation (phase 1)Search of Area A successful
Exemplar Mission in Simulation (phase 2)Sample environment in Area B successful
Exemplar Mission in Simulation (phase 3)Search of Area C successful
Exemplar Mission in Simulation (phase 4)Rendezvous with UUV-2 failed
Exemplar Mission in Simulation (phase 5)Transit to recovery position successful
Looking aheadObservations from work to date• AVCL goal and constraint definition—exhaustive
enumeration ultimately prove (impossible?)• Vehicle-independent does not mean vehicle-universal• Mission flow validation does not equate to trust in the
vehicle
Broader definitions for RBM goals and constraints• Goal: any discrete activity that can be executed
through implemented Tactical-level behaviors of a target vehicle while being monitored in real time (by the Tactical level) for success and failure
• Constraint: any atomic or complex Boolean condition that can be reliably evaluated in real time by the Tactical level implementation of a target level
But are these definitions enough to underpin a more thorough implementation of MEA-based ethical autonomy?
Probably not!
Future work
Goal and Constraint Definition
Description Logics (DL)• Well-researched mathematical formalism for
describing entities, characteristics and relationships
• Inherently supportive of reasoning and inference
Ontology: a mathematically rigorous definition of a domain of knowledge possibly including instantiations of individual entities within that domain• Provide a full, mathematical definition of “goal”
and “constraint” that aligns with the previous definitions
• Provide a means of applying these definitions to build and test arbitrary missions for specific target vehicles
Future work
Description Logic and Ontology Tools
Web Ontology Language (OWL)• W3C recommendation for semantic web ontology
definition• Implements a highly expressive DL (SROIQ+)
Open-source tools• Protégé ontology development environment• Ontology development environment• Application Programmer’s Interface (Java)• Reasoning engine for automated consistency checking,
satisfiability determination, rule-based inference, etc.• Large user base and support infrastructure
Future work
Candidate Student Thesis Projects
Explore missions that identify ethical conundrums related to autonomous robot operations that must be resolvable by a Tactical-level implementation
Utilize description logics to mathematically describe the nature autonomous vehicle goals, behaviors and constraints
Utilize open-source tools (Protégé) to extend AUV Workbench capabilities to work with OWL ontologies and ontology-based mission definitions
Build, test both operations order and ROEs for unmanned systems supporting fleet assets• Your topic here, perhaps…
Future work
Duane Davis, Ph.D., CDR USN (Ret.)
Code CS/DaNaval Postgraduate School
Monterey California 93943-5000 USA1-831-656-2239 work
Contact
Don Brutzman, Ph.D., LCDR USN (Ret.)
[email protected] [email protected]
http://faculty.nps.edu/brutzman
Code USW/Br, Naval Postgraduate SchoolMonterey California 93943-5000 USA
1-831-656-2149 work1-831-402-4809 cell
Contact
George R. Lucas, Jr., [email protected]
Professor of Ethics & Public PolicyNaval Postgraduate School
Monterey California 93943-5000 USA
Contact
Robert B. McGhee, Ph.D.
Emeritus Professor, Computer ScienceNaval Postgraduate School
Monterey California 93943-5000 USA
Contact