intelligent systems branch johnson space center engineering directorate automation, robotics and...

Intelligent Systems Branch

Johnson Space CenterEngineering Directorate

Automation, Robotics and Simulation Division

Hierarchical Architectures for Controlling Real-World Agents

David KortenkampPete Bonasso

NASA Johnson Space CenterMetrica Inc./TRACLabs

[email protected]




Robot Architectures

• Sense Plan Act: Shakey 1969

• Subsumption: Brooks 1986

• Want power of deliberation and speed of reactivity

• Hybrid architectures [Erann Gat “Three Layer Architectures” in Artificial Intelligence and Mobile Robots 1998]




3T

World

1/10 sec. limited state limited projection

memory interpreter

task1 subtask subtasktask2task3

1 sec.memory of

immediate actionsno projection

10s sec.persistent state

and choicesprojection




Multi-agent

path planner

simulationIVHM

scheduler

Planner

Planner

Planner




3T at JSC• Particular instantiation

of hybrid architecture– Skill manager– RAPS– Adversarial Planner

• Robotic and non-robotic applications

• User interaction




Skills• Small pieces of C code

that do something• Data passed from one

skill to another• Actions and events• Skill manager

– Handles scheduling of skills

– Handles data transfer between skills

– Handles comms with second tier

sensor sensor actuator

to RAPS




RAPS• Jim Firby’s Reactive

Action Packages [Firby 1989]

• Library of tasks• Conceptual memory• Run-time

decomposition of tasks via context

• Tasks bottom-out in activation of skills

Agendatask

sub-task

sub-task

primitive

primitive

wait-for

RAP Memory

RAP Library

RAPInterpreter

to/from skills




Example RAP(define-rap (bioreactor-startup ?configuration) (language-mappings (applicability (and (isa ?object i-bwp-system) (or (isa ?state i-stand-alone-drain-state) (isa ?state i-on) (isa ?state i-stand-alone-reject-state) (isa ?state i-integrated-state) (isa ?state i-innoc-no-feed-state)))) (method integrated-or-stand-alone (context (and (not (= ?configuration innoc-no-feed)) (manual-valve-position pbbwpskm bio_innoc ?value ?error) (= ?value open) (default-timeout ?to) (nominal-pump-speed pbbwpskm recycle ?nmrc))) (task-net (sequence (t0 (stop-watchdog pbbwpskm)) (t0-1 (kill-monitors monitor-bioreactor)) (t1 (configure-bwp-valves ?configuration)) (t2 (set-pump-speeds start) (update-bwp-text ?configuration)) (t2-2 (start-watchdog pbbwpskm)) (t3 (install-goal (quote (monitor-bioreactor ?configuration)) 3 => ?mgoal)))))




Language in RAPS




Adversarial Planner (AP)• Non-linear, hierarchical task net (HTN) planner developed at MITRE• Multi-agent• Some scheduling• Leaves of plan are matched to top-level RAPS in RAP library

– RAPS provides abstraction from low-level for planner

• Execution monitoring




AP Example(Operator replace-crystals

:purpose (state crystals replaced):agents (?robot ?human):constraints ((instance-of ?robot ‘robot)

(instance-of ?human ‘human)):preconditions ((state bay opened)):plot (sequential

(covers(monitor-crystals ?robot in-monitor)(display ?human monitored))(crystals ?robot are-replaced))

:effects ((state crystals replaced)))




Integrated Application

• Four skill managers• RAPS (100s of RAPS >

50,000 lines of Lisp code)• Scheduler• 24/7 autonomous

operation• 18 months of operation• AI Magazine next issue




Skills-------------------------------------------------------Skills -- for the RO agent----------------------------------Name ROType deviceParams intervalOuts noneFunction: A device skill that gets all the sensor values and provides them to the other skills. Also sends commands to the pumps and valves. Also every interval seconds, this skill broadcasts a data message with the values of all the channels listed above to the IPC server so that clients (e.g., a logging facility) can access them (see the IPC structure at the end of this document).

Name valve_positionType queryParams valve (process/pps_select)Outs value (for process:primary/secondary/purge/off/unknown; for pps_select:pps/tank/reject/off/unknown), and result (okay or Err)Function: Checks V02 or V03. One of lines V02_i1 through V02_i3 or V03_i1 through V03_i3 will be hi, and the rest will be low. If all are low, the result is off. Any other pattern is unknown.

Name valve_atType eventParams valve (process/pps_select), value (for process:primary/secondary/purge/off; for pps_select:pps/tank/reject/off)Outs result (okay/ERR) Function: Waits for V02_i1 through V02_i3 or V03_i1 through V03_i3 to indicate value (see the valve_position skill). When the condition is achieved the event returns result.

Name turn_valveType blockParams valve (process/pps_select), value (for process:primary/secondary/purge/off; for pps_select:pps/tank/reject/off)Outs noneFunction: Sets one of V02_o1 through V02_03 to hi the rest to low, except for off when all lines will be set lo.




RAPs(define-primitive-event (valve-at ?agent ?valve ?open-closed ?error) (event-definition (:valve_at (:valve . ?valve) (:value . ?open-closed))) (event-values :bound :bound :bound :unbound)) (define-rap (turn-valve-p ?agent ?valve ?open-closed ?timeout) (succeed (and (valve-position ?agent ?valve ?value ?error) (= ?value ?open-closed))) (timeout ?timeout) (method (primitive (enable (:turn_valve (:valve . ?valve) (:value . ?open-closed)) (wait-for (valve-at ?agent ?valve ?open-closed ?result) :succeed (?result)) (disable :above) )) )(define-rap (processing-start ?stage ?adjust-time) ...(method purge (context (and (= ?stage purge) (valve-position roskm pps_select ?old-pos ?error) (= ?old-pos pps) (nominal-pump-speed roskm feed ?wwsp) (default-timeout ?dto))) (task-net (sequence (t1 (syringe-pump-p roskm start feed ?wwsp 30)) (t2 (water-flowing-p roskm stop recirc 0 ?dto)) (t3 (turn-valve-p roskm pps_select reject ?dto)) (t4 (turn-valve-p roskm process purge ?dto)) (t5 (turn-valve-p roskm pps_select tank ?dto)))))




Scheduler




Learning in 3T• Intra-tier learning

• Inter-tier learning– Planning RAPS– RAPS behaviors

• Automatic activation of behaviors

– Behaviors RAPS– RAPS Planning




Motivation in 3T• Goals and motivations come from outside or are

pre-built for the specific application– Ideally motivations would come into the top layer and

then “trickle down” to the other layers– However, often useful to manually place goals on the

RAPS agenda via GUI or NL– Less useful to enable/disable skills except for testing

• Resource allocation mostly at top-level, some mechanisms in RAPS to handle contention for limited resources




Adjustable Autonomy• Allows system to operate with dynamically

varying levels of independence, intelligence and control

• Some levels of control– Completely in control– Shared control– Supervising manual control

• System still updates state, world model, etc. through its sensors




sensors

perceptual memorylocal, short-term, dynamic

robot-centric: where, identity, role

object ontologyglobal, long-term, static

allo-centric: relationships, roles, recognition, etc.

DAML/OIL

task-levelcontrol

how to do things,procedures,

monitors, etc.

controlmulti-agent,

distributed, real-timeclosed-loop

actuators

RepresentationReasoning

about worldReasoningabout task

spatial reasoning

learning

task planning

object recognition

context

objectspriming

color, neural networksstereo vision, etc.

World

Generating tasksequences, monitoring

resourcestemporal reasoning

contextual reasoninggoals

set points

intelligent systems branch johnson space center engineering directorate automation, robotics and...

Documents

state isa

simulation division

state iinnoc

persistent state

mgoal slide

feedstate method

choices projection slide

configuration innoc