Unifying Model-based and Reactive Programmingwithin a Model-based Executive

Abstract . .

Real-time model-based deduction has recently emergedas a vital component in Al's tool box for develop-ing highly autonomous reactive systems . Yet one ofthe current hurdles towards developing model-basedreactive systems is the number of methods simultane-ously employed, and their corresponding melange ofprogramming and modeling languages . This paper of-fers an important step towards unification of reactiveand model-based programming, providing the capabil-ity to monitor mixed hardware/software systems . Weintroduce RMPL, a rich modeling language that com-bines probabilistic, constraint-based modeling with re-active programming constructs, while offering a simplesemantics in terms of hidden state Markov processes .We introduce probabilistic, hierarchical constraint au-tomata, which allow Markov processes to be expressedin a compact representation that preserves the modu-larity of RMPL programs . Finally, a model-based ex-ecutive, called RBurton is described that exploits thiscompact encoding to perform efficent simulation, beliefstate update and control sequence generation .

IntroductionHighly autonomous systems, such as NASA's DeepSpace One spacecraft(Muscettola et al. 1999) andRover prototypes, leverage many of the fruits of AI'swork on automated reasoning - planning and schedul-ing, task decomposition execution, model-based rea-soning and constraint satisfaction . Yet a likely showstopper to widely deploying this level of autonomy isthe multiplicity of AI modeling languages employed fortasks such as diagnosis, execution, control and planningfor these systems . Currently separate models of thesame artifacts are built for each of these tasks, leadingto considerable overhead on the part of the modelingteams - for the Deep Space One spacecraft, as much ashalf the total development time was devoted to ensuringthat the multiple models were mutually consistent .

This paper concentrates on a developing a unifiedlanguage for monitoring, diagnosis and recovery, andreactive execution . Key to this challenge is the devel-opment- of a unified language that can express a rich

tCaelum Research Corporation .

QR99 Loch Awe, Scotland

Brian C. Williams and Vineet Gupta tNASA Ames Research Center, MS 269-2

Moffett Field, CA 94035 USAE-mail: {williams, vguptalAptolemy .arc . nasa .gov

set of mixed hardware and software behaviors (ReactiveModel-based Programming Language - RMPL), a com-pact encoding of the underlying Markov process (hier-archical constraint, automata - HCA), and an executivefor this encoding that supports efficient. state estima-tion, monitoring and control generation (RBurton) .

RMPL. RMPL achieves expressiveness by mergingkey ideas from synchronous programming languages,qualitative modeling and Markov processes . Syn-chronous programming offers a class oflanguages (Halb-wachs 1993) developed for writing control programs forreactive systems (Harel & Pnueli 1985 ; Berry 1989) .Qualitative modeling and Markov processes providemeans for describing continuous processes and uncer-tainty.

RBurton achieves efficient execution through a care-ful generalization of state enumeration algorithms thatare successfully employed by the Sherlock(de Kleer& Williams 1989) and Livingstone(Williams & Nayak1996) systems on simpler modeling languages .

Monitoring software/hardware systems . Theapplications for which modeling in RMPL will be neces-sary are mixed hardware software systems . In any com-plex system, the overall control system and its individ-ual components are mixed hardware software systems .With respect to diagnosis this means that tracking thesystem's behavior requires tracking the software execu-tion at some level of abstraction . In addition, failurescan effect the execution trace of the software and hencesymptoms might manifest themselves as software errorconditions .A key feature of RMPL is that the execution and con-

trol software can also be written in RMPL, thus we cando our monitoring and diagnosis on the same modelsthat are executed - "What you monitor is what youexecute", a variant of Berry's principle* . This elimi-nates the need for separately maintaining the execution

* "What you prove is what you execute" (Berry 1989) .Berry argues that verification must be done directly on ex-ecutable programs, eliminating the gap between specifica-tions about which we prove properties, and the programsthat are supposed to implement them .

327

software and models of it for diagnosis, a potentially ex-pensive and error-prone task .

Organization of this paper .

We start with a sketchof RBurton . The first half of the paper then intro-duces hierarchical constraint automata, their determin-istic execution, and their expression using RMPL . Thedirect mapping from RMPL combinators to HCA, cou-pled with HCA's hierarchical representation avoids thestate explosion problem that frequently occurs whilecompiling large reactive programs .The second half of the paper turns to model-based ex-

ecution under uncertainty. First we generalize HCAs toa factored representation of partially observable Markovprocesses . We then develop RBurton's stochastic mon-itoring and execution capabilities, while leveraging offthe compact encoding offered by probabilistic HCA .The paper concludes with a discussion of related work .

The RBurton ExdcutiveThe execution task consists of controlling a physicalplant according to a stream of high-level commands(goals), in the face of unexpected behavior from thesystem . To accomplish this the executive controls somevariables of the plant, and senses the values of somesensors to determine the state of the plant .

RBurton Executive ExternalGoals

PlantModel

StateEstimatioi~lModule

Observables

I ControlsPlant

A schematic of RBurton is shown above. RBurtonconsists of two main components . The state estima-tion module determines the current most likely states ofthe plant from observed behavior using a plant model.This generalizes mode identification (MI), (Williams &Nayak 1996) . The key difference is the expressivenessof the modeling languages employed . RMPL allows arich set of embedded software behaviors to be modeled,hence RBurton's state estimator offers a powerful toolfor monitoring mixed software/hardware systems .RBurton's control sequencer executes a program for

controlling the plant that is also specified using RMPL .Actions are conditioned on external goals and proper-ties of the plant's current most likely state . Given mul-tiple feasible options, RBurton selects the course of ac-tion that maximizes immediate reward . As a controllanguage RXIPL offers the expressiveness of reactivelanguages like Esterel(Berry & Gonthier 1992), alongwith many of the goal-directed task decomposition andmonitoring capabilities supported by robotic execution

OR99 Loch Awe, Scotland

Controlprogram

ControlSequenceModule

languages like RAPS(Firby 1995), TCA(Simmons 1994)and ESL(Gat 1996) .

Example.

We will consider the Autonavigation sys-tem on the spacecraft Deep Space 1 . This system isused on the spacecraft once a week to perform smallcourse corrections . It takes pictures of three asteroids,and uses the difference between their actual locationsfrom their projected locations to determine the courseerror . This is then used by another system to deter-mine a new course . The plant model for this consistsof models of the camera, the thrusters which turn thespacecraft, and the memory system which stores thepictures . Each of these has various failure modes, whichare described in the models . The control program issuescommands to the thrusters to turn to each asteroid inturn, to the camera to take pictures, and to the mem-ory to store the pictures . If there is any error, it stopsthe sequence and takes a recovery action .We will need to make sure that RMPL is suffi-

ciently expressive to model this system . This behav-ior draws upon many of the constructs supported byLivingstone-qualitative interactions, multiple modes,concurrent operation and probabilistic transitions . Inaddition the model must embody a richer set of behav-iors that cannot be expressed within Livingstone's mod-eling language- preemption, forking concurrent pro-cesses and iteration . In the next few sections we willshow how these are combined into a succinct language .

Hierarchic Constraint AutomataRMPL programs may be viewed as specifications of par-tially observable Markov processes, that is probabilis-tic automata with partial observability . While Markovprocesses offer a natural way of thinking about reac-tive systems, as a direct encoding they are notoriouslyintractable . To develop an expressive, yet compact en-coding we introduce five key attributes . First, transi-tions are probabilistic, with associated costs . Second,the Markov process is factored into a set of concur-rent automata. Third, each state is labeled with a con-straint that holds whenever the automaton marks that.state . This allows an efficient, intentional encoding ofco-temporal processes, such as fluid flows . Fourth, au-tomata are arranged in a hierarchy - the state of anautomaton may itself be an automaton, which is ac-tivated by its parent . This enables the initiation andtermination of more complex concurrent and sequentialbehaviors than were possible with the modeling lan-guage used in Livingstone. Finally, each transition mayhave multiple targets, allowing an automaton to be inseveral states simultaneously, enabling a compact rep-resentation of recursive behaviors .

These attributes are a synthesis of representationsfrom several areas of computation. The first attributecomes from the area of Markov processes, and is essen-tial for tasks like stochastic control or failure analysisand repair . The second and third attributes are preva-lent in areas like digital systems and qualitative mod-

328

eling. The fourth and fifth are prevalent in the field ofsynchronous programming, and form the basis for re-active languages like Esterel(Berry & Gonthier 1992) .Lustre (Halbwachs, Caspi, & Pilaud 1991), Signal(Guer-nic et al. 1991) and State Charts(Harel 1987) . Togetherthey allow modeling of complex systems that involvesoftware, digital hardware and continuous processes.

Hierarchic constraint automata (HCA) incorporateeach of these attributes . An HCA models physical pro-cesses with changing interactions by enabling and dis-abling constraints within a constraint store (e .g ., a valveopening causes fuel to flow to an engine) . Transitionsbetween successive states are then conditioned on con-straints entailed by that store (e .g ., the presence or ab-sence of acceleration) .A constraint system (D, ~=)' is a set of tokens D,

closed under conjunction, together with an entailmentrelation ~--C D x D (Saraswat 1992) . The relationsatisfies the standard rules for conjunctiont.RBurton, uses propositional state logic as its con-

straint system . Each proposition is an assignmentxi = vii, where variable xi ranges over a finite domainD(xi). Propositions are composed into formulas usingthe standard logical connectives - and (n), or (V) andnot (~) . If a variable can take on multiple values, thenxi = vij is replaced with vii E xi. .

Definition 1 A deterministic hierarchical constraintautomaton S is a 5-tuple (E, O, II, Cp, Tp), where:

" E is a set of states, partitioned into primitive statesEP and composite states E, . Each composite stateitself is a hierarchical constraint automaton.

" O C E is a set of start states ." II is a set of variables with each xi E II ranging over a

finite domain D[xi] . C[II] denotes the set of all finitedomain constraints over H.

" Cp : EP -4 C[II], associates with each primitive statesi a finite domain constraint Cp(si) that holds when-ever si is marked.

" Tp : EP x C[II] -+ 2E associates with each primitivestate si a transition function Tp(si) . Each Tp(si)C[II] -4 2E, specifies a set of states to be marked attime t + 1, given assignments to fl at time t .

Simulating Deterministic HCA

A full marking of an automaton is a subset of statesof an automaton, together with the start states ofany composite states in the marking. This is com-puted recursively from an initial set of states A7 usingMF(M) = MU U{MF(O(s)) I s E M, s composite} .Given a full markingMon an automaton A, the func-

tion Step(A, M) computes a new full marking corre-sponding to the automaton transitioning one time step .

t 1) a P a (identity)2) a A b [-- a and a n b ~= b (A elimination)3) a

b and b n c ~= d implies a n c ~= d (cut)4)'a

U ana'a f= c iinpiies a ~-- d it c kip iiicl -cniLUOinr)~~

QR99 Loch Awe, Scotland

Step(,4, A-1) : :l .

M1 := {s E A7 I s primitive}2 .

C := n~ tii, cp(s)3 .

M2 := v8EM1 TP(s,C)4 .

return A4F(M2)Step 1 throws away any composite marked states,

they are uninteresting as they lack associated con-straints or transitions . Step 2 computes the conjunctionof the constraints implied by all the primitive states in1'11 . Step 3 computes for each primitive state the setof states it. transitions to after one time step . In step4, applying -MF to the union of these states marks thestart states of any composite state. The result is thefull marking for the next time step .A trajectory of an automaton A is a finite or infinite

sequence of markings mo , ml , . . ., such that mo is theinitial marking MF(O(A)), and for each i > 0, mi+i =Step(A, mi) .

Elaborating on step 3, we represent the transitionfunction for each primitive state Tp(s) as a set of pairs(li, si), where si E E, and li is a set of labels of the form~-- c or K- c, for some c E C[II]. This is the traditionalrepresentation of transitions, as a labeled arc in a graph.If the automaton is in state s, then at the next instant.it will go to all states si whose label li is entailed byconstraints C, as computed in the second step of thealgorithm . l i is said to be entailed by C, written C k-- li,ifVHcEli .CHc,andforeach ~'- cEli .C K c. Itis straightforward to translate this representation intoour formal representation : Tp(s,C) = {si I C ~= li} .Two properties of these transitions are distinctive :

Transitions are conditional on what can be deduced,not just what is explicitly assigned, and transitions areenabled based on lack of information .

Step provides adeterministic simulator for the plant,when applied to an HCA that specifies a plant model.Alternatively Step provides a deterministic version ofthe control sequencer for RBurton, by placing appro-priate restrictions on the control HCA. Constraints at-tached to primitive , tates on this HCA are restrictedto control assignments, while transition labels are con-ditioned on the external goals and the estimated cur-rent state. The set of active constraints collected frommarked states during step 2 of the algorithm is then theset of control actions to be output to the plant.

A Simple Example

VVe illustrate HCA with a sim-ple automaton. In this picture c represents aconstraint,start states of an automaton are marked with arrows,and all transitions are labeled . For convenience we usec to denote the label ~-- c, and c to denote the label K c.Circles represent primitive states, while rectangles rep-resent composite states .The automaton has two start states, both of which

are composite. Every transition is labeled K d, henceall transitions are disabled and the automaton is pre-empted whenever d becomes true . The first state hasone primitive state, which asserts the constraint c. Ifd does not hold, then it _goes back to itself - thus it

32 9

repeatedly asserts c until d becomes true . The secondautomaton has a primitive start state . Once again, atanytime if d becomes true, the entire automaton willimmediately terminate . Otherwise it waits until a be-comes true, and then goes to its second state, which iscomposite . This automaton has one start state, whichit repeats at every time instant until d holds. In ad-dition, it starts another automaton, which checks if eholds, and if true generates b in the next state . Thus,the behavior of the overall automaton is as follows : itstarts asserting c at every time instant . If a becomestrue, then at every instant thereafter it checks if e istrue, and asserts b in the succeeding instant . Through-out it watches for d to become true, and if so halts . AnRNIPL program that produces an equivalent automatonis :

do(always c,when a donext always if e. thennext b)

watching d

The combinators of this program are developed dur-ing the remainder of this paper .

RMPL: Primitive CombinatorsWe now present the syntax for the reactive model-basedprogramming language . Our preferred approach is tointroduce a minimum set of primitives, used to con-struct programs - each primitive that we add to thelanguage is driven by a desired feature . We then de-fine on top of these primitives a variety of programcombinators, such as those used in the simple exam-ple above, that make the language usable . The prim-itives are driven by the need to write reactive controlsoftware in the language, as well as to model physicalsystems. As mentioned earlier, to write reactive controlprograms we require combinators for preemption, con-ditional branching and iteration . For modeling hard-ware, we require constructs for representing constraintsand probability. Finally we need logical concurrency tobe able to compose models and programs together .As we introduce each primitive we show how to con-

struct its corresponding automaton . In these definitionslower case letters, like c, denote constraints, while uppercase letters, like A and B, denote automata . The term"theory" refers to the set of all constraints associatedwith marked primitive states at some time point.

QR99 Loch Awe, Scotland

c. This program asserts that constraint c is true atthe initial instant of time . This construct is used torepresent co-temporal interactions, such as a qualita-tive constraint between fluid flow and pressure . Theautomaton for it is :

Note that the start state in this automaton has noexit transitions, so after this automaton asserts c in thefirst time instant it terminates .

if c thennext A.

This program starts behaving likeA in the next instant if the current theory entails c.This is the basic conditional branch construct. Giventhe automaton for A, we construct an automaton forif c thennext A by adding a new start state, and goingfrom this state to A if c is entailed .

unless c thennext A.

This program executes A inthe next instant if the current theory does not entail c.The automaton for this is similar to the automaton forif c thennext A . This is the basic construct for build-ing preemption constructs -- it is the only one thatintroduces conditions K c (written in the automatonas c) . This introduces non-monotonicity, but since thenon-monotonic conditions trigger effects in the next in-stant, the logic is stratified and monotonic in each state .This avoids the kinds of causal paradoxes possible inlanguages like Esterell .

We also allow generalized sequences for if . . . thenand unless . . . then, terTninated with thennext . (e.g .if c then unless d thennext A) . The compilation toautomata proceeds exactly as above.

A, B.

This is the parallel composition of two au-tomata, and is the basic construct for introducing con-currency. The composite automaton has two startstates, given by the two automata for A and B.

2Esterel allows programs like unless c then c, whichhave no behavior . These are rejected by its compiler.

33 0

always A .

This program starts a new copy of A ateach instant of time - this is the only iteration con-struct needed . The automaton is produced by markingA as astart state and by introducing an additional newstart state. This state has the responsibility of initiat-ing A during every time step after the first . A tran-sition back to itself ensures that this state is alwaysmarked . A second transition to A puts a new mark onthe start state of A at every next step, each time invok-ing a virtual copy of A. The ability of an automatonto have multiple states marked simultaneously is keyto this novel encoding, which avoids requiring explicitcopies of A.

always A

Adding Uncertainty to RMPLThe presentation has concentrated thus far on a de-terministic language and an algorithm for deterministi-cally executing hierarchical constraint automata . Thiscan be used to simulate a deterministic plant or to gen-erate deterministic plant control sequences. Howeverphysical plant models are frequently uncertain, thusforcing us to build probabilistic models . The plant'sobservables are used to predict its internal state, andto determine when it deviates from the intended effect .Uncertainty is modeled by introducing transition prob-abilities, turning the plant into a partially observableMarkov process. The efficient estimation of the inter-nal state for complex systems is notoriously difficult.An efficient estimate of the plant's possible states (the

belief state) is enabled through the compact encodingof the plant's model in terms of hierarchical constraintautomata . This estimate is used to guide the evaluationof the control program at each time tick . To expressprobabilistic knowledge into RMPL we introduce theprobabilistic combinator choose

choose [A with p, B with q] .

This combinator re-duces to A with probability p, to B with probability q,and so on. In order to ensure that the current theorydoes not depend upon the probabilistic choices madein the current state, we make the following restriction- all assertions of constraints in A and B must be inthe scope of a next . This restriction ensures that noconstraints are associated with the start states of A andB (technically the attached constraint is "true"), andthus the probabilities are associated only with transi-tions. The corresponding automaton is encoded with asingle probabilistic start transition, which allows us tochoose between A and B.To incorporate probabilistic transitions into HCA we

change the definition of Tp . Recall for deterministicHCA that Tp(si) denotes a single transition function .For probabilistic HCA Tp(si) denotes a distribution

QR99 Loch Awe, Scotland

over transition functions Tp-' (si), whose probabilitiesP(TP-1 (si)) sum to l.

Tp(si) is encoded as a probabilistic, AND-ORtree . This supports a simple transformation of nestedchoose combinators to probabilistic HCA. Each leafof this tree is labeled with a set of one or more targetstates in E, which the automaton transitions to in thenext time tick .The branches ai -~ bid of a probabilistic OR node

ai represent a distribution over a disjoint set of alter-natives, and are labeled with conditional probabilitiesP[bid I ai] . The probability of branches emanating fromeach ai sum to unity.The branches of a deterministic AND node represent

an inclusive set of choices. Each branch is labeled by aset of conditions lid of the form ~= 0 or K ¢, where 0 isany formula in propositional state logic over variablesII . Every branch whose conditions are satisfied by thecurrent state is taken .

Each AND-OR tree is compiled into a two level tree(shown above), with the root node being a probabilisticOR, and its children being deterministic ANDs . Compi-lation is performed using distributivity, as shown above,and commutativity, which allows adjacent AND nodesto be merged, by taking conjunctions of labels, and ad-jacent OR nodes to be merged, by taking products ofprobabilities.

This two level tree is a direct encoding of Tp(si) .Each AND node represents one of the transition func-tions Tpi (si), while the probability on the OR branch,terminating on this AND node, denotes P(Tpi (si)) .

RBurton: State EstimationTo implement belief state update recall that a prob-abilistic HCA encodes a partially observable Markovprocess. A partially observable Markov process can bedescribed as a tuple (E, Nl, O, P T, P(9) . E, Nl and Odenote finite sets offeasible states si, control actions pi,and observations of respectively . The state transitionfunction, PT[si(0,pil0 r-) si (t+i)] denotes the proba-bility that si(t+i) is the next state, given current statesilt) and control action pi (t) at time t. The observationfunction, Po[si(t) t--> oi(t

)] denotes the probability that

331

oi (0 is observed, given state si(t) at time t .RBurton incrementally updates the plant belief state,

conditioned on each control action sent and each obser-vation received, respectively :U(0t+l)[Si]

=

P[Si(t+l) I Oao )

. . . , Ove), 1tuo ) . . . Pv,)]a(t+l-) [8i]

-

P[si(t+l)

1020), . . . , Ov,+1), Ftvp) . . . Pv e)]Exploiting the hlarkov property, the belief state at timet + 1 is computed from the belief state and control ac-tions at time t and observations at t + 1 using the stan-dard equations :

n

01(t+18) si

=

or (.t+l) Si

P0[si HOk][ ]

[ ]y~ 1 O(ot+l)[S7]P(7[S7 HOk]

To calculate PT recall that al transition T is com-posed of a set of primitive transitions, one for eachmarked primitive state . Assuming conditional inde-pendence of primitive transition probabilities, given thecurrent marking, the combined probability of each setis the product of the primitive transition probabilitiesof the set . This is analogous to the various indepen-dence of failure assumptions exploited by systems likeGDE(de Kleer & Williams 1987), Sherlock(de Kleer &Williams 1989) and Livingstone. However unlike thesesystems, multiple sets of transitions may go to the salvetarget marking. This is a consequence of the fact thatin an HCA primitive states have multiple next states .Hence the transition probabilities for all transitions go-ing to the same target must be summed according tothe above equation for a('t+l)[Sj] .Given P-T, the belief update algorithm for o('t+l)[si]

is a modified version of the Step algorithm presentedearlier. This new version of Step returns a set of mark-ings, each with its own probability. Step 3a builds thesets of possible primitive transitions - here the prod-uct is a Cartesian product of sets . Step 3b computesthe combined next state marking and transition prob-ability of each set . Step 3c sums the probability of allcomposite transitions with the same target :

Stepp(A, M) : :1 .

A71 := {s E M I s primitive}2. ~3a.

M2a:= EXrl TP(S, C)3b .

M2b := I(MF(U

j Si), Hi'=1 pi)

C := A 8 1N1 Cp(s)

((Si Ipi), . . .,(Sn,pn)) E M2aI3c.

M2 := {(S, 1:(S p)EM2bP) I (S, -) E M2b}4.

return M2The best first enumeration algorithms developed for

Sherlock and Livingstone are directly used by RBurtonto generate the composite transitions in step 3a and bin order from most to least likely . However, since thecorrespondence between transitions and next states ismany to one, there is no guarantee that the belief statesare enumerated in decreasing order.

GR99 Loch Awe, Scotland

Instead we assume that most of the probability den-sity resides in the few leading candidate transition sets .Hence a best first enumeration of the few leading tran-sition sets will quickly lead to a reasonable approxima-tion . We enumerate transitions in decreasing order untilmost of the probability density space is covered (e.g .,95`x), and then perform step 3c to merge the results.

Computing

a(t+l')[s,i]

requires

PO[si(t) H oi(0].P0 is computed using the standard approach in model-based reasoning, first introduced within the GDE sys-tem . For each variable assignment in each new obser-vation, RBurton uses the model, current state and pre-vious observation to predict or refute this assignment,giving it probability 1 or 0 respectively. If no predic-tion is made, then a prior distribution on observables isassumed (e.g ., 1/n for n possible values).

RBurton : Mode Reconfiguration andSequencing

A full decision theoretic executive that maximizes ex-pected reward using HCA is well beyond the scope ofthis paper. However, a simplified executive that doestask decomposition based on immediate rewards can beeasily constructed by a simple extension to the abovemodel. Thus RBurton maximizes immediate rewardunder the assumption that the most likely estimatedstate is correct . We further assume that rewards areadditive . The hierarchical automaton provides a wayof structuring tasks, subtasks and solution methods.We restrict the constraint's c of a control program to

plant control assignments . In addition, to support se-lection of methods for tasks, we replace the probabilis-tic combinator choose with an analogous combinatorbased on reward :

choosereward [A with p, B with q] .

This combina-tor reduces to A with reward p, to B with reward q,and so on . choosereward has restrictions analogousto choose that associate rewards only to expressionscontaining next .The AND-OR Tree formed by nested applications of

choosereward is analogous to choose . The tree isreduced in a similar manner, except that rewards areadded while probabilities are multiplied .

Control sequence generation again uses a variation ofStep . For step 3 of this algorithm a best first enumer-ation algorithm is given the sets of enabled transitionsfrom each primitive state that is marked in the mostlikely current marking. During the enumeration it mustrule out any sets of transitions that lead to an incon-sistent (conflicting) control assignment . It then returnsthe set of transitions that maximize combined reward .This is analogous in RAPS(Firby 1995) to selecting ap-plicable methods based on priority numbers .

Extending RMPL: Definable operatorsGiven the basic operators defined earlier, we can de-fine a variety of common language constructs, snaking

332

the task of programming in reactive MPL considerablyeasier . Common constructs in RMPL include recursion,next, sequencing and preemption .

Recursion and procedure definitions. Given adeclaration P : : A[P], where A may contain oc-currence~ of procedure name P, we replace it. byalways ifp then A[plP] . At each time tick this looksto see if p is asserted (corresponding to P being in-voked), and if so starts A .

next A. This is simply if true thennext A. Wecan also define if c thennext A elsenext B asif c thennext A, unless c thennext B.

if c then A .

This construct, has the effect of start-ing A at the time instant in which c becomes true.It can be defined as follows, where the expression tothe left of the equality is replaced with the expressionon the right. (note that sequences of if . . . then andunless . . . then, terminated with thennext are OK) :

ifcthen d=c->dif c then always A =

if c then A, if c thennext always Aif c then (A,B) =

if c then A, if c then Bif c then choose [A with p, B with q] _

choose [if c then A with p, if c then B with q]

A; B.

This does sequential composition of A and B.It keeps doing A until A is finished . Then it starts B.It can be written in terms of the other constructs bydetecting the termination of A by a proposition, andusing that to trigger B. RMPL detects the termina-tion of A by a case analysis of the structure of A (see(Fromherz, Gupta, & Saraswat 1997) for details) .

do A watching c.

This is a weak preemption oper-ator. It executes A, but if c becomes true in any timeinstant, it terminates execution of A in the next instant.The automaton for this is derived from the automatonfor A by adding the label K c on all transitions in A.

suspend A on c reactivate on d.

This is like the"Control - Z, fg" pair of Unix - it suspends the pro-cess when c becomes true, and restarts it from the samepoint when d becomes true .

when c donext A.

This starts A at the instant afterthe first one in which c becomes true . It is a temporallyextended version of if c thennext A. It can be codedusing recursion and if c thennext A.

DS1 Optical Navigation ExampleTo illustrate RMPL further, we give some fragments ofthe RMPL models for the Autonav example. MICASis a miniature camera on DS1 .

MICAS :: always {choose

QR99 Loch Awe, Scotland

if MICASon thenif TurnMicasOff thennext MICASoffelsenext MICASon,

if MICASoff then . . .,if MICASfail then . . . .

} with 0.99,next MICASfail with 0.01

This model shows how probability is used - at eachstep, MICAS will fail with probability 0.01. Other-wise it will continue to exhibit normal behavior, duringwhich it can respond to commands . Probability canalso be used to simulate the effects of reseting a failedMICAS - a reset corrects the failure with some prob-ability.AutoNav is the control routine that. performs the

course correction .Autonav() : :TurnMica9On,if IPSon thennext SwitchIPSStandBy,do when IPSstandby A MICASon donext {

TakePicture(1) ; . . .

TurnMicasOff,OpticalNavigation ()

} watching MICASfail V OpticalNavError,when MICASfail donext {MICASReset, Autonav()},when OpticalNavError donext AutoNavFailed

This routine shows how iteration and preemption areused . The preemption operator do . . . watchingaborts the current program in the face of an error, andrestarts the program. The modular nature of the pre-emption is particularly useful here - the program in-side the do . . . watching can be written without anyconcern for failures, which are captured by the preemp-tion operation, somewhat like exceptions in Java .

Discussion and Related WorkThe RMPL compiler is written in C, and generates hi-erarchical constraint automata as its target . This sup-ports all primitive combinators and a variety of definedcombinators. RBurton is written in Lisp, and buildsupon the best-first enumeration code at the heart of theLivingstone system . In addition the language is suffi-ciently expressive and compact to support the full DSlspacecraft models developed for Livingstone. RBur-ton's behavior is equivalent to Livingstone for thoseexamples .

Turning to related work, RMPL synthesizes ideas un-derlying constraint-based modeling, synchronous pro-gramming languages and Markov processes. Syn-chronous programming languages (Halbwachs 1993 ;Berry & Gonthier 1992 ; Halbwachs, Caspi, & Pilaud1991 ; Guernic et al. 1991 ; Harel 1987 ; Saraswat, Ja-gadeesan, & Gupta 1996) were developed for writing

33 3

control code for reactive systems . They are based on thePerfect Synchrony Hypothesis - a program reacts in-stantaneously to its inputs . Synchronous programminglanguages exhibit logical concurrency, orthogonal pre-emption, multiform time and determinacy, which Berryhas convincingly argued are necessary characteristicsfor reactive programming . RMPL is a synchronous lan-guage, and satisfies all these characteristics .

In addition, RMPL is distinguished by the adoptionof Markov processes as its underlying model, its treat-ment of partial observability and its extensive use ofcontraint modeling to observe hidden state . This pro-vides a rich language for continuous process, failure,uncertainty and repair .As previously discussed, RMPL and RBurton over-

lap substantially with AI robotic execution languagesRAPS, ESL and TCA. For example, method selection,monitoring, preemption and concurrent execution arecore elements of these languages,, shared with RMPL .One key difference is that R} ,IPL's constructs fully

cover synchronous programing, hence moving towardsa unification of the executive with the underlyingreal-time language . In addition RBurton's deduc-tive monitoring capability handles a rich set of soft-ware/hardware models that go well beyond those han-dled by systems like Livingstone . This moves executionlanguages towards a unification with model-based, de-ductive monitoring .

Finally, note that hierarchical state diagrams, likeState Charts(Harel 1987), are becoming common toolsfor system engineers to write real-time specifications .These specifications are naturally expressed withinRMPL, due to RMPL's simple correspondence with hi-erarchical constraint automata, which are closely re-lated to state charts . Together this offers a four wayunification between synchronous programming, roboticexecution, model-based autonomy and real-time speci-fication, - a significant step towards our original goal .

Nevertheless substantial work remains . Many execu-tion and control capabilities key to highly autonomoussystems fall well outside the scope of RMPL and RBur-ton . For example, RNIPL has no construct for express-ing metric time . Hence RBurton cannot execute ormonitor temporal plans without the aid of an executivelike RAPS or Remote Agent's Exec . In addition, out-side of monitoring, RBurton does not employ any de-duction or planning during control sequence generation .Unifying the kinds of sequence generation capabiltiesthat are the hallmark of systems like HSTS(Muscettola1994) and Bllrton(Williams & Nayak 1997), requiressignificant research .

ReferencesBenveniste, A., and Berry, G ., eds . 1991 . Another Look atReal-time Systems, volume 79:9 .Berry, G., and Gonthier, G . 1992 . The ESTEREL pro-gramming language : Design, semantics and implementa-tion . Science of Computer Programming 19(2) :87 - 152 .

QR99 Loch Awe, Scotland

Berry, G . 1989 . Real-time programming : General purposeor special-purpose languages . In Ritter, G., ed ., Informa-tion Processing 89, 11 - 17 . Elsevier .de Kleer, J ., and Williams, B . C . 1987 . Diagnosing multiplefaults. Artificial Intelligence 32(1):97-130 .de Kleer, J ., and Williams, B . C . 1989 . Diagnosis withbehavioral modes . In Proceedings of IJCAI-89, 1324-1330 .Firby, R . J . 1995 . The RAP language manual . Ani-mate Agent Project Working Note AAP-6, University ofChicago .Fromherz, M . ; Gupta, V . ; and Saraswat, V. 1997 . cc - Ageneric framework for domain specific languges . In POPLWorkshop on Domain Specific Languages.Gat, E . 1996 . Esl : A language for supporting robust planexecution in embedded autonomous agents . In Proceedingsof the 1996 AAAI Fall Symposium on Plan Execution.Guernic, P. L . ; Borgne, M . L . ; Gauthier, T . ; and Maire,C . L . 1991 . Programming real time applications with SIG-NAL. In Proceedings of the IEEE (1991) 1321-1336 .Halbwachs, N. ; Caspi, P . ; and Pilaud, D . 1991 . The syn-chronous programming language LUSTRE . In Proceedingsof the IEEE (1991) 1305-1320 .Halbwachs, N . 1993 . Synchronous programming of reac-tive systems. Series in Engineering and Computer Science .Kluwer Academic .Harel, D., and Pnueli, A . 1985 . Logics and Models ofConcurrent Systems, volume 13 . NATO Advanced StudyInstitute . chapter On the development of reactive systems,471-498 .Harel, D . 1987 . Statechaats : A visual approach to complexsystems . Science of Computer Programming 8:231 - 274 .Muscettola, N . ; Nayak, P. P . ; Pell, B . ; and Williams, B . C .1999 . The new millennium remote agent : To boldly gowhere no ai system has gone before . Artificial Intelligence100 .Muscettola, N . 1994 . HSTS: Integrating planning andscheduling. In Fox, M ., and Zweben, NI ., eds ., IntelligentScheduling . Morgan Kaufmann .Saraswat, V . A . ; Jagadeesan, R . ; and Gupta, V . 1996 .Timed Default Concurrent Constraint Programming . J ofSymbolic Computation 22(5-6):475-520 .Saraswat, V . A . 1992 . The Category of Constraint Systemsis Cartesian-closed . In Proc . 7th IEEE Symp . on Logic inComputer Science, Santa Cruz.Simmons, R . 1994 . Structured control for autonomousrobots . IEEE Transactions on Robotics and Automation10(1) .Williams, B . C ., and Nayak, P . P. 1996 . A model-based ap-proach to reactive self-configuring systems . In Proceedingsof AAAI-96, 971-978 .Williams, B . C ., and Nayak, P. P . 1997 . A reactive plannerfor a model-based executive . In Proceedings of IJCAI-97.

334