Implementing Instructions for Autonomous EntitiesCarlos Ho Shih Ning

Divisão de Processamento de ImagensInstituto Nacional de Pesquisas Espaciais - INPE

Caixa Postal 515 - 12.227-010 São José dos Campos, SP - Brasilho@dpi.inpe.br

Abstract: This paper presents an AI based technique for automatic control of several robotscollaboratively undertaking tasks within a dynamic environment. In lhe particular applicationdescribed, that of computer animation, lhe robots have human form and are endowed with a repertoireof autonomous behaviour. The robots are endowed with a degree of intelligence enablíng them torespond to lhe high levei instructions and to events in lhe environment. An application demonstratingthis technique has been simu1ated and visualised using computer graphics.

Resumo: Este artigo apresenta uma técnica baseada em IA para o controle de diversos robos querealizam tarefas dentro de um ambiente dinâmico. Nesta aplicação em particular em animação porcomputador, os robos tem a forma humana e são dotados de um repertório de movimentos autonomos.Os robos tem um grau de inteligência que os permite responder às instruções de alto nível e aoseventos do ambiente. Uma aplicação utilizando esta técnica foi simulada e visualizada utilizandocomputação gráfica.

1 IntroductionThe systern to be described is concerned with lhecontrol of a number of agents, specifical1y robots,that have autonomous behaviour and can react tochanges in lhe environment [Ho, 1996]. Theproblem of controlling lhe behaviour of a collectionof agents is a general one and lhe frameworkdeveloped is adaptable to many applications.

Many high-1evel programming languages havebeen developed to control the motion of robots inlhe areas of computer animation (Calvert et al.,1994; Morawetz and Cal vert, 1990] and robotics(Arkin, 1987; Brooks and Connell, 1986; Meijer etai., 1991]. However such programming languagesdo not provide a complete solution and additionalsoftware is necessary to imp1ement complexmotions. Thus, in lhe present work, lhe approach ofbehavioural control [Calvert et al., 1994; Ridsdaleet al., 1986; Sun and Green, 1993] has beenadopted and a control structure or framework hasbeen implemented based on lhe blackboard model[Engelmore and Morgan, 1988]. This frameworkdoes not rule out existing work on motion control(Georgeff, 1990; Hayes-Roth, 1993], but can beadapted to support it. An application demonstratingthis technique has been simulated and visualisedusing computer graphics.

2 Requirernents of the Sirnulated EnvironrnentThe motion of the robot figures can be regarded asactions [Wilkins, 1988] or processes that are created,suspended, interrupted, and completed asynchronously.Some of lhe characteristics considered desirable for asimulated animation of robots are :• a multiplicity of objects that may have similar, or

different, behaviour;• agents that perform actions in response to different

sources of stimuli in lhe environment;• provision for lhe execution of actions that cause

changes in lhe environment which can be perceivedby lhe agents;

• a decision mechanism, based on criteria, to chooselhe most appropriate action from severa! possiblealternatives.

Based on lhe above requirements, the activities ofthe simulated environment are controlled by ' aknowledge-based framework comprising four co-ordinated types of control:• scheduling. The scheduling or interruption of actions

as required by lhe contexto• instruction. A control structure from which may be

derived lhe planning and execution of activities in thepursuit of goals.

• task. The execution of primitive motions whichrequire lhe allocation of resources .

477

Fig. 2. The three controI stages of processo

Reasoning Component

frame instruction.default state is plan_selectiondefault type is instructiondefault history is nonedefault parameter_list is []default parameter is []defauIt plan_name is nothingdefault plan_sequence is []default child is nothingdefauIt quality is primary 11 or secondarydefault owner is system 11 user script

environment. An instruction can also be interpreted as amachine instruction that is generated consequent uponevents in the environment and carried out by the robotitself. In either case such instructions are at a very high-levei , lhe performance 01' which will depend on thecapability of the robot and the state 01' the environment.An animation instruction has the general structure:

[<subject> <verb> <object>]where <subject» is the agent performing the actiondescribed by the «verb» using lhe parameter listspecified by the «object>, The <verb> is lhe focus of theplanning activity which gives the name to the instruction.The «subject» and <object» are lhe parameters of theinstruction. Each instruction has a generic forrn, asshown in Fig. 3. A specific instruction can be derivedfrom lhe generic form with the addition of slots initialisedto default values. The natural strategy for dealing withvarying leveis of abstractions is to use hierarchies,schernatically shown in Fig. 2, where actions aredescribed by a number of sirnpler actions, therebyassociating lhe major instruction with a number of sub-instructions. In the following sections some problem-solving strategies used in the planning of instructions aredescribed.

Fig. 3. The instruction frame.

3.1 Planning in the InstructionFor lhe purpose 01' behavioural animation, planning anaction can be understood as an explanation of how anactivity is to be achieved according to the ·expectations ofthe system developer. The planning of this activityretlects lhe knowledge and experience of lhe animator.The instruction levei is solely concerned with lhe plangeneration, which is accomplished by symbolmanipulation. A fully developed plan cornprises anumber of simple actions that are processed by the Tasklevei using the associated Knowledge Source.

This kind 01' planning for behavioural animationmatches script-based planning in which plans are formedby a priori knowledge rather than through lhe use ofalgorithms or heuristics. For example, the high-levelaction for picking up a glass from a counter can bedescribed by either of lhe sequences shown in Fig. 4.

Instructionnodes

Task nodes

ObjectsMotions Library

• message. The means by which animated figurescommunicate by exchanging messages.Each of lhe above activities represents one

levei of abstraction and is handled by separate setsof knowledge called knowledge sources (KSs) ,namely, the Scheduling KS, lhe Instruction KS, lheTask KS, and Message KS, This results in anefficient overall control structure incorporatingknowledge representation. Fig. I shows lhe generalorganisation of the controller. Arrows within lheblackboard indicate transitions of control from oneproblem solving stage to another.

Fig. 1. Overview of the animation framework.

There can be many animated agents in lheenvironment, with at least as many sirnultaneousprocesses being pursued in lhe blackboard datastructure, each being in a different state of analysisand with its solution tree spanning across leveis inlhe blackboard. In addition, lhe animated entitiespossess internai information, such as posture andlocation, which are also taken into account. Each oflhe control leveis of Fig. I are shown in Fig. 2 asconnected nodes across three stages and aredescribed in more detail in lhe following sections.

Ooal schedulin

3 The Instruction ConceptIn lhe present work, lhe term instruction is lhecomputational entity that implements lhe AIconcept of action [Georgeff, 1990]. It is a controlstructure which will eventually be transformed intosequences of motions. An instruction can beinterpreted as a command given by the animator toa robotic agent to be performed in lhe animation

478

1) look at the glass2) walk to the counter3) approach the Iett-front side of the counter,4) extend the right arm5) pick up the glass from the counter

detailed plan

1) approach the counter,2) pick up the glass from tl1e counter

higher-Ievel plan

Fig.4. Two abstractions for the same instruction.

The specific planning of an action may take manyforrns, but a pragmatic is that plans should notcornprise more than four sub-actions. Plans withmany sub-actions may include segments which arecornmon to others and an intermediate leveiinstruction that implements part of a plan may beused in this case.

3.2 Plan Representation and Plan SelectionThe general forrnat of a plan is:

plan Lst; <plan juime>, [<iltstrl>, ..., <iltstrN>] )< instrl» => [<iltstructiolt>, «paraml » ...

«paramlz-i or <instruction>

where lhe clause plan Ls is associates theplan ruime, with a sequence of instructions, ortasks, and their relevant parameters.

Each instruction has a number of mies forselecting the corresponding plans. The mies areorganised in groups of rulesets, Unless anotherselection criterion is in place, these mies areconsidered in the order in which they are listedbecause a recursive strategy is employed, asdiscussed in a later section. For the simplicity, thename of the ruleset is the same as that of theinstruction :group «instru ction ruime»«plan ruime I», ...,

«plan.nametv» .The mechanism for the selection of an appropriateplan is decided by mies . Each mie identifies oneclass of situation that is accomplished by a plan andcauses some of the internai parameters to be set up.One coIlection of parameters in the plan comesfrom the internal parameters of the instruction andthe other part from the templated parameters. Theinference mie for plan selection is typicaIly of theform:rule «plançname>ir plan( «instruction ruime», Instance, Agent )

and «conditions»thenchoose_plan( «plan ruime», Instance ) .

which means that, if the «condiiions» are satisfied, thenthe <plan_name> is chosen to implement the instruction.The «conditions» serves to identify the current contextand leads to the appropriate choice of a plan and tl1econsequential passing of the appropriate parameters tothe plan. The presence of the variable Instance ismandatory in the plan cIause, and may occur in the«conditions» because it links and stores informationrelated to the plan selection and context identificationstages. The variable Agent also provides importantinformation for context identification.

The context is the environment in which an agentrelates to objects that may be involved in an action. Thecurrent situation of the agent is aIso part of the contextand is aIso considered in ilie planning processo Forexample, suppose that a waiter intends to pick a glass,some elements of the context considered for the selectionof a plan are the posture of tl1é waiter and the distancebetween thewaiter and the glass. If thewaiter is standingup and the glass is out of the reach of his arm, he wilIhave to approach the glass first. However, if thewaiter isstanding up with ilie glass within his reach, he can simplypick up the glass. Although the difference between thesecases is only the distance between the waiter and theglass, they represent two distinct contexts requiringdifferent anaIyses.

3.3 Instruction ParameterisationEach instruction has a parameter list serving as atemplate by which communication can be establishedwith its immediate parent. The parameterisation of aninstruction permits data to be passed between parent andchild nodes. The instantiation of parameters connects theobjects from one instruction to another in a major planoThe parameter data is bound to the instruction instancebefore context identification takes place.

3.4 Recursion in Instructíon PlanningComplex goals can be satisfied using a recursive processoThe solution of a problem is a self reference to a simplercase until it is replaced by a set of non-recursiveinstructions, which can subsequently be reaIised as tasks.

A similar approach is employed in planning anaction when the goaI can be achieved in differentcontexts, each one with a varying levei of complexity. Ifthe goaI is to be achieved in a complex context, iliesolution in the recursive approach is to realise apreliminary action such that a new simpler context isobtained for solving the original goaI. This can also beunderstood as a succession of changes of contexts, eachane requiring a simpler plan for achieving the goaI. Aninstruction can be employed in a number of contexts withsolutions that could be cIassified as trivial, simple, orelaborate. In the particular case of a recursiveinstruction, at least one of its plans is of elaborate typeand at least one is a trivial or simple type. The trivialcase occurs when the goal of an instruction is aIreadysatisfied and no further action is needed. In the simple

479

case the salutian camprises a sequence of actionswhere self reference does nat accur in the plan. Inthe elaborate case, the cantext for the instructianmight be toa camplex so it is canverted to simplercases by recursian .

4 The Message ConceptAn agent can request an actian fram another agentthraugh the use of the message exchange scheme.The message is an entity that behaves as a carrierthat embeds an instructian. Such an instructian,which is called a requested instruction , specifies thedesired actian which the target agent is expected toperfarm. The target agent can then be scheduled toevaluate the requested instruction and, if possible,fulfil it.

5 The Task ConceptThe task is a canceptual entity for cantralling well-defined matianal activities such as: walking,pasitianing of a bady, waving a hand, picking up arputting dawn of abjects, etc . Tt is analogous to thecancept of the operator in planning actians in AI.Task entities contraI the actual motions perfarmedby agents which can be achieved if the requiredresaurces are available. Since tasks may be incampetitian for resources, resaurces are explicitlya1lacated and tasks are executed selectivelyaccarding to executian priarity. An active can besuspended in a favaur of a higher priarity task.

5.1 Allocation of ResourcesA task can be perforrned by an agent pravided thenecessary resources (internai ar externai to theagent) can be a1lacated to it. The internai resaurcesare the agent's arms, legs, head, and torso; while theexternaI resaurces can be glasses, tables, ar placesat a caunter. The resources a1lacated by tasks arerecarded in lhe resources_used slot in the agentinstance. Carrespandingly, each af an agent's limbsis characterised by a pair of items: the state arpositión of the limb and the name of the instance ofthe task that has been allacated it. For example,the statementaWaiter's cleg becames st( straight, walk_tl )

means thattheposition of the waiter's right leg hasbecame straight and is currently being used by thetask instance named walk_tl.

5.2 Priority to Access ResourcesIn a typical dynamic environment there are severaltasks that can patentially come into operatian ifcanditians permit. Thus, if for a particular agentthere is no conflict in the use of resources thesetasks could be performed. However, if there is a. conflict lhe access is resolved by priorities. It ís

obviously desirable to give preference to the mostinteresting motians .

5.3 Database UpdatingFollowing of each step of a movement, lhe animationdatabase is updated with new facts about the objects.These updates pravide conditions to guide subsequentactions ar serve as the canstraints to be tested by othertasks. Examples are: in a plan selectian, "if the agent'sposition is seated then select a plan that includes thestanding up action", "if the counter's front Left isengaged then check other sides", and "if the counter'sobject List is not empty then collect an object"; in thewalking task, "if the left Leg is forward, then the nextstep will be to swing the righcleg forward".

6 The Scheduling StageThe Scheduling stage is the first and the highestcomponent in lhe contrai hierarchy. The main goal of thescheduling procedure is to keep the agents occupied withactivities . It is also responsible for deciding when toinitiate new instructions and for giving preference to themost appropriate ones . The scheduling of motions isactually achieved by the combinatian of two leveis ofcontraI: lhe Scheduling levei and the Task leveI. At theScheduling levei, gaal instructions are scheduled to rundepending on lhe current load placed on agents, to avoidunnecessary overlaading. At lhe task leve I, where go alinstructions are developed inta partially expanded plans,decisions are made about the execution ar suspensíon ofan individual task of a processo The suspension artermination of one process will consequently bring othersinto consideration, Normally such decisions are based onthe availability of resaurces and the priarities of thecontending processes. This issue gives rise to the mainaspects considered in the following sections namely: thecategory af the instructions, the sources of theinstructions, the triggering of subsidiary instructions, andíssues regarding priority for performing actions .Consideration is given to identifying feasiblecambinatians of motions to achieve an interesting andrealistic animation.

6.1 Sources of InstructionsInstructions leading to a gaal executed during ananimation originate fram different saurces. The twomain sources are the anírnator's script and the animatiansystem. Because the saurce of an instruction is animportant criterion in scheduling the animated activities,the animation system as a source of instructions allows afurther classification into at least four types of sources asshown in 8°.

480

Animation Environment

SpecialMessageTaskDefault

Figure 9-1. Source of activities.

6.2 The Animation ScriptActivities to be realised in an animation areconstitut ed as a collection of instructions which canbe organised by the animator as a script. Suchinstructions are prescribed to happen at establishedtimes or under defined conditions. Theseinstructions define a main line of actions serve as askeleton of an animation plan . They are expandedduring the animation. It is not unusuaI for ananimation script be composed of up to five mainactions, having many more gaps than prescribedactions. Additionally, through scripts the animatorcan introduce auxiliary comrnands to aIter thecont1guration of the environment or the attributes oflhe animated objects, and these are facts that maybe considered. in Lhe planning process o Theanimator controls the animation through the script ,which as the source of instructions, has precedenceover instructions originating from the system. Thegeneral formar of a script instruction is given by:agenda( «instr. no.>, «Condition>, <.Activity> )

«Condition» gives information about the timeat which the instructi on is due tooccur ;

<.Activity> is the action to be performed by theagent or a command to create a new object.

6.3 Actions Triggered by the SystemThe automatic generation of actions by theanimation system extends those actions planned inthe script. Three kinds of sources of actions aredescribed in this section , however, variants ofsources might be identitied by the animator whichmay provide the animation with more versatilealternatives. New actions can be initiated in anypoint of the process (task, instructi on, or message)by asserting special cIauses onto the blackboard.These clauses are recognised and processed by theScheduler which gener ates instructions that .can beperformed by the agents as opportunities arise.

Actions Requested Througb Messages. TheMessage ís a means by which agents cancommunic ate. In fact, the communication is limited

to requests for actions and acknowledgements. Thus themessage is an entity which provides Lhe carrier for aninstruction . This instruction is then scheduled to beundertaken by an executor agent, which is the receiver ofLhe message. There are two complementary types ofcommunictions between two agents: Lhe request messageandLhe acknowledgement message. The request messagetypicaIly sends a request for aprimary motion to beperformed by Lhe receiver while Lhe acknowledgementmessage sends a request for secondary motion to Lhe firstagent. Because such actions are instigated intentionaIIyby agents and because more than one agent is involved,Lhe message as a source of instruction is regarded asbeing at a higher priority for scheduling than most othersources in Lhe animation system.

Default Action. In Lhe absence of prescribedactivities for a given agent, Lhe scheduler will initiate adefauIt instruction to cause an .agent to perform anactivity appropriate to its category. Thus a waiter mightclear tables as a default operation. Such an action isspecified in the activity slot of Lhe agent frame. TheScheduler will realise that an agent has become inactivebecause a speciaI cIause (idle_agent) has been placed onthe blackboard by Lhe end state of the last instructionperformed by Lhe agent.

7 ConclusionsA knowledge-based framework for computer animationhas been presented, focusing specificaIly on behaviouraIanimation which is concerned with animated entities thatcan develop autonomous behaviours in dynamicenvironments. The aim of this approach is to shift mostof the control to the agents because dynamicenvironrnents can exhibit an excessive number of detailedactivities, which are toa difficult to (be controlled, orscripted , by the human animator. AI techniques have thusbeen exploited to produce control mechanisms thatimplement autonomous behaviour.

8 References

R.C. Arkin. Motor Schema Navigation for a MobileRobot: An approach to Programming by Behaviour.Proc. IEEE Conf. on Robotics and Automation,Raleigh. 1987. Pp..264--271.

R.A. Brooks, J .H. Connel!. Asynchronous DistributedControl System for a Mobile Robot. Proc. SPIE1986, Mobile Robots, 727. 1986. Pp. 77--84.

T.W. CaIvert, R. Ovans , S. Mah. Towards LheAutonomous Animation of Multiple HumanFigures. Proc. Computer Animation'94. 1994. Pp.69--75.

M.P. Georgeff. Planning. Readings in Planning (1.AlIen, J. Hendler, A. Tate (Eds.)) . MorganKaufmann Publishers, Inc., San Mateo, CA. 1990.Pp.5--25.

481

R. Engelmore, T. Morgan. Blackboard. Addison-Wesley Publishing Company, Wokingham,1988.

B. Hayes-Roth. Opportunistic Control of Action inIntelligent Agents. IEEE Transactions onSystems, Man, and Cybernetics. V.213, NO.6.1993. Pp. 1575--1587.

C.S.N. Ho. A Framework for Behavioural Controlin Computer Animation. University of Sussex,England, 1996.

G.R. Meijer et aI. Robot Control with ProceduralExpert System. In: Expert Systems andRobotics (T. Jordanides and B Torby (Eds.)) .NATO ASI Series. 1991. Pp. 217--232.

C. Morawetz, T.W . Calvert. Goal-Directed HumanAnimation of Multiple Movements. Proc.Graphics Interface'90 . 1990. Pp. 60--67 .

Ridsdale, G., S. Hewitt, T.W. Calvert. TheInteractive Specification of Human Animation .Proc . Graphics Interface'86. 1986. Pp. 121--130.

H. Sun, M. Green. The Use of Relations for MotionControl in an Environment With MultipleMoving Objects. Proc. Graphics Interface'93.1993. Pp. 209--218.

D.E. Wilkins. Practical planning: extending theclassical AI planning paradigm. Morgan

- Kaufmann Publishers, Inc. 1988.

482