rai notes on reactive paradigm

Reactive Paradigm – Overview

Subsumption Architecture

DARSHAN INGLE

1Prepared by Prof. Darshan Ingle.

Reactive Paradigm Overview

• A robotic paradigm can be described by the relationship between the three primitives of robotics: Sense, Plan, and Act.

• It can also be described by how sensory data is processed and distributed through the system, and where decisions are made.

• Two Representative Reactive Architectures:

1) Subsumption

2) Potential Fields Summation

• Reactive Paradigm emerged late 1980‟s

• Reactive Paradigm still important for two reasons:

1) Robotic systems in limited task domain are still being constructed

2) Forms the basis for the Hybrid Reactive-Deliberative Paradigm


http://en.wikipedia.org/wiki/Robotics

http://en.wikipedia.org/wiki/Sense

http://en.wikipedia.org/wiki/Plan

Reactive Paradigm• Sense-act type of organization.

• The robot has multiple instances of Sense-Act couplings.

• The SENSE-ACT organisation does not specify how the behaviors

are coordinated and controlled.

• So each behavior has its own dedicated sensing which results in

more than one behavior taking the same output from the sensor and

process it differently(via behaviors perceptual schema).

• Thus, one behavior does not know what another behavior is doing

or perceiving.


http://en.wikipedia.org/wiki/Reactive_planning

Hierarchical Paradigm

• The robot operates in a top-down fashion, heavy on

planning.

• The robot senses the world, plans the next action, acts;

at each step the robot explicitly plans the next move.

• All the sensing data tends to be gathered into one global

world model.


Hybrid Deliberate/Reactive Paradigm

• The robot first plans (deliberates) how to best decompose a task into

subtasks (also called “mission planning”) and then what are the

suitable behaviours to accomplish each subtask.

• Then the behaviours starts executing as per the Reactive Paradigm.

Reactive Paradigm grew out of the dissatisfaction of the hierarchical

paradigm and with the influx of ideas from biological intelligence.


Reference(Biological Intelligence)

• To gather information requires senses, or sensors and the ability to

attach labels to the sensed perceptions and to manipulate these

labels to make predictions (algorithms, thinking).

• Biological intelligence evolves out of multi-cellular bodies with

senses, nervous systems or brains (wet-ware).

• Non-biological forms of intelligence require sensors, electrical

networks or computer brains (hardware).

• Although various reactive systems may or may not strictly adhere to

the principles of biological intelligence, they generally mimic some

aspect of biology.


Horizontal Decomposition of Hierarchical Model

SENSE PLAN ACT

SensorsExtract

Features

Combine

Features

into

Model

Plan

Tasks

Task

Execution

Motor

ControlActuators

This dissatisfaction was best characterised by those systems having

Horizontal Decomposition.


Vertical Decomposition

• Instead, examination of ethological literature suggests that

intelligence is layered in a vertical decomposition:

1. Agent starts with primitive survival behaviours.

2. Evolve new layers of behaviour which either:

a) Reuse the lower, older behaviour

b) Inhibit older behaviour

c) Create parallel tracts of more advanced behaviours

3. Parallel tracks can be thought of as vertically stacked layers

4. Each layer has access to sensors and actuators independently of

any other layer.

5. If anything happens to the more advanced layer, lower levels

should still operate. i.e. human brain (breathing) continue

independently of higher order functions (counting, face

recognition).8Prepared by Prof. Darshan Ingle.

Vertical Decomposition

SENSE ACT

SENSE ACT

SENSE ACT

SENSE ACT

Sensors Actuators

Avoid Collisions

wander

explore

Build maps

Vertical decomposition of tasks into a S-A Organisation, associated with

the Reactive Paradigm


• The major theme of this chapter is that all

reactive systems are composed of

behaviors, though the meaning of the

behavior may be slightly different in each

reactive architecture.

• Behaviors can execute concurrently and/or

sequentially.


Attributes of Reactive Paradigm

• The fundamental attribute of reactive paradigm is all actions are

accomplished through behaviours

• As in ethological systems, behaviours are a direct mapping of

sensory inputs to a pattern of motor actions that are then used to

achieve a task.

• From the mathematical perspective, behaviors are simply transfer

functions i.e. they transfer SENSORY INPUTS into ACTUATOR

COMMANDS.

• Here, we consider BEHAVOIR as a schema which will consist of

atleast one motor schema and one perceptual schema.

• Motor schema contains the algoritm for generating the pattern of

action in a physical actuator and Perceptual schema contains the

algorithm for extracting the percept and its strength.


S-A organisation of the

Reactive Paradigm into

multiple, concurrent

behaviours

Behaviour

SENSE ACTBehaviour

SENSE ACTBehaviour

SENSE ACTBehaviour

SENSE ACT

Attributes of Reactive Paradigm ( Contd . . . )

•The reactive Paradigm literally threw away the PLAN component of

the SENSE, PLAN , ACT triad.

•The SENSE and ACT components are tightly coupled into behaviors

and all robotic activities emerge as the result of these behaviors

operating either in sequence or concurrently.


Characteristics of Reactive

Behaviours

• Reactive robotic systems execute rapidly (tight coupling of senses permits real-time operation)

• Behaviours can be implemented directly in hardware circuits or low computational complexity algorithms (O(n)).

• Have no memory (limiting behaviours to stimulus-response reflexes)

• Main point “Behaviours controlled by what is happening in the world, rather than the program storing & remembering what the robot last did”

• Five Characteristics of reactive Paradigm are:

1) Situated Agent (integrated part of world)Robots are situated agents operating in an ecological niche.

i.e. when a robot acts, it changes the world, and receives immediate feedback about the world through sensing.


2) Emergent Behaviours

Behaviours serve as the basic building blocks for robotic action, and the overall behaviour of the robot is emergent.

Behaviours are independent computational entities and operate concurrently, Hence there is no explicit “controller” module which determines what will be done, or which function call other functions.

3) Ego-centric / Robot centric

Only local, behaviour specific sensing is permitted.

i.e.consider OBSTACLE AVOIDANCE example.It does not matter that an obstacle is in the world at coordinates (x,y,z), only what matters is where it is relative to the robot.

Characteristics of Reactive Behaviours (continued)


Characteristics of Reactive Behaviours

(continued)

4) Modular Behaviours

These systems inherently follow good software design principles.

The modularity of behaviours supports the decomposition of a task into component behaviours.

The behaviors are tested independently, and the behaviors may be assembled from primitive behaviors.

5) Biological Motivation

Animal models of behaviour are often cited as a basis for these systems or a particular behaviour (unlike earlier AI days where a conscious effort to not mimic biological intelligence was made)

Thus, it is very acceptable under Reactive Paradigm to use animals as motivation for a collection of behaviors.


Advantages of Programming by Behaviour

• Constructing a robotic system under Reactive Paradigm often referred to as programming by behaviour.

• Programming by Behaviour has number of advantages most of them consistent with good software engineering principles:

1. Behaviours are Modular.

2. Easy to test in isolation (Unit Testing).

3. Also supports incremental expansion of capabilities of a robot i.e. a robot becomes more intelligent by having more behaviors.

• Modules have Low Coupling

can function independently of each other with minimal connections or interfaces promoting easy reuse.

• Modules have High Cohesion

Data and operations contained by a module relate only to the purpose of that module.


Reactive Paradigm Representative

Architectures

• The overall action of the robot emerges from the multiple concurrent behaviours and the architecture must provide mechanisms for:

1) Triggering behaviours

2) Conflict resolution when multiple behaviours are active at any one time.

• The two most well known Reactive Architectures are

1) Subsumption – Decomposition into layers of task achieving behaviours.

2) Potential Fields – Behaviours combined by summation of fields


Subsumption Architecture

• Rodney Brooks Subsumption Architecture is the most influential of the purely Reactive Paradigm systems.

• These robots look like shoe-box sized insects (6 legs and antennae)

• Implementations quite often have the behaviours embedded directly in the hardware or small micro processors allowing the robots to have all on-board computing.

• Robots could now walk, avoid collisions and climb over obstacles without the move-think-move-think pauses of Shakey.

• A behaviour is a network of sensing and acting modules which accomplish a task.

• The modules are Augmented Finite State Machines (AFSM), or finite state machines which have registers, timers & other enhancements to permit them to be interfaced with other modules.


• Behaviours are released in a stimulus-response way, without an external program explicitly coordinating and controlling them.

• Four interesting aspects of subsumption in terms of releasing and control are:

1. Layers of Competence:

Modules are grouped as layers of competence.

The layers reflect a hierarchy of intelligence or competence.

lower layers encapsulate basic survival functions (collisions)

higher levels create more goal directed actions (mapping)

Each of the layers can be viewed as an abstract behaviour for a particular task.

2. Subsumption of lower layers:

- Modules in a higher layer can override or subsume the output from behaviours in the next lower layers.

- Behaviour layers operate concurrently and independently and hence need mechanism to handle potential conflicts winner always takes all (always the highest layer)

Subsumption Architecture (continued)


3. No World Model (No Internal State)

-The use of the internal state is avoided.

-Because robot is a situated agent,most of its info comes directly

from the world

- If the robot depends on internal representation,what it believes

may begin to dangerously diverge from reality.

- Some internal state needed for releasing behaviours (i.e. scared,

hungry), but good design minimizes this.

4. Taskable

- Tasks are accomplished by activating the appropriate layer, which

then activate the lower layers.

- In practice, Subsumption style systems are not easily taskable,

that is, they can‟t be ordered to do another task without being

reprogrammed.

Subsumption Architecture (continued)


Level 0 AVOID Architecture: obstacle avoidance

• Sensor: A SONAR module that gives the distance to the objects in polar

coordinates.

• Internal Modules

– COLLIDE detects if front obstacle is too close (i.e. halt signal)

– FEELFORCE it treats sensor reading acts as repulsive force field.It

can be thought of as summing the vectors from each of thesonar

readings.This results in a new vector.

– RUNAWAY provides direction to move

• Actuators

– TURN provides motor output to turn robot.It splits off the direction to

turn and passes it to steering actuators. It also passes it to the

FORWARD module which uses magnitude to determine the magnitude

of next forward motion(how far/how fast)

– FORWARD switches forward motion on or off


“robo-centric” view of range readings

0

1

2

3

4

7

6

50 1 3 4 5 6 72

Unrolled into a plot

Sonar module reads the sonar ranges.

Polar plot range readings in polar coordinates (r,) surrounding the robot.


Level 0 Architecture Diagram


Level 1 Architecture: WANDER

How does Turn know which module to take heading from?

Wander - Can‟t pass directly onto Turn since will sacrifice obstacle avoidance


Level 1 (Contd…)

• Consider a robot which actually wandered around instead of sitting

motionless, but was still able to avoid obstacles.

• This is achieved using WANDER module

-which computes the random headings(vector) every n seconds

-pass it to TURN and FORWARD module

• But, it cannot be passed to TURN module directly since TURN

accepts only one input.

• One solution is to add another module in level 1, AVOID, which

combines FEELFORCE vector with WANDER vector.

• This also offers an opportunity to create a more sophisticated

response to obstacles.

• AVOID combines the direction of force of avoidance with desired

heading which results in actual heading being mostly in right

direction.This is now fed to TURN module.

• Note:AVOID module was able to EAVESDROP on components of

next lower layer.25Prepared by Prof. Darshan Ingle.

Level 1 (Contd…)• The issue now appers to be when to accept the heading vector from

which layer.

• Subsumption makes it simple: the o/p from the higher layer

subsumes the o/p from the lower layer

• This is done in one of the two ways:

1. Inhibition

O/P of subsuming module is connected to O/P of another

module.

If O/P of subsuming module is ON/has any value-> O/P of

subsumed moduleis blocked/turned OFF.

2. Suppression

O/P of subsuming module is connected to I/P of another

module.

If O/P of subsuming module is ON,it replaces normal I/P to

subsumed module.


27

• In this case, AVOID module supresses o/p from RUNAWAY.

• RUNAWAY is still executing but its o/p doesn‟t go anywhere.

• Instead, o/p from AVOID goes to TURN.

• The use of layers and subsumption allows new layers to be built on

top of less competent layers, without modifying the lower layers.

• This facilitates MODULARITY and simplifies TESTING.

• It also adds some ROBUSTNESS i.e. if something disables level1

behjavoir, level0 behavior is still intact. Thus the robot would atleast

be able to preserve its self-defense of fleeing from approaching

obstacles.

Level 1 (Contd…)

Prepared by Prof. Darshan Ingle.

Level 2 Architecture: follow corridorIntegrate - estimates how far robot has travelled off course

Supplies dangerous internal state


Level 2 (Contd…)

• Third layer is used to permit a behavior „move down the corridor‟.

• LOOK module-> examines the sonar polar plot & identifies the

corridor.

• LOOK module isnt included in the lower layer since identifying the

corridor is more computationally expensive than just extracting

range data.Thus it would have taken longer time to execute at lower

layer.

• STAYIN MIDDLE module-> LOOK passes the vector representing

the direction of middle to STAYIN MIDDLE module.

• Now this STAYIN MIDDLE module subsumes the WANDER module

and delivers the o/p to AVOID module.

• Thus the robot can now swerve around obstacles.

• How does the robot get back on course if the LOOK module has not

computed a new direction?The solution is INTEGRATE module


Level 2 (Contd…)

• INTEGRATE module->observes the robots actual motions from

shaft encoders in actuators.This gives an estimate of how far the

robot has travelled since the last update by LOOK.

• Thus INTEGRATE module supplies a dangerous internal state i.e. it

substitites for the feedback from the real world.

• If for some reason , the LOOK module never updates,then the robot

would operate without any sensor data forever/atleast until it is

crashed.

• So subsumption style systems include time constants on supression

& inhibition.

• If the supression from STAYINMIDDLE ran for longer than n

seconds without a new update, the supression would cease.

• The robot would then begin to wander, and hopefully whatever

problem(like the corridor being totally blocked) that had led to the

loss of signal would fix itself.


Potential Fields Methodologies(Visualizing

potential fields)• Potential Field styles of behavior always use vectors to represent

behaviors & vector summation to combine vectors from different

behaviors to produce an emergent behavior.

• Potential field-> is an array or field of vectors.

• Vector->is a mathematical construct which consists of a mafnitude &

direction.

• The array represents a region of space. It is represented in 2-

dimensions, representing a birds eye view of the world,just like a

map.Map can be divided into squares creating a (x,y) grid.Each

element of the array represents a square of the space.

• Perceivable objects in the world exert a force fieldanalgous to

magnetic or gravitational fied) on surrounding space


Visualizing potential fields( Contd…)


5 Types of potential fields


5 Types of Potential fields(Contd …)

1. Uniform:

• Here , the robot would feel the same force no matter where it is.

• From any position or orientation, the robot would align itself to the

direction the arrow points and will move in that direction at a

velocity proportional to the length of the arrow.

2. Perpendicular:

• Here, the robot is oriented perpendicular to the object/wall/border.

• The field is directed away from gray wall, but perpendicular field

can be pointed towards the object as well.

3. Attractive:

• Here, whereever the robot is, it will feel a force relative to the

object.

• It is usually used when the agent is literaly attracted to

light/food/goal.


5 Types of potential fields(Contd…)

4. Repulsive:

• Opposite of attractive field

• These are normally associated with obstacles/things that the agent

should avoid.

• Closer the robot is to the object,stronger the repulsive force,180

degree away from it.

5. Tangential:

• This field is tangent around the object.

• They can spin either clockwise/anti –clockwise.

• They are useful for directing the robot to go around the

obstacle/having a robot investigate something.


Magnitude Profiles

• Notice slide

33(fig c & fig

e),length of

the arrow

get smaller

closer to the

object.

• The way of

changing of

this

magnitude

of the

vectors is

called

magnitude

profile.


Magnitude Profiles(Contd…)

1. Constant magnitude:

• When a robot encounters a robot a obstacle, it either tries to run

away from it or stops there.

• This whole process is happening at a constant speed,which leads

to jerky motion.

2. Linear drop off:

• When a robot encounters a robot encounters a obstacle,it will turn

and move quickly towards it ,then slow up to keep from over

shooting & hitting the object.

3. Exponential drop off:

• Here, the drop off is proportional to the square of the distance.

• Thus, for every unit of the distance away from the object, the force

on the robot drops in half.


Combination of fields & behaviors

• The first attribute of true potential field methodology is that it requires all

behaviors to be implemented as potential fields.

• The second attribute is that it combines behaviors by vector summation.

• Now, how are this behaviors implemented & combined?


Combination of fields & behaviors(Contd…)

• The motor schema of MOVE2GOAL is represented with attractive

potential field. It uses shaft encoders on the robot to tell if it had

reached the goal position.

• The RUNAWAY behavior is the repulsive behavior. It uses a range

sensor to detect if something is in front of it.

• Combined field behavior->

1. Robot starts at lower right corner.

2. At time t0,the robot senses the goal. It can only perceives the goals

& cannot perceive the obstacle.So, it feels only attraction vector at

the start i.e. RUNAWAY vector has magnitude 0.0



3. It moves along a straight line for the goal.

4. At time t2, it updates it sensors and now perceives both the goal

and the obstacle..

5. Thus both the behaviors contribute a vector.these are summed

and the robot now moves offcorse.

6. At time t3, the robot has almost moved beyond the obstacle. Thus

goal exerts a stronger force now.

7. At time t4, it resumes cousrse and reaches the goal.



• Some disadvantages of it are that it suffers from the following problems:

1. Update Rate:

o The distance (length of the arrows)between updates is different. This

is due to the changes in the magnitude of the outer vector, which

controls the robot velocity.

o If it has shorter range vector->it moves slowlyi.e. covers less distance

in the same amount of time.

o If it has longer range vector->it moves faster(i.e. it overshoots between

t3 & t4 where the robot actually goes farther without turning & has to

turn back to the goal).

o As a result , the path is jagged with sharp lines.

o The resulting path would have been smoother if the robot has a faster

update interval.



2. Local minima:

o Along the line behind the obstacle, the vectors have only head

i.e. direction of the arrow & no body i.e. length of the arrow(refer

slide no. 41)

o This means that the potential fields have summed upto zero i.e.

magnitude has become zero

o So if our robot reaches at this spot, it will stop and not move

again.

o This is called as the local minima problem.

o This is because potential fields have minima or valley, that traps

the robot.



3. Holonomicity is the relationship between the controllable degrees

of freedom of the robot and the total degrees of freedom of the

robot.

o If the number of controllable degrees of freedom are greater than

or equal to the total degrees of freedom a robot is said to be

holonomic.

o By using a holonomic robot many movements are much easier to

make and returning to a past pose is much easier.

o Potential Field Planning places values over the map with the goal

having the lowest value raising the value depending on the

distance from the goal.

o Obstacles are defined to have an incredibly high value. The robot

then simply moves to the lowest potential value adjacent to it,

which should lead it to the goal. However this technique often gets

trapped in local minima.


Example using one behavior per sensor

• Robot is able to get

out of the cave-like

trap called as the

BOX CANYON.

• It does so without

building a model of

the wall.

• Each instance

contributes a

vector.,some of which

have X or Y

component that

cancel out.


Potential fields compared with Subsumption

• In level 0 in subsumption , if there are no obstacles within range,

the robot feels no repulsive forces and is motionless.

• If an obstacle comes within range & is detected by more than one

sonar,each of the sonar readings create a vector , pointing the robot

in opposite direction.

• The COLLIDE module in the subsumption does not map over ton a

behavior in potential field methodology.

• COLLIDE is to stop the robot if it touches an obstacle i.e. if the

RUNAWAY behavior has failed.

• But it does not produce any potential field, unless a uniform field of

magnitude zero is permissible.

• So, COLLISION vector would have no impact.

• Infact, collisions are often treated as panic situations which trigger

an emergency response.


48

Potential fields compared with Subsumption(Contd…)

Prepared by Prof. Darshan Ingle.


• In level1 , WANDER behavior

generates a new direction to

move every n seconds.

• RUNAWAY behavior

generates a behavior which

will cause a robot to move 180

degree away from the

obstacle.

• By combining the o/p from

WANDER with the o/p from

RUNAWAY , the need for

AVOID behavior is eliminated.

• The robot now moves in the

desired direction i.e. resultant

direction.



• In level 2, the FOLLOW-

CORRIDOR behavior causes the

robot to stay in the middle of the

corridor.

• This is accomplished using two

potential fields:

1. A uniform field perpendicular to the left

boundary & pointing to the middle.

2. A uniform field perpendicular to the

right boundary & pointing to the middle.

• However, these two uniform fields

are not sufficient for the robot to

move forward.

• So a third uniform field is added

which is parallel to the corridor.

• All three fields combined yield a

smooth field which sharply pushes

the robot back to middle of the

corridor. 51Prepared by Prof. Darshan Ingle.


• Docking behavior:

1. Docking is when a robot moves to a specific

location& orientation relative to docking

station.

2. This is useful for robots performing material

handling in industry.

3. So in order to accept a piece of material to

carry,the robot has to be close enough to the

correct side of the end of conveyer and

facing the right way(i.e. in right position &

orientation)

4. So we go for selective attraction field. It

makes the robot feel the attractive forces

when it is within certain angular range of

docking station.

5. However , it does not cover the case of when

the robot approaches from behind or to the

side. 52Prepared by Prof. Darshan Ingle.


6. So we go for

tangential field.It

makes the robot

orbit the dock until it

gets the selective

attraction area.

7. This combination of

the 2 motor

schemas produces

very smooth field

which keeps the

robot in correct

position &

orientation.


Advantages of potential field styles

1. It is a continous representation which is easy to visualize over a

large region of space.

2. Thus its also easy for the designer to visualize the robot‟s overall

behavior.

3. Its also easy to combine fields & languages like C++ support

making behaviorial libraries.

4. The potential field can be parameterised i.e. the range of

influence can be limited and any continous function can express

the change in magnitude over distance(linear,exponential,etc.)

5. 2D fields can be usually extended into 3D fields, & so behaviors

developed for 2D will work for 3D.


Disdvantage of potential field styles

• Multiple fields can sum upto a vector with zero magnitude->local

minima problem.This causes the robot to become stationary.

• Solutions to local minima problem:

1. One solution is to always have a motor schema producing vectors

with a small magnitude from random noise.

This noise in the motor schema would serve to bump the robot off

the local minima.

2. A more recent solution is to express the fields as harmonic

functions. These guarantee to not to have a local minima of 0.

The only disadvantage of this technique is that it is

computationally expensive & has to be implemented on a VLSI

chip in order to run in real-time for large areas.


Disdvantage of potential field styles( Contd…)

3. Another solution is using Navigation Templates(NaTs)

--The local minima problem occurs due to the avoid behavior.

--However, in NaTs, avoid behavior receives as input the vector

summed from the other behaviors.

--This vector represents the direction the robot would go if there

were no obstacles nearby. We refer this vector as the strategic

vector.

--If the robot has strategic vector, this vector gives a clue as to

whether an obstacle should be passed on the right or the left.

--eg:A robot crossing the bridge,it will be passing to the left of the

obstacles on the right in order to stay in middle.

--The o/p of this avoid behavior is called as the tactical vector as it

takes the takes the robot in the right direction so as to achieve the

strategic goal.


THANK YOU.


Designing a Reactive

Implementation

DARSHAN INGLE


Overview

• By this time, the sheer simplicity & elegance of reactive robots incite

people to start designing and implementing their own robots.

• Several kits(Lego Mindstorms, Rug Warrior) permit the rapid

coupling of sensors & actuators, enabling users to build reactive

behaviors.

• However new issues are how to program more intelligence into

software and how to exploit better sensors than those which come

with kits.

• Unfortunately, good intentions in robot programming are often

frustrated by two deficits:

1. Designing behaviors tend to be an art, not a science.Novice roboticists

are often uncertain as to how to even start the design process.

2. Once the designer has few well-designed and tested behaviors, how

are they integrated into a system?


Overview (Contd…)

• Emergent behavior-> When a small set of behaviors are combined

internally to produce an overall behavior.

• Series of behavior-> Here, behaviors are operating in a recognizable

sequence.

• This chapter attempts to aid the novice designer in constructing a

reactive robot system by addressing each of these two deficits:

1. An object oriented approach to designing behaviors is

introduced.

2. Two techniques for managing behaviors of robot are

introduced:

Finite state automata

Scripts.


Behaviors as Objects in OOP

• Object consists of data and methods, also called as attributes &

operations.

• Schemas( refer slide no. 63):

1. Schemas contain specific knowledge & local data structure & other

schemas.

2. A schema as a programming object will be a class.

3. The class will have an optional method called as the coordinated

control program.

4. Coordinated control program is a function that coordinates any

methods or schemas in derived class.

• Behaviors ( refer slide no. 64):

1. Behaviors are composed of atleast one perceptual schema and one

motor schema.

2. These schemas act as method for the behavior class.


Behaviors as Objects in OOP->Classes-Schema

( Contd … )


Behaviors as Objects in OOP->Classes-Behavior

( Contd … )


Behaviors as Objects in OOP ( Contd…)

3. Perceptual schema has atleast one method, which takes sensory

i/p & transforms it into data structure called a percept.

4. Motor schema has atleast one method, which transforms the

percept into a vector or other form of representing an action.

• The OOP organization allows a behavior to be composed of

multiple perceptual schema & motor schema and even behaviors.

• Primitive behavior:

It is composed of only one perceptual schema and one motor

schema.

They are monolithic i.e. they do only one thing.

They are simple mapping from stimulus to response.

Hence , they are often programmed using single methods or

objects.


Behaviors as Objects in OOP ( Contd …)

• Abstract behaviors:

These are the behaviors which are assembled from other behaviors

or have multiple perceptual schema or motor schema.

The term of “abstract behavior” should not be confused with an

abstract class in OOP.


Example: Primitive move-to-goal behavior

• Pick up the Trash event(refer slide no. 68):

The robot was placed in an empty arena about the size of an office.

The arena would have Coca-Cola cans & white stryofoam cups

paced at random locations.

In two of the four corners, there would be a blue recycling , in the

other two, a different colored trash bin.

The robot who picked up the most trash and placed them in correct

bins, in allotted time was the winner.

In most years, strategy was to find & recycle the Coco-Cola cans

first, because it was easier for the robot‟s vision processing

algorithm to perceive red and blue.

One of the most basic behaviors needed for picking up the red soda

can and moving to the blue bin is move_to_goal.


Example: Primitive move-to-goal behavior( Contd …)



When the robot sees the red can, it must move to it.

When it has a can , it must find & then move to blue bin.

It is better to write a general move_to_goal behavior, where only

what is goal- a red or a blue region-varies.

The goal for the current instance can be passed in as instantiation

through the object constructor.

So we write a single generic behavior move_to_goal(color) instead

of writing move_to_red and a move_to_blue behavior.

This behavior of move_to_goal would consist of a perceptual

schema, which is called as the extract-goal & the motor schema,

which uses an attractive field called pfields.attraction.extract-goal

which uses the affordance of the color to extract where the goal is in

the image. It then computes the angle to the center of the colored

region and the size of the region.

The affordance of the coke can is the color , while the information

extracted from the perception is the angle & the size.



The above table implies some very important points about

programming with behaviors:

1. Behavior is the glue between the perceptual & motor schema i.e. both

the schemas are independent & they do not communicate.

2. Behaviors can( and should ) use libraries of schema.

3. Behaviors can be reused if written properly. In this example,

move_to_goal behavior was written to accept a structure( or object)

defining a color & then moving to a region of that color i.e. the

behavior can be used with both red Coke cans & blue trash cans.


Example: An abstract follow corridor behavior• The move_to_goal example used the potential field methodology

using a single motor schema with a single perceptual schema.

• The follow_corridor potential field consists of 2 primitive fields:

Two instances of perpendicular to the walls

One uniform parallel to the walls.

• The follow_corridor field could be implemented in schemas in

atleast two different ways:

1. Each primitive field would be a separate motor schema(refer slide

73 fig 5.2 a)

The follow corridor motor schema consists of 3 primitives & the

coordinated control program..

The coordinated control program would be the function that knows

that one field is going towards the center of the corridor, which way

is forward, etc.

These are summed up by the coordinated control program in the

behavioral schema to produce a single o/p vector.


Example: An abstract follow corridor behavior

The perceptual schema for the follow_corridor would examine the

sonar polar plot and extract the relative location of the corridor walls

& then return the distance to the left wall and the right wall.

2. Another way to have achieved the same overall bahavior is to have

follow_wall composed of 2 instances of follow_wall behavior:

follow_wall(left)

follow_wall(right)

Each instance of follow_wall would receive the sonar polar plot and

extract the relevant wall as shown in the associated class

diagram on slide no. 74 fig 5.2 b

• In both the behaviors , motor schema ran continously and the

vectors were summed internally in order to produce a single o/p

vector.

• Since there were multiple motor schemas, the coordinated control

program for follow_corridor behavior is not null as it was for

move_to_goal.

• Thus the vector summation & the concurrency form the conceptual

coordinated control program in this case.72Prepared by Prof. Darshan Ingle.

Where do releasers go in OOP?

• The previous examples showed how behaviors can be implemented

using OOP constructs, such as classes.

• Another important of the behavior is how it is activated.

• Perception serves two purposes:

To release a behavior and

To guide it.

• But what object or construct contains the releaser and how is it

“attached” to the behavior?

• The answer to the first part of the question is that the releaser is

itself a perceptual schema.It can execute independently of whatever

else is happening with the robot, it is perceptual schema not bound

to the motor schema.

• Eg: The robot is looking for the red coke cans with extract_color

perceptual schema.One way to implement this is when the schema

sees red,It can signal the main program that there is red. The main

program can determine now that the releaser for the move_to_goal

behavior has been satisfied.In this approach, the main program is

responsible for calling the right objects at the right time.The releaser

is attached to the behavior by the designer. 75Prepared by Prof. Darshan Ingle.

Where do releasers go in OOP? ( Contd … )

• So we have two move_to_goal objects , one instantiated with goal of

“red” and the other with goal of “blue”.

• Thus, the move_to_goal behavior can use any perceptual schema

that can produce a goal angle and a goal strength.

• If we want to program the robot to move to bright light

(photomorphism), only the perceptual schema would need to be

changed..

• This is an example of Software Reusability.


Steps in designing a Reactive Behavioral System


Steps in designing a Reactive Behavioral System( Contd...)

• This methodology assumes that a designer is given a task for a

robot to do, & a robot platform.

• The goal is to design a robot as a situated agent.

• Therefore, the first three steps serve to remind the designer to

specify the ecological niche of the robot.

• Fourth step begins a iterative process of identifying and refining a

set of behaviors for the class.

• It asks questions:

what does the robot do?

How it acts & reacts to the range of variability in the ecological niche.

• Eg: In Pick Up Trash event, most of the teams used partitioning

along the lines of : Random search until see red, move to red, pick

up can,random search until see blue,move to blue, drop can.

• Steps 5-7 are less abstract.

• Here we specify the motor & the perceptual schemas.


Steps in designing a Reactive Behavioral System( Contd...)

• This is where the designer has to write a algorithm for finding red

blobs in a camera image for the random search until find red and

move to red behavoirs.

• The designer usually does this as follows:

Programs each schema independently

Integrates them into a behavior and

Tests the behaviors in isolation before integrating all behaviors.

• This style of testing is consistent with good s/w engineering

principles, and also emphasizes the practical advantages of the

Reactive Paradigm.


Assemblages Of Behaviors

• There are three common wys of representing how a sequence of

behaviors: finite state automata, scripts & skills.

• Finite state automata and scripts are logically equivalent, but they

differ slightly in their implementation way.

• Skills collect behavior-like primitives called Reaction-Action

Packages(RAPs) into a “sketchy plan” which can be filled in as the

robot executes.


Finite State Automata

• FSA are used for specifying what a given program should be doing

at a given time or circumstance.

• It is written as a table or drawn as a state diagram, thereby giving

the designer a visual representation.

• States: The designer has to be able to specify a finite number of

discrete states, that the robot should be in ( represented by K ).

And each state qЄK is a listing of behaviors that should be active at

the same time.

States are represented in the figure under the heading q, and by

circles in the graph.

• Start State: It is the state at which the robot starts.( represented as s

or q0 and drawn with double circle).

• Inputs/ Alphabet: These are the behavioral releasers and appear

under the column heading σ.FSM table considers what happens to

each state q for all possible inputs.


Finite State Automata ( Contd …)

• Transition Function: called δ specifies what state the robot is inـ

after it encounters an input stimulus ,σ.

A set of stimulus or affordances σ that can be recognized by the

robot is represented by Σ( a capital σ).

These stimuli are represented by arrows.

Each arrow represents a releaser for a behavior. The new behavior

triggered by the releaser depends on the state the robot is in.

• Final State: It represents the state when the robot has completed its

task.

Each state that the robot can reach that terminates the task is a

member of the set of final state, F.



• Thus, the FSA table is an extension of the behavioral table. This

resulting table is known as finite state machine abbreviated as M.

• The notation is:

M={K, Σ, δ, s,F}

• The table below summarizes the relationships of FSA to behaviors:



• Unmanned Ground Robotics Competition:

The objective of this competition was to have a small unmanned

vehicle ( no larger than a golf cart) to autonomously navigate around

an outdoor course of white lines painted on grass.

• In case of this UGR , there were only two states:

Following the line &

Moving forward

• Follow-line behavior was the start state.

• There are just two releasers for the UGR example, so the table does

not have many rows.

• Also , there is no final state ,since the robot is never done i.e. it

needs to be turned off manually or it runs out of power.

• Thus, both the states are final states.

• If our robot could recognize the finish line, then it could have a finish

state. It could be just stopped, or it could be another behavior, such

as a victory wag of a camera.

• However, this adds more rows to the FSA.84Prepared by Prof. Darshan Ingle.

Abstract Behaviors

• FSA are useful tool for expressing the coordinated control program

of an abstract behavior.They fall short as a programming tool for

abstract behaviors in a number of ways.

• First in many cases,the assemblages of behavior represents a

prototypical sequence of events that should be slightly adapted to

different situations i.e. a TEMPLATE or a ABSTRACT BEHAVIOR.

• Eg: In the Pick Up The Trash event, recycling the coke cans was not

the only the part of task,the robots were also supposed to pick up

white Styrofoam cups and deposit them into yellow trash

cans.These behaviors represented by the FSA could be collected

into an abstract behavior:

pick-up-the-trash( trash-color, trash can-color, size-trash can)

• Second, templates need to handle different initialization conditions.

• Eg: In the start state itself, the robot is facing a bale/obstacle.


Abstract Behaviors ( Contd …)

• Another common initialization behavior is imprinting, where the

robot is presented with an object and then records the perceived

color(or other color) of the object for use with the nominal behavior.

• Eg: In the Pick Up Trash competition, several teams literally showed

the robot the Coke can and let it determine the best values of “red”

for current lighting conditions.

• Third, sometimes robot fail in their task,these events are called a

exceptions.

• Eg: An exception might be when a robot does not pick up a soda

can in over 10 minutes.In this case, another behavior can be

substituted for the exception behavior i.e. we stop the random

wander & start a Raster scan instead.


Scripts

• Abstract behaviors often use scripts, or a related construct called

skills, to create generic templates of assemblages of behavior.

• Scripts encourage the designer to think of the robot and the task

literally in terms of screen play.

• Scripts were originally used in Natural Language Processing (NLPs)

to help the audience(a computer) understand actors (people talking

to the computers or writing summaries of what they did).

• In case of robots, actors are the robots reading the scripts.

• The script has more room for improvisation though, if the robot

encounters an unexpected condition, the robot begins with the sub-

script . This main sequence of events is called as the casual chain.

• In programming robots, we often abbreviate the routine portions of

control and concentrate on representing and debugging the

important sequence of events.

• FSA force the designer to consider and enumerate every possible

transition, while Scripts simplify the specification.


THANK YOU.


rai notes on reactive paradigm

Documents