biological foundations of the reactive paradigm

Introduction to AI Robotics (MIT Press) Chapter 3: Biological Foundations 1

3 Biological Foundations of the Reactive Paradigm

ReviewWhy?-comp. theoryIRMPerception-Summary

• Describe the three levels in a Computational Theory.

• Explain in one or two sentences each of the following terms: reflexes, taxes, fixed-action patterns, schema, affordance.

• Be able to write pseudo-code of an animal’s behaviors in terms of innate releasing mechanisms, identifying the releasers for the behavior.

• Given a description of an animal's sensing abilities, its task, and environment, identify an affordance for each behavior.

• Given a description of an animal's sensing abilities, its task, and environment, define a set of behaviors using schema theory to accomplish the task.

3 Robots In the Hierarchical Paradigm

3 Timeline of Influences

1980 19901970

Braitenberg’sVehicles

Arbib’sSchemas

Tinbergen &Lorenz & von Frisch

Marr’s ComputationalTheory

J.J. Gibson

Neisser

Middlestat

Arkin’s Schemas

Brook’s Insects

Payton

3 Marr’s Computational Theory

Level 1:What is the phenomena

we’re trying to represent?

for (i=nCol..Level 2:How it be represented as

a process with inputs/outputs?

Level 3:How is it implemented?

3 Level 1: Existence Proof

Level 1:What is the phenomena

we’re trying to represent?

Goal: how to make line drawings of objects?

people can do this by age 10, computers should

3Level 2: Inputs, Outputs, Transforms

for (i=nCol..Level 2:How it be represented as

a process with inputs/outputs?

light drawing

retina(gradient)

light lines (edges) drawing

3 Level 3: Implementation

Level 3:How is it implemented?

Center Surround Cellin retinal ganglion

Sobel Edge Detectorin computer vision

-1 0 +1

-1-2-1

+1 +2 +1

3 Class Discussion

• Give three examples of how biology has informed modern technology?

– ex. Wright Brothers- control flaps on airplane wings

3 Behavior Definition (graphical)

BEHAVIOR

SensoryInput

Patternof MotorActions

3 Types of Behaviors

• Reflexive – stimulus-response, often abbreviated S-R

• Reactive – learned or “muscle memory”

• Conscious – deliberately stringing together

WARNING Overloaded terms:Roboticists often use “reactive behavior” to mean purely reflexive,

And refer to reactive behaviors as “skills”

WARNING Overloaded terms:Roboticists often use “reactive behavior” to mean purely reflexive,

And refer to reactive behaviors as “skills”

3 Ethology: Coordination and Control of Behaviors

Nobel 1973 in physiology or medicine•von Frisch•Lorenz•Tinbergen

www.nobel.se

INNATE RELEASING MECHANISMSINNATE RELEASING MECHANISMS

3 Arctic Terns

• Arctic terns live in Arctic (black, white, gray environment, some grass) but adults have a red spot on beak

• When hungry, baby pecks at parent’s beak, who regurgitates food for baby to eat

• How does it know its parent?– It doesn’t, it just goes for the largest red spot in its field of

view (e.g., ethology grad student with construction paper)

– Only red thing should be an adult tern

– Closer = large red

3 behavior template

BEHAVIOR

SensoryInput

3 “the feeding behavior”

FeedingBEHAVIOR

SensoryInput

RED PECK AT RED

3 the releaser template

Releaser

present? N

/dev/null

Sensory inputand/or

internal state

3 “the feeding releaser”

FeedingBEHAVIOR

RED PECK AT RED

Releaser

present? N

/dev/null

internal state

RED &HUNGRY

sensory input

3 Innate Releasing Mechanisms

BEHAVIOR

SensoryInput

Releaser

present? N

/dev/null

Sensory inputand/or

internal state

3 Example: Hide Behavior

• Programmed in C++, << 100 LOC

• shows – taxis (oriented relative to light, wall, niche)– fixed action pattern (persisted after light was

off)– reflexive (stimulus, response)– impliciting sequencing– use of internal state

3 Example: Cockroach Hide

• light goes on, the cockroach turns and runs

• when it gets to a wall, it follows it

• when it finds a hiding place (thigmotrophic), goes in and faces outward

• waits until not scared, then comes out

• even if the lights are turned back off earlier

3 Reflexive Behaviors S-R

3 Fixed Pattern Actions

3 Exhibits Taxis

to light

to wall

to niche

3 Class Exercise

• Draw flowchart of how this works

3 Break into Behaviors

Follow-wall

3 Find Releasers

Follow-wall

present?N

SCARED &SURROUNDED

present?N

present?N Ooops, need internal state:

ScaredOoops, need internal state:

Scared

3 Internal State Set

Follow-wall

present?N

SCARED &SURROUNDED

present?N

BLOCKED &SCARED

present?N

SCARED

3 Action

Follow-wall

present?N

SCARED &SURROUNDED

present?N

BLOCKED &SCARED

present?N

SCARED

steer 360,drive forward

steer =F(dist to wall)drive forward const.

3 Sensory Input

Follow-wall

present?N

SCARED &SURROUNDED

present?N

BLOCKED &SCARED

present?N

SCARED

encoders

3 How Do You Link Them?

Follow-wall

present?N

SCARED &SURROUNDED

present?N

BLOCKED &SCARED

present?N

SCARED

encoders

3 Analogy:IRMs work on THREADS,not sequential processing!

• Very simple modules

• Nice building blocks since not directly linked

• If one module (part of brain) fails, what happens?

3 What happens when there’s a conflict from concurrent

behaviors?

• Equilbrium– Feeding squirrels-feed,

flee: hesitate in-between

• Dominance– Sleepy, hungry: either

sleep or eat

• Cancellation– Sticklebacks defend,

attack: build a nest

3 Perception

• Two uses of perception (can be the same percept)– Release a behavior– Guide a behavior

• Action-oriented perception (Neisser)– Planning is not needed to act – Perception is selective

CognitiveActivity

Perceptionof

Environment

Samples, FindsPotential Actions

Acts &ModifiesWorld

Directs whatto look for

3 Gibson’s Ecological Approach

• Acting and sensing co-evolved as agent survived in a particular environment. The environment affords the agent what it needs to survive.

• The perception needed to release or guide the “right action” is directly in the environment, not inferred or memorized – Ex. Red on Artic Terns== food

– Ex. Sound of filling container==full

• Percepts are called affordances or said to be obtained through direct perception

3 Gibsonian Affordances• How do you know you’re going fast in a car? Or in a

space movie?

• How do animals know when to mate?

• How do mosquitoes know to bite in the most tender areas?

• What should you do when you think you’re being stalked by a mountain lion?

• What’s your favorite fishing lure?

3 Sittability

3 But does this really hold for everything?

• “my car” versus “your car”?– Difference is

• where I parked it (memory)

• Semantic meaning (cars aren’t generic like nuts to a squirrel)

• Neisser’s Two Systems– Direct Perception: older, behavioral

– Recognition: evolved later, deliberative

3 Review Questions• What are the levels of a computational theory?

• Existence proof, inputs-outputs-transformations, implementation

• What is a behavior?

• A behavior is a mapping of sensory inputs to a pattern of motor actions

• Is sequencing normally implicit or explicit in IRM?

• implicit

• What is an affordance?

• A potentiality in the environment for an action

3 Schema Theory

schema- is used in cognitive science and ethology to refer to a particular organized way of perceiving cognitively and responding to a complex situation or set of stimuli

• is generic, equivalent to an object in OOP– schema specific knowledge (local data)

– procedural knowledge (methods)

• schema instantation is specific to a situation, equivalent to an instance in OOP

• a behavior is a schema, consists of– perceptual schema

– motor schema

3 Behavioral Schema

MotorSchema

PerceptualSchema

percept,gain

action,intensity

alternative PS, MSsequencing logic for reactive skills

(judgment value function)

Reflexive behaviors usually just have “methods”, not data

3 Ex. Fly Snapping Behavior IRM

snap(blob)track(blob)

x,y,z,100%

snap,100%

Releaser:small moving dark blob

present? N

/dev/null

3 Schema Instantiation (SI)

x,y,z,100%

snap,100%

present? N

/dev/null

3 Schema/Schema Instantiation

behaviorschema

releaser

present?N

snap(blob)track(blob)track(blob) snap(blob)snap(blob)track(blob)

PerceptualSchema Library

MotorSchema Library

x,y,z,100%

3 Advantages

• modular

• can assemble new behaviors from existing schemas– learning by experimentation

• can substitute alternatives– reroute nerves

3 Instantiation for each eye

snap,100%

x,y,z,100%

present?N

/dev/null

snap,100%

x,y,z,100%

present?N

/dev/null

Left eye

Right eye

Snap atvector sum(middle)

3 Where’s the MS and PS?

Follow-wall

present?N

SCARED &SURROUNDED

present?N

BLOCKED &SCARED

present?N

SCARED

encoders

3 General Principles

• All animals possess a set of behaviors

• Releasers for these behaviors rely on both internal state and external stimulus

• Perception is filtered; perceive what is relevant to the task

• Some behaviors and associated perception do not require explicit knowledge representation (e.g., rely on affordances)

3 Silicon v. Carbon

• Individual robots must survive, not species– detection of non-productive behaviors

– graceful degradation

• Must be able to predict emergent behaviors

• Not clear how to learn quickly

• Robots need more alternative perceptual schemas since poorer understanding of the environment

3 Unresolved Issues

• How to resolve conflicts?– behavioral arbitration/combination

• When is explicit representations, memory needed?

• How to set up or learn new sequences of behaviors

• What are the affordances for a particular ecology?

3 Take Home Thoughts…• Ideas bubbling up for robotics

– Maybe programming in terms of behaviors is better than STRIPS or trying to set up a complex hierarchy

– Intelligence has something to do with agent’s ecological niche: its abilities, its tasks (survival), and environment

– Perception is going to be critical because it releases and guides actions

– IRMs, Schemas are nice ways to start thinking about the computational structure of programming a robot

3 Review Questions• Think about the robots at the WTC. What are affordances of

victims?

• color, motion, sound, heat

• Can schema theory represent behaviors in both biological and computational systems?

• yes

• A behavior schema is composed of at least the following:

• motor schema and a perceptual schema

• What is an example of behavior-specific knowledge?

• sequencing in a skill, alternative PS or MS

BEHAVIOR

SensoryInput

Releaser

present? N

/dev/null

3 Inhibition

while (TRUE) { predator = sensePredator(); //has a time delay if (predator==PRESENT) //as long as predator persists flee(); else { food = senseFood(); hungry = checkStateHunger(); ... }}

Could also be done as an interrupt

biological foundations of the reactive paradigm

Documents

reactive paradigm – overview subsumption architecture by...

rxjava - yow! conferences · what is reactive programming?...

3 introduction to ai robotics (mit press)chapter 3:...

biological foundations of reactive and behavior-based...

pawel gburzynski & wlodek olesinski olsonet...

crm real time: the new paradigm · reactive outbound)...

reactive - land of confusion - github pages“reactive...

the theory and practice of causal commutative arrows ›...

rai notes on reactive paradigm

study on corporate foundations: an emerging...

chapter 7: hybrid deliberative/reactive paradigm

functional reactive programming,...

biological foundations of reactive and behavior-based...

probability in robotics trends in robotics research reactive...

jfokus-reactive the reactive landscapethe reactive landscape...

human development as an alternative paradigm conceptual...

4 introduction to ai robotics (mit press)chapter 4: the...

shibaura institute of technology advanced driver ... ·...

the reactive paradigm

an axiomatic semantics for functional reactive...