Overview

S-Learning Rotary Robot

Reaching

Grasping

Context-Based Similarity

Natural Language Processing

Perception

Bio-Inspiration

Applications

BECCA

BECCA—A Brain Emulating Cognition and Control

Architecture

Brandon Rohrer Intelligent Systems, Robotics, and Cybernetics Group

Sandia National Laboratories

2008 AAAI, BICA Workshop

Overview

S-Learning Rotary Robot

Reaching

Grasping

Context-Based Similarity

Natural Language Processing

Perception

Bio-Inspiration

Applications

BECCA

Biases: What “biologically-inspired” means to me

• Self preservation and perpetuation is the implicit goal of biological systems.– If it is not, they don’t last very long

• Implies real-world interaction • There is no a priori model of the environment

– All categories and concepts are acquired• Human cognition violates rationality, formal logics,

optimality, universal concept definitions, strict ontologies, and strict perceptual categorizations– A BICA should not presuppose any of these

• Broad functional equivalence to humans is the goal of BICA research – All empirical data (neuroanatomy, cognitive

neuroscience, experimental psychology, etc.) are rich sources for clues about how this might be achieved

– Existing theories and constructs describing empirical data should be used with caution

Overview

S-Learning Rotary Robot

Reaching

Grasping

Context-Based Similarity

Natural Language Processing

Perception

Bio-Inspiration

Applications

BECCA

BECCA—A Brain Emulating Cognition and Control Architecture

Brandon Rohrer

• BECCA is a biomimetic approach to achieving human-like reasoning, perception, learning, and movement control in machines

• It has two core algorithms – S-Learning: A solution to some Dynamic

Reinforcement Learning problems

– Context-Based Similarity: A solution to the problem of semantically clustering symbols

• Given an ordered set of symbols, determine the semantic distance (similarity) between any two

),,( tsafs titit ),( tsgr tit

0iirV

Overview

S-Learning Rotary Robot

Reaching

Grasping

Context-Based Similarity

Natural Language Processing

Perception

Bio-Inspiration

Applications

BECCA

BECCA Operational Diagram

• Sensor and control information are handled symbolically– are passed in “episodic” fashion, quantized in time,– are discretized in magnitude,– are treated categorically

• extrapolation and interpolation does not occur explicitly.

• Allows very general application

Overview

S-Learning Rotary Robot

Reaching

Grasping

Context-Based Similarity

Natural Language Processing

Perception

Bio-Inspiration

Applications

BECCA

S-Learning Algorithm:A solution to some Dynamic RL problems

• S-Learning (sequence learning) – records observed sequences and uses them

• to make predictions about future events • and control decisions

• Algorithm description– 1. Learning

• At each iteration, record the longest novel sequence• Reinforce the longest familiar sequence

– 2. Prediction• Find sequences that begin with the most recent state(s)• The remainder of those sequences constitute predictions

– 3. Control• Evaluate each prediction for potential reward• Select a prediction to re-enact• Execute the same action sequence

Overview

S-Learning Rotary Robot

Reaching

Grasping

Context-Based Similarity

Natural Language Processing

Perception

Bio-Inspiration

Applications

BECCA

S-Learning: Rotary robot

• Simulation of a one degree-of-freedom rotary pointer robot, – Sensor quantized in 10º increments– Movement by 10º increments

• S-Learning demonstrated the ability to learn and predict hard nonlinearities

• S-Learning performed optimally even in the presence of– Scrambled sensor conditions– Gain reversals– Stochastic movement errors– Random time delays

• No explicit model of the system was provided—its workings were discovered by S-Learning

Overview

S-Learning Rotary Robot

Reaching

Grasping

Context-Based Similarity

Natural Language Processing

Perception

Bio-Inspiration

Applications

BECCA

S-Learning: Reaching simulation

• Two degree-of-freedom robot reaching simulation– Approximately human parameters

used for inertia. movement characteristics, and sensing capabilities

• Robot learned to reach a fixed target at an arbitrary position in the plane

• Demonstrated generalization– Learning in one task was applied to

a second task– This, despite the complete

separation of the sensory representations of the two tasks

• No explicit model of the system was provided —its workings were discovered by S-Learning

Early

Mature

Intermediate

Click to play videos

Overview

S-Learning Rotary Robot

Reaching

Grasping

Context-Based Similarity

Natural Language Processing

Perception

Bio-Inspiration

Applications

BECCA

S-Learning: Grasping simulation

• Three degree-of-freedom robot grasping simulation with rich sensors:– Coarse vision– Coarse position– Contact pressure

• Robot learned to reach a fixed target at an given position in the plane

• Learned to coordinate grasp with motion to grab target

• No explicit model of the system was provided —its workings were discovered by S-Learning

Click to play video

Overview

S-Learning Rotary Robot

Reaching

Grasping

Context-Based Similarity

Natural Language Processing

Perception

Bio-Inspiration

Applications

BECCA

Context-Based Similarity (CBS):A solution to semantic symbol clustering

• Underlying concept: Symbols are similar if they occur in identical contexts.– Context refers to the symbols that temporally precede

and follow the symbol of interest.• CBS finds the word “great” and the phrase “very

large” to be similar because they are preceded and followed by the same symbols (words)—they have identical contexts

Overview

S-Learning Rotary Robot

Reaching

Grasping

Context-Based Similarity

Natural Language Processing

Perception

Bio-Inspiration

Applications

BECCA

CBS: Natural Language Processing

• After reading 25 million words, CBS performed synonym extraction (finding sets of words that occurred in contexts identical to a seed word)

• No part-of-speech tags were given – In fact, CBS did not do anything that was specific to

English, text, or language in general. It would have handled encoder readings, force data, or image properties the same way.

• Plausible synonym groups were created

Overview

S-Learning Rotary Robot

Reaching

Grasping

Context-Based Similarity

Natural Language Processing

Perception

Bio-Inspiration

Applications

BECCA

CBS: Perception and concept formation

• Formulating the concept of the object <vehicle> by compiling specific examples

• Repeated sequences of 1) a static video background, 2) a dynamic video component created by a moving vehicle, 3) detected motion, and 4) detected sound allow the semantic cluster or concept of <vehicle> to be formed.

Overview

S-Learning Rotary Robot

Reaching

Grasping

Context-Based Similarity

Natural Language Processing

Perception

Bio-Inspiration

Applications

BECCA

BECCA Operational Diagram

• When combined, S-Learning and Context-Based Similarity provides BECCA’s full capabilities.

– the ability to manage the curse of dimensionality in complex systems

– concept generation– hierarchical processing for additional abstraction and

sophistication

Overview

S-Learning Rotary Robot

Reaching

Grasping

Context-Based Similarity

Natural Language Processing

Perception

Bio-Inspiration

Applications

BECCA

Biological Inspiration

• Discrete motor actions observed in slow movements (Vallbo and Wessberg 1993), infants’ movements (von Hofsten 1991) and repetitive movements (Woodworth 1899)

• Discrete sensory states proposed by James (1890); evidence seen in carefully structured psychophysical experiments (Gho and Varela 1988, VanRullen and Koch 2003)

• Periodic processing and storage suggested by thalamcortical loops (Rodriguez, Whitson, and Granger 2004)

• Sequence storage transition from hippocampus to parahippocampus to cortex described (Eichenbaum et al. 1999)

• Possible mechanism for classifying and selecting between sequences in cerebellar-cortical-basal-ganglionic loops (Houk 2005)

Overview

S-Learning Rotary Robot

Reaching

Grasping

Context-Based Similarity

Natural Language Processing

Perception

Bio-Inspiration

Applications

BECCA

Helping robots handlereal-world interactions

Potential problem spaces for BECCA-driven robots:

• Learn to manipulate unfamiliar objects• Learn complex perceptuo-motor tasks • Create high-level abstractions • Make predictions about future events• Use reasoning to achieve goals• Solve poorly-posed problems• Generalize experience to novel

situations