ee141 emergence of the embodied intelligence how to motivate a machine ? janusz starzyk school of...

74
EE141 Emergence of the Emergence of the Embodied Intelligence Embodied Intelligence How to Motivate a How to Motivate a Machine ? Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University, USA www.ent.ohiou.edu/~starzyk Neuroscience and Neuroscience and Embodied Embodied Intelligence Intelligence

Upload: lawrence-sparks

Post on 28-Dec-2015

225 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Emergence of the Emergence of the Embodied IntelligenceEmbodied Intelligence How to Motivate a Machine ?How to Motivate a Machine ?

Janusz StarzykSchool of Electrical Engineering and Computer Science, Ohio University, USA

www.ent.ohiou.edu/~starzyk

Cognitive Neuroscience Cognitive Neuroscience and Embodied Intelligenceand Embodied Intelligence

Page 2: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Traditional Artificial Intelligence Embodied Intelligence (EI) Embodiment of Mind EI Interaction with Environment How to Motivate a Machine Goal Creation Hierarchy GCS Experiment Motivated Learning Challenges of EI

We need to know how to organize it We need means to implement it We need resources to build and

sustain its operation Promises of EI

To economy To society

OutlineOutline

Page 3: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

“…Perhaps the last frontier of science – its ultimate challenge- is to understand the biological basis of consciousness and the mental process by which we perceive, act, learn and remember..” from Principles of Neural Science by E. R. Kandel et al. E. R. Kandel won Nobel Price in 2000 for his work on physiological

basis of memory storage in neurons. “…The question of intelligence is the last great

terrestrial frontier of science...” from Jeff Hawkins On Intelligence. Jeff Hawkins founded the Redwood Neuroscience Institute devoted

to brain research

IntelligenceIntelligence

AI’s holy grailFrom Pattie Maes MIT Media Lab

Page 4: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Is what is intelligence?Is what is intelligence?

Page 5: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Various Definitions of IntelligenceVarious Definitions of Intelligence The American Heritage Dictionary:

The capacity to acquire and apply knowledge. The faculty of thought and reason.

Webster Dictionary: The act or state of knowing; the exercise of the understanding. The capacity to know or understand; readiness of comprehension;

Wikipedia – The Free Encyclopedia: The capacity to reason, plan, solve problems, think abstractly, comprehend ideas

and language, and learn. Kaplan & Sadock:

The ability to learn new things, recall information, think rationally, apply knowledge and solve problems.

On line dictionary dict.die.net The ability to comprehend; to understand and profit from experience

The classical behavioral/biologists: The ability to adapt to new conditions and to successfully cope with life situations.

Dr. C. George Boeree, professor in the Psychology Department at Shippensburg University: A person's capacity to (1) acquire knowledge (i.e. learn and understand), (2) apply

knowledge (solve problems), and (3) engage in abstract reasoning. Stanford University Professor of Computer Science Dr. John McCarthy, a pioneer in AI:

The computational part of the ability to achieve goals in the world. Scientists in Psychology:

Ability to remember and use what one has learned, in order to solve problems, adapt to new situations, and understand and manipulate one’s reality.

Page 6: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141From http://www.indiana.edu/~intell/map.shtml

Mainstream Science on Intelligence December 13, 1994: An Editorial With 52 Signatories, by Linda S. Gottfredson, University of Delaware

Intelligence is a very general mental capability that, among other things, involves the ability to reason, plan, solve problems, think abstractly, comprehend complex ideas, learn quickly and learn from experience.

IntelligenceIntelligence

Page 7: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Animals’ IntelligenceAnimals’ Intelligence Defining intelligence

through humans is not appropriate to design intelligent machines:

– Animals are intelligent too

Dog IQ test: Dogs can learn 165 words (similar to 2 year olds) Average dog has the mental abilities of a 2-year-old child (or better) They would beat a 3- or 4-year-old in basic arithmetic, Dogs show some basic emotions, such as happiness, anger and disgust “The social life of dogs is very complex - more like human teenagers -

interested in who is moving up in the pack, who is sleeping with who etc,“ says professor Stanleay Coren from University of British Columbia

Border collies, poodles, and german shepards are the smartest dogs

Page 8: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Traditional AITraditional AI Embodied Intelligence Embodied Intelligence Abstract intelligence

attempt to simulate “highest” human faculties:

– language, discursive reason, mathematics, abstract problem solving

Environment model Condition for problem

solving in abstract way “brain in a vat”

Embodiment knowledge is implicit in the

fact that we have a body– embodiment supports brain

development

Intelligence develops through interaction with environment Situated in environment Environment is its best model

Page 9: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Design principles of intelligent systemsDesign principles of intelligent systems

Design principlessynthetic methodologytime perspectivesemergencediversity/complianceframe-of-referencecomplete agent

principle

from Rolf Pfeifer “Understanding of Intelligence”

From: www.spectrum.ieee.org/.../biorobot11f-thumb.jpg

Page 10: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Design principles of intelligent systemsDesign principles of intelligent systemsfrom Rolf Pfeifer “Understanding of Intelligence”, 1999

Interaction with complex environment

ecological balance redundancy principle parallel, loosely

coupled processes asynchronous sensory-motor

coordination value principle cheap design Agent

Drawing by Ciarán O’Leary- Dublin Institute of Technology

Page 11: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

The principle of “cheap design”The principle of “cheap design”

intelligent agents: “cheap” exploitation of ecological

niche economical (but redundant) exploitation of specific

physical properties of interaction with real world

Page 12: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Principle of “ecological balance”Principle of “ecological balance”

balance / task distribution between morphology neuronal processing (nervous

system) materials environment

balance in complexity given task environment match in complexity of sensory,

motor, and neural system

Page 13: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

The redundancy principleThe redundancy principle

redundancy prerequisite for adaptive behavior

partial overlap of functionality in different subsystems

sensory systems: different physical processes with “information overlap”

Page 14: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Generation of sensory stimulation Generation of sensory stimulation through interaction with environmentthrough interaction with environment

multiple modalities constraints from

morphology and materials

generation of correlations through physical process

basis for cross-modal associations

Page 15: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

The principle of sensory-motor The principle of sensory-motor coordinationcoordination

self-structuring of sensory data through interaction with environment

physical process —not „computational“

prerequisite for learning

Holk Cruse•no central control•only local neuronal communication•global communication through environment

neuronal connections

Page 16: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

The principle of parallel, loosely The principle of parallel, loosely coupled processescoupled processes

Intelligent behavior emergent from agent-environment interaction

Large number of parallel, loosely coupled processes

Asynchronous Coordinated through agent’s –sensory-motor system–neural system–interaction with environment

Page 17: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

So what is an Embodied So what is an Embodied Intelligence ?Intelligence ?

Page 18: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Embodied Intelligence Embodied Intelligence

Definition Embodied Intelligence (EI) is a mechanism that learns

how to survive in a hostile environment

– Mechanism: biological, mechanical or virtual agent

with embodied sensors and actuators– EI acts on environment and perceives its actions– Environment hostility is persistent and stimulates EI to act– Hostility: direct aggression, pain, scarce resources, etc– EI learns so it must have associative self-organizing memory– Knowledge is acquired by EI

Page 19: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

EI mimics biological intelligent systems, extracting general principles of intelligent behavior and applying them to design intelligent agents.

Knowledge is not entered into such systems, but rather is a result of their successful interaction with the environment.

Embodied intelligent systems adapt to unpredictable and dynamic situations in the environment by learning, which gives them a high degree of autonomy.

Learning in such systems is incremental, with continuous prediction of the input associations based on the emerging models - only new information is registered in the memory.

Embodied IntelligenceEmbodied Intelligence

Page 20: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

What is Embodiment of a Mind?What is Embodiment of a Mind?

Page 21: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Embodiment

Actuators

Sensors

Intelligence core

channel

channel

Embodiment

Sensors

Intelligence core

Environment

channel

channelActuators

Embodiment

Actuators

Sensors

Intelligence core

channel

channel

Embodiment

Sensors

Intelligence core

Environment

channel

channelActuators

Embodiment of a MindEmbodiment of a Mind Embodiment of a mind is a

part of environment under control of the mind

It contains intelligence core and sensory motor interfaces to interact with environment

It is necessary for development of intelligence

It is not necessarily constant or in the form of a physical body

Boundary of embodiment transforms modifying brain’s self-determination

Page 22: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Brain learns own body’s dynamic Self-awareness is a result of

identification with own embodiment Embodiment can be extended by

using tools and machines Successful operation is a function

of correct perception of environment and own embodiment

Embodiment of MindEmbodiment of Mind

Page 23: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Requirements for Embodied IntelligenceRequirements for Embodied Intelligence

State oriented Learns spatio-temporal patterns

Situated in time and space

Learning Perpetual learning

Screening for novelty

Value driven Pain detection

Pain management

Goal creation

Competing goals Emergence

artificial evolution

self-organization

Page 24: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

INPUT OUTPUT

Simulation or

Real-World System

TaskEnvironment

Agent Architecture

Long-term Memory

Short-term Memory

Reason

ActPerceive

RETRIEVAL LEARNING

EI Interaction with EnvironmentEI Interaction with Environment

From Randolph M. Jones, P : www.soartech.com

Page 25: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Kandel Fig. 23-5

Sensory Inputs CodingSensory Inputs Coding

How do we process and represent sensory

information?

Richard Axel, 1995

Foot

Hip Trunk

ArmHand

Face

Tongue

Larynx

Kandel Fig. 30-1

Visual, auditory, olfactory, tactile, smell -> motor

Page 26: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Challenges of Embodied IntelligenceChallenges of Embodied Intelligence

Page 27: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Challenges of Embodied IntelligenceChallenges of Embodied Intelligence Development of sensory interfaces

Active vision Speech processing Tactile, smell, taste, temperature, pressure

sensing Additional sensing

– Infrared, radar, lidar, ultrasound, GPS, etc.– Can too many senses be less useful?

Development of pain sensors Energy, temperature, pressure,

acceleration level Teacher input

Development of motor interfaces Arms, legs, fingers, eye movement

Intelligence core

Embodiment

Sensors

ActuatorsEnvironmentEnvironment

Page 28: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Challenges of Embodied Intelligence (cont.)Challenges of Embodied Intelligence (cont.) Finding algorithmic solutions for

Association, memory, sequence learning, invariance building, representation, anticipation, value learning, goal creation, planning

Development of circuits for neural computing Determine organization of artificial

minicolumn Self-organized hierarchy of

minicolumns for sensing and motor control

Self-organization of goal creation pathway

Page 29: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

V. Mountcastle argues that all regions of the brain perform the same computational algorithm

Groups of neurons (minicolumns) connected in a pseudorandom way

Same structure Minicolumns organized in

macrocolumns

VB Mountcastle (2003). Introduction [to a special issue of Cerebral Cortex on columns]. Cerebral Cortex, 13, 2-4. 

Human Intelligence – Cortex Uniform StructureHuman Intelligence – Cortex Uniform Structure

Page 30: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

“The basic unit of cortical operation is the minicolumn …

It contains of the order of 80-100 neurons except in the primate striate cortex, where the number is more than doubled.

The minicolumn measures of the order of 30-50 m in transverse diameter, separated from adjacent minicolumns by vertical, cell-sparse zones …

The minicolumn is produced by the iterative division of a small number of progenitor cells in the neuroepithelium.” (Mountcastle)

Mini ColumnsMini Columns

Copyright © 2006-2008, all rights reserved, Visualbiotech

Page 31: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Sensory neurons are responsible for representing environment receive inputs from sensors or sensory neurons on lower level represent the environment receive feedback input from motor and higher level neurons help to activate motor and reinforcement neurons

Motor neurons are responsible for actions and skills are activated by reinforcement and sensory neurons activate actuators or provide an input to lower level motor neurons provide planning inputs to sensory neurons

Reinforcement neurons are responsible for building the value system, goal creation, learning, and exploration receive inputs from reinforcement neurons on the lower level receive inputs from sensory neurons provide inputs to motor neurons initiate learning and force exploration

Artificial Minicolumn OrganizationArtificial Minicolumn Organization

Page 32: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Sensory and motor pathways are interconnected on different hierarchical levels

outputs

internal representations

Sensory-motor CoordinationSensory-motor Coordination

inputs

Page 33: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Sensory-motor CoordinationSensory-motor Coordination

R: representationE: expectationA: associationD: directionP: planning Environment

D

R

E

A

Sensor path Motor path

Increasing connection’s adaptability’

Goal creation &Value system

Page 34: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

How to Motivate a Machine ?How to Motivate a Machine ?

A fundamental question is what motivates an agent to do anything, and in particular, to enhance its own complexity?

What drives an agent to explore the environment and learn ways to effectively interact with it?

Page 35: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

How to Motivate a Machine ?How to Motivate a Machine ? Pfeifer claims that an agent’s motivation should emerge

from the developmental process. He called this the “motivated complexity” principle. Chicken and egg problem? An agent must have a motivation to

develop while motivation comes from development?

Steels suggested equipping an agent with self-motivation. “Flow” experienced when people perform their expert activity well

would motivate to accomplish even more complex tasks. Humans get internal reward for activities that are slightly above their

level of development (Csikszentmihalyi). But what is the mechanism of “flow”?

Oudeyer proposed an intrinsic motivation system. Motivation comes from a desire to minimize the prediction error. Similar to “artificial curiosity” presented by Schmidhuber.

Page 36: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

How to Motivate a Machine ?How to Motivate a Machine ?

Can a machine that only implements externally given goals be intelligent?

If not how these goals can be created?

•There is a need for a hierarchy of values.•Not all values can be predetermined by the designer.•There is a need for motivation to act, explore and learn.•As machine makes new observations about the environment, there is a need to relate them to goals and values and create new goals and values.

Page 37: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

How to Motivate a Machine ?How to Motivate a Machine ?

Exploration is needed in order to learn and to model the

environment. But is this mechanism the only motivation we need to develop

intelligence? Can “flow” ideas explain goal oriented learning? Can we find a more efficient mechanism for learning?

I suggest a simpler mechanism to motivate a machine.

Although artificial curiosity helps to explore the environment, it leads to learning without a specific purpose. It may be compared to exploration in

reinforcement learning. internal reward motivates the machine to

perform exploration.

Page 38: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

How to Motivate a Machine ?How to Motivate a Machine ? I suggest that it is the hostility of the environment, in the

definition of EI that is the most effective motivational factor. It is the pain we receive that moves us. It is our intelligence determined to reduce this pain that motivates us

to act, learn, and develop.

Both are needed - hostility of the environment and

intelligence that learns how to reduce the pain. Thus pain is good. Without pain there would be no intelligence. Without pain we would not be motivated to develop.

Fig. englishteachermexico.wordpress.com/

Page 39: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Motivated Learning Motivated Learning I suggest a goal-driven mechanism to motivate

a machine to act, learn, and develop. A simple pain based goal creation system is

explained next. It uses externally defined pain signals that are

associated with primitive pains. Machine is rewarded for minimizing the primitive

pain signals. Definition: Motivated learning (ML) is learning based on the

self-organizing system of goal creation in embodied agent. Machine creates higher level (abstract) goals based on the primitive

pain signals. It receives internal rewards for satisfying its goals (both primitive and

abstract). ML applies to EI working in a hostile environment.

Page 40: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Pain-center and Goal Creation for MLPain-center and Goal Creation for ML

Simple Mechanism Creates hierarchy of

values Leads to formulation of

complex goals Pain comparators release

reinforcement neurotransmitter:• Pain increase -

inhibitory• Pain decrease -

excitatory Forces exploration

+

-

Environment

Sensor

MotorPain level

Dual pain levelPain increase

Pain decrease

(-)

(+)

Excitation

(-)

(-)

(+)

(+)

Page 41: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Pain-center and Goal Creation for MLPain-center and Goal Creation for ML

+

-

Sensor

Motor

Pain detection

Dualpain

memory

Pain increase

Paindecrease

(-)

(+)

Stimulation

(-)

(+)

activation

need

Sensory neuron

Motor neuron

Pain detection/goal creation center

Reinforcement neurotransmitter

Missing objects

inhibition

expe

ctat

ion

Page 42: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Abstract Goal Creation for MLAbstract Goal Creation for ML

The goal is to reduce the primitive pain level Abstract goals are created if they satisfy the primitive goals

Expectation

AssociationInhibitionReinforcementConnectionPlanning

- +

PainDual pain

Food

refrigerator

- +

Stomach

Abstract pain(Delayed memory of pain)

“food” becomes a sensory input to

abstract pain center

Sensory pathway(perception, sense)

Motor pathway(action, reaction)

Primitive Level

Level I

Level II

Eat

Open

Page 43: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Abstract Goal HierarchyAbstract Goal Hierarchy

Hierarchy of abstract goals is created if theysatisfy the primitive goals

ActivationStimulationInhibitionReinforcementEchoNeedExpectation

- +

+

Sugar levelPrimitive Level

Level I

Level IIMoney

-

Food

Spend

Eat

+

Sensory pathway(perception, sense)

Motor pathway

(action, reaction)

Level IIIJob

-

Work

Page 44: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

The Three Pathways CombinedThe Three Pathways Combined

Goal creation, sensory and motor pathways interact on different hierarchy levels

Pain driven goal creation sets goal priorities

Pain tree IPain tree IIMotor pathwaySensory pathwayPain center to motor Sensor to motor Sensor to pain center

Page 45: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

EI Interaction with EnvironmentEI Interaction with Environment

INPUT OUTPUT

Simulation or

Real-World System

TaskEnvironment

EI Architecture

Goal Creation

ActPerceive Competing goals

Planning

Pain

INPUT OUTPUT

Simulation or

Real-World System

TaskEnvironment

EI Architecture

Goal Creation

ActPerceive Competing goals

Planning

Pain

EI machine interacts with environment using its three pathways

Page 46: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Goal Creation Experiment in MLGoal Creation Experiment in ML

Sensory-motor pairs and their effect on the environment

PAIR # SENSORY MOTOR INCREASES DECREASES

1 Food Eat sugar level food supplies

8 Grocery Buy food supplies money at hand

15 Bank Withdraw money at hand spending limits

22 Office Work spending limits

job opportunities

29 School Study job opportunities

-

Page 47: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Goal Creation Experiment in MLGoal Creation Experiment in ML

Goal Creating Neural Network

WTA WTA

S MP GB

Sk

Bk

Mk

GkPk 1

Pl

WTA WTA

S MP GB

Sk 1

Bk

Mk

GkPk 1

Pl

WTA WTA

S MP GB

Sk

Bk

Mk

GkPk 1

Pl

WTA WTA

S MP GB

Sk 1

Bk

Mk

GkPk 1

Pl

Page 48: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Goal Creation Experiment in MLGoal Creation Experiment in ML

Trainable connections between pain, bias, and goal neurons

GP

2

11

w

10

P

GP

2

11

UA10

P

GP

2

11 10

P

GS2S2S2S

2

11

UA10

P

GP

2

1

s

1

w

10

P

GP

2

11

UA10

P

GP

2

1

BB

1 10

P

GS2S2S2SS2S2S2S

M 2

11

PG

BP

UA10

P

w

GP

2

11

w

10

P

GP

2

11

UA10

P

GP

2

11 10

P

GS2S2S2SS2S2S2S

2

11

UA10

P

GP

2

1

s

1

w

10

P

GP

2

11

UA10

P

GP

2

1

BB

1 10

P

GS2S2S2SS2S2S2S

M 2

11

PG

BP

UA10

P

w

Page 49: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Goal Creation Experiment in MLGoal Creation Experiment in ML

Pain signals in CGS simulation

0 100 200 300 400 500 6000

1

Primitive Hunger

Pa

in

0 100 200 300 400 500 6000

0.5

Lack of Food

Pa

in

0 100 200 300 400 500 6000

0.5

Empty Gorcery

Pa

in

Discrete time

Page 50: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Goal Creation Experiment in MLGoal Creation Experiment in ML

Action scatters in 5 CGS simulations

0 100 200 300 400 500 6000

5

10

15

20

25

30

35

40Goal Scatter Plot

Go

al I

D

Discrete time

Page 51: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Goal Creation Experiment in MLGoal Creation Experiment in ML

The average pain signals in 100 CGS simulations

0 100 200 300 400 500 6000

0.5

Primitive Hunger

Pai

n

0 100 200 300 400 500 6000

0.10.2

Lack of FoodP

ain

0 100 200 300 400 500 6000

0.10.2

Empty Gorcery

Pai

n

0 100 200 300 400 500 6000

0.10.2

Lack of Money

Pai

n

0 100 200 300 400 500 6000

0.050.1

Lack of JobOpportunitites

Pai

n

Discrete time

Page 52: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Goal Creation Experiment in MLGoal Creation Experiment in ML

Comparison between GCS and RL

Page 53: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Compare RL (TDF) and ML (GCS)Compare RL (TDF) and ML (GCS)

Mean primitive pain Pp value as a function of the number of iterations:

- green line for TDF -blue line for GCS.

Primitive pain ratio with pain threshold 0.1

Page 54: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Comparison of execution time on log-log scale TD-Falcon green GCS blue

Combined efficiency of GCS 1000 better than TDF

Compare RL (TDF) and ML (GCS)Compare RL (TDF) and ML (GCS)

Problem solved

Conclusion: embodied intelligence, with motivated learning based on goal creation system, effectively integrates environment modeling and decision making – thus it is poised to cross the chasm

Page 55: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Reinforcement LearningReinforcement Learning Motivated Learning Motivated Learning Single value function Measurable rewards

Can be optimized

Predictable Objectives set by

designer Maximizes the reward

Potentially unstable

Learning effort increases with complexity

Always active

Multiple value functions One for each goal

Internal rewards Cannot be optimized

Unpredictable Sets its own objectives Solves minimax problem

Always stable

Learns better in complex environment than RL

Acts when needed

Page 56: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

How can we make human level How can we make human level intelligence?intelligence?

We need to know how We need means to

implement it We need resources to

build and sustain its operation

Page 57: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006

Resources – Evolution of ElectronicsResources – Evolution of Electronics

Page 58: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141 By Gordon E. MooreBy Gordon E. Moore

Page 59: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Page 60: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006

Clock Speed (doubles every 2.7 years)

Page 61: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Doubling (or Halving) timesDoubling (or Halving) times

Dynamic RAM Memory “Half Pitch” Feature Size 5.4

years Dynamic RAM Memory (bits per dollar) 1.5

years Average Transistor Price 1.6 years Microprocessor Cost per Transistor Cycle 1.1

years Total Bits Shipped 1.1

years Processor Performance in MIPS 1.8

years Transistors in Intel Microprocessors 2.0 years Microprocessor Clock Speed 2.7

yearsFrom Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006

Page 62: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006

Page 63: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141From Hans Moravec, Robot, 1999

Page 64: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Software or hardware?Software or hardware?

Sequential Error prone Require programming Low cost Well developed

programming methods

Concurrent Robust Require design Significant cost Hardware prototypes

hard to build

Software Hardware

Page 65: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

2005 2010 2015 2020 2025 2030 2035 204010

4

105

106

107

108

109

1010

1011

Year

Num

ber

of n

euro

ns

Software Simulation (PC based) Hardware approach (FPGA)

Analog VLSI

Future software/hardware capabilitiesFuture software/hardware capabilities

Human brain complexity

Page 66: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Why should we care?Why should we care?

Source: SEMATECHSource: SEMATECH

Page 67: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

0%

20%

40%

60%

80%

100%

1999

2002

2005

2008

2011

2014

% Area Memory

% Area ReusedLogic

% Area New Logic

Percent of die area that must be occupied by memory to maintain SOC design productivity

Design Productivity Gap Design Productivity Gap Low-Value Designs? Low-Value Designs?

Source = Japanese system-LSI industry

Page 68: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Self-Organizing Learning Arrays SOLARSelf-Organizing Learning Arrays SOLAR

Integrated circuits connect transistors into a system-millions of transistors easily assembled-first 50 years of microelectronic revolution

Self-organizing arrays connect processors into a system-millions of processors easily assembled-next 50 years of microelectronic revolution

* Self-organization * Sparse and local

interconnections * Dynamically

reconfigurable * Online data-driven

learning

Page 69: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Promises of embodied intelligencePromises of embodied intelligence To society

Advanced use of technology– Robots– Tutors– Intelligent gadgets

Intelligence age follows– Industrial age– Technological age– Information age

Society of minds– Superhuman intelligence– Progress in science– Solution to societies’ ills

To industry Technological development New markets Economical growth

ISAC, a Two-Armed Humanoid RobotISAC, a Two-Armed Humanoid RobotVanderbilt UniversityVanderbilt University

Page 70: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

2002 2010 2020 2030

Biomimetics and Bio-inspired SystemsBiomimetics and Bio-inspired SystemsImpact on Space Transportation, Space Science and Earth ScienceImpact on Space Transportation, Space Science and Earth Science

Mis

sio

n C

om

ple

xity

Biological Mimicking

Embryonics

Extremophiles

DNA Computing

Brain-like computing

Self Assembled Array

Artificial nanoporehigh resolution

Mars in situlife detector

Sensor Web

Biological nanoporelow resolution

Skin and Bone

Self healing structureand thermal protection

systems

Biologically inspired aero-space systems

Space Transportation

Page 71: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Sounds like science fictionSounds like science fiction

If you’re trying to look far ahead, and what you see seems like science fiction, it might be wrong.

But if it doesn’t seem like science fiction, it’s definitely wrong.

From presentation by Feresight Institute

Page 72: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Embodied Artificial IntelligenceEmbodied Artificial IntelligenceBased on: [1] E. R. Kandel et al. Principles of Neural Science, McGraw-Hill/Appleton & Lange; 4 edition, 2000. [2] F. Inda, R. Pfeifer, L. Steels, Y. Kuniyoshi, “Embodied Artificial

Intelligence,” International seminar, Germany, July 2003.[3] R. Chrisley, “Embodied artificial intelligence, ” Artificial

Intelligence, vol. 149, pp.131-150, 2003.[4] R. Pfeifer and C. Scheier, Understanding Intelligence, MIT

Press, Cambridge, MA, 1999. [5] R. A. Brooks, “Intelligence without reason,” In Proc. IJCAI-91.

(1991) 569-595 .[6] R. A. Brooks, Flesh and Machines: How Robots Will Change

Us, (Pantheon, 2002).

[7] R. Kurzweil The Age of Spiritual Machines: When Computers

Exceed Human Intelligence, (Penguin, 2000).

Page 73: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141

Questions?Questions?

Page 74: EE141 Emergence of the Embodied Intelligence How to Motivate a Machine ? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University,

EE141