Pat LangleyPat LangleyComputational Learning LaboratoryComputational Learning Laboratory

Center for the Study of Language and InformationCenter for the Study of Language and InformationStanford University, Stanford, California USAStanford University, Stanford, California USA

http://cll.stanford.edu/http://cll.stanford.edu/

Cognitive Architectures: Cognitive Architectures: Psychological Constraints as Effective Psychological Constraints as Effective

Heuristics for Designing Intelligent SystemsHeuristics for Designing Intelligent Systems

Thanks to D. Choi, K. Cummings, N. Nejati, S. Rogers, S. Sage, and D. Shapiro for their Thanks to D. Choi, K. Cummings, N. Nejati, S. Rogers, S. Sage, and D. Shapiro for their contributions. This talk reports research. funded by grants from DARPA IPTO and the contributions. This talk reports research. funded by grants from DARPA IPTO and the National Science Foundation, which are not responsible for its contents. National Science Foundation, which are not responsible for its contents.

Newell’s ClaimNewell’s Claim

view cognitive psychology and artificial intelligence as close view cognitive psychology and artificial intelligence as close allies with distinct but related goals; allies with distinct but related goals;

move beyond isolated phenomena and capabilities to develop move beyond isolated phenomena and capabilities to develop complete models of intelligent behavior; complete models of intelligent behavior;

demonstrate our systems’ intelligence on the same range of demonstrate our systems’ intelligence on the same range of domains and tasks as humans can handle. domains and tasks as humans can handle.

In 1973, Allen Newell argued “In 1973, Allen Newell argued “You can’t play twenty questions You can’t play twenty questions with nature and winwith nature and win”. Instead, he proposed that we: ”. Instead, he proposed that we:

Newell was criticizing experimental psychology and looked to AI, Newell was criticizing experimental psychology and looked to AI, with its systems perspective, for the solution. with its systems perspective, for the solution.

The Fragmentation of AI ResearchThe Fragmentation of AI Research

action

perception

reasoning

learning

planninglanguage

……and is not taking us and is not taking us toward human-level toward human-level intelligent systems. intelligent systems.

Unfortunately, AI Unfortunately, AI has changed … has changed …

Psychological Constraints as HeuristicsPsychological Constraints as Heuristics

how the system should represent and organize knowledge; how the system should represent and organize knowledge;

how the system should use that knowledge in performance; how the system should use that knowledge in performance;

how the system should acquire knowledge from experience. how the system should acquire knowledge from experience.

To develop intelligent systems, we must constrain their design, To develop intelligent systems, we must constrain their design, and findings about human behavior can suggest: and findings about human behavior can suggest:

Today, the most interesting research on intelligent systems uses Today, the most interesting research on intelligent systems uses psychological ideas as heuristics in this manner. psychological ideas as heuristics in this manner.

This approach has led to many new insights, and we need more This approach has led to many new insights, and we need more work in this tradition. work in this tradition.

One Approach to Intelligent SystemsOne Approach to Intelligent Systems

action

perception

reasoning

learning

planninglanguage

software engineering / multi-agent systemssoftware engineering / multi-agent systems

Newell’s vision for research on theories of intelligence was that:Newell’s vision for research on theories of intelligence was that:

cognitive systems should make strong theoretical assumptions cognitive systems should make strong theoretical assumptions about the nature of the mind; about the nature of the mind;

theories of intelligence should change only gradually, as new theories of intelligence should change only gradually, as new structures or processes are determined necessary;structures or processes are determined necessary;

later design choices should be constrained heavily by earlier later design choices should be constrained heavily by earlier ones, not made independently. ones, not made independently.

Integration vs. UnificationIntegration vs. Unification

A successful framework is all about mutual constraints, and it A successful framework is all about mutual constraints, and it should provide a should provide a unifiedunified theory of intelligent behavior. theory of intelligent behavior.

He associated these aims with the idea of a He associated these aims with the idea of a cognitive architecturecognitive architecture..

Another Approach to Intelligent SystemsAnother Approach to Intelligent Systems

action

perception

reasoning

learning

planninglanguage

short-termbeliefs and goals

long-term memorystructures

cognitive architecturescognitive architectures

A Constrained Cognitive ArchitectureA Constrained Cognitive Architecture

action

perception

reasoning

learning

planninglanguage

short-termbeliefs and goals

long-term memorystructures

The IThe ICARUSCARUS Architecture Architecture

In this talk I will use one such framework In this talk I will use one such framework I ICARUSCARUS to illustrate to illustrate the influence of psychology on cognitive architectures. the influence of psychology on cognitive architectures.

IICARUSCARUS incorporates a variety of assumptions from psychological incorporates a variety of assumptions from psychological theories; the most basic are that: theories; the most basic are that:

These claims give IThese claims give ICARUSCARUS much in common with other cognitive much in common with other cognitive architectures like ACT-R, Soar, and Prodigy. architectures like ACT-R, Soar, and Prodigy.

1.1. Short-term memories are distinct from long-term stores Short-term memories are distinct from long-term stores

2.2. Memories contain modular elements cast as list structuresMemories contain modular elements cast as list structures

3.3. Long-term structures are accessed through pattern matchingLong-term structures are accessed through pattern matching

4.4. Cognition occurs in retrieval/selection/action cyclesCognition occurs in retrieval/selection/action cycles

5.5. Performance and learning compose elements in memoryPerformance and learning compose elements in memory

A cognitive architecture makes a specific commitment to:A cognitive architecture makes a specific commitment to:

long-term memorieslong-term memories that store knowledge and procedures; that store knowledge and procedures;

short-term memoriesshort-term memories that store beliefs and goals; that store beliefs and goals;

sensori-motor memoriessensori-motor memories that hold percepts and actions. that hold percepts and actions.

Architectural Commitment to MemoriesArchitectural Commitment to Memories

These memories correspond to ones postulated in psychology. These memories correspond to ones postulated in psychology.

For each memory, a cognitive architecture also commits to:For each memory, a cognitive architecture also commits to:

the the representationrepresentation of content in that memory; of content in that memory;

the the organizationorganization of structures within the memory; of structures within the memory;

the the connectionsconnections among structures across memories. among structures across memories.

IICARUSCARUS’ Memories’ Memories

Long-TermLong-TermConceptualConceptual

MemoryMemory

Long-TermLong-TermSkill MemorySkill Memory

Short-TermShort-TermBelief MemoryBelief Memory

Short-TermShort-TermGoal MemoryGoal Memory

EnvironmentEnvironment

PerceptualPerceptualBufferBuffer

MotorMotorBufferBuffer

concepts and skills encode different aspects of knowledge concepts and skills encode different aspects of knowledge that are stored as distinct cognitive structures;that are stored as distinct cognitive structures;

cognition occurs in a physical context, with concepts and cognition occurs in a physical context, with concepts and skills being grounded in perception and action; skills being grounded in perception and action;

many mental structures are relational in nature, in that they many mental structures are relational in nature, in that they describe connections or interactions among objects; describe connections or interactions among objects;

long-term memories have hierarchical organizations that long-term memories have hierarchical organizations that define complex structures in terms of simpler ones; define complex structures in terms of simpler ones;

each element in a short-term memory is an active version of each element in a short-term memory is an active version of some structure in long-term memory. some structure in long-term memory.

Ideas about RepresentationIdeas about Representation

Cognitive psychology makes important representational claims: Cognitive psychology makes important representational claims:

IICARUSCARUS adopts these assumptions about the contents of memory. adopts these assumptions about the contents of memory.

Representing Long-Term StructuresRepresenting Long-Term Structures

Conceptual clauses:Conceptual clauses: A set of relational inference rules with A set of relational inference rules with perceived objects or defined concepts in their antecedents;perceived objects or defined concepts in their antecedents;

Skill clauses:Skill clauses: A set of executable skills that specify: A set of executable skills that specify: a head that indicates a goal the skill achieves;a head that indicates a goal the skill achieves; a single (typically defined) precondition;a single (typically defined) precondition; a set of ordered subgoals or actions for achieving the goal. a set of ordered subgoals or actions for achieving the goal.

These define a specialized class of These define a specialized class of hierarchical task networkshierarchical task networks in in which each task corresponds to a goal concept. which each task corresponds to a goal concept.

IICARUSCARUS encodes two forms of general long-term knowledge: encodes two forms of general long-term knowledge:

IICARUSCARUS Concepts for In-City Driving Concepts for In-City Driving

((in-rightmost-lane ?self ?clane)((in-rightmost-lane ?self ?clane) :percepts (:percepts ( (self ?self) (segment ?seg) (self ?self) (segment ?seg)

(line ?clane segment ?seg))(line ?clane segment ?seg)) :relations ((driving-well-in-segment ?self ?seg ?clane):relations ((driving-well-in-segment ?self ?seg ?clane)

(last-lane ?clane) (last-lane ?clane) (not (lane-to-right ?clane ?anylane))))(not (lane-to-right ?clane ?anylane))))

((driving-well-in-segment ?self ?seg ?lane)((driving-well-in-segment ?self ?seg ?lane) :percepts ((self ?self) (segment ?seg) (line ?lane segment ?seg)):percepts ((self ?self) (segment ?seg) (line ?lane segment ?seg)) :relations ((in-segment ?self ?seg) (in-lane ?self ?lane):relations ((in-segment ?self ?seg) (in-lane ?self ?lane)

(aligned-with-lane-in-segment ?self ?seg ?lane)(aligned-with-lane-in-segment ?self ?seg ?lane)(centered-in-lane ?self ?seg ?lane)(centered-in-lane ?self ?seg ?lane)(steering-wheel-straight ?self)))(steering-wheel-straight ?self)))

((in-lane ?self ?lane)((in-lane ?self ?lane) :percepts (:percepts ( (self ?self segment ?seg) (line ?lane segment ?seg dist ?dist))(self ?self segment ?seg) (line ?lane segment ?seg dist ?dist)) :tests (:tests ( (> ?dist -10) (<= ?dist 0)))(> ?dist -10) (<= ?dist 0)))

((in-segment ?self ?seg)((in-segment ?self ?seg) :percepts (:percepts ( (self ?self segment ?seg) (segment ?seg)))(self ?self segment ?seg) (segment ?seg)))

((in-rightmost-lane ?self ?line)((in-rightmost-lane ?self ?line) :percepts:percepts ((self ?self) (line ?line))((self ?self) (line ?line)) :start ((last-lane ?line)):start ((last-lane ?line)) :subgoals ((driving-well-in-segment ?self ?seg ?line))) :subgoals ((driving-well-in-segment ?self ?seg ?line)))

((driving-well-in-segment ?self ?seg ?line)((driving-well-in-segment ?self ?seg ?line) :percepts:percepts ((segment ?seg) (line ?line) (self ?self))((segment ?seg) (line ?line) (self ?self)) :start ((steering-wheel-straight ?self)):start ((steering-wheel-straight ?self)) :subgoals ((in-segment ?self ?seg):subgoals ((in-segment ?self ?seg)

(centered-in-lane ?self ?seg ?line)(centered-in-lane ?self ?seg ?line)(aligned-with-lane-in-segment ?self ?seg ?line)(aligned-with-lane-in-segment ?self ?seg ?line)(steering-wheel-straight ?self)))(steering-wheel-straight ?self)))

((in-segment ?self ?endsg)((in-segment ?self ?endsg) :percepts:percepts ((self ?self speed ?speed) (intersection ?int cross ?cross)((self ?self speed ?speed) (intersection ?int cross ?cross)

(segment ?endsg street ?cross angle ?angle))(segment ?endsg street ?cross angle ?angle)) :start ((in-intersection-for-right-turn ?self ?int)):start ((in-intersection-for-right-turn ?self ?int)) :actions:actions ((((steer 1)))steer 1)))

IICARUSCARUS Skills for In-City Driving Skills for In-City Driving

Representing Short-Term Beliefs/GoalsRepresenting Short-Term Beliefs/Goals

(current-street me A)(current-street me A) (current-segment me g550)(current-segment me g550)(lane-to-right g599 g601)(lane-to-right g599 g601) (first-lane g599)(first-lane g599)(last-lane g599)(last-lane g599) (last-lane g601)(last-lane g601)(at-speed-for-u-turn me)(at-speed-for-u-turn me) (slow-for-right-turn me)(slow-for-right-turn me)(steering-wheel-not-straight me)(steering-wheel-not-straight me) (centered-in-lane me g550 g599)(centered-in-lane me g550 g599)(in-lane me g599)(in-lane me g599) (in-segment me g550)(in-segment me g550)(on-right-side-in-segment me)(on-right-side-in-segment me) (intersection-behind g550 g522)(intersection-behind g550 g522)(building-on-left g288)(building-on-left g288) (building-on-left g425)(building-on-left g425)(building-on-left g427)(building-on-left g427) (building-on-left g429)(building-on-left g429)(building-on-left g431)(building-on-left g431) (building-on-left g433)(building-on-left g433)(building-on-right g287)(building-on-right g287) (building-on-right g279)(building-on-right g279)(increasing-direction me)(increasing-direction me) (buildings-on-right g287 g279)(buildings-on-right g287 g279)

Encoding Perceived ObjectsEncoding Perceived Objects

(self me speed 5 angle-of-road -0.5 steering-wheel-angle -0.1)(self me speed 5 angle-of-road -0.5 steering-wheel-angle -0.1)(segment g562 street 1 dist -5.0 latdist 15.0)(segment g562 street 1 dist -5.0 latdist 15.0)(line g564 length 100.0 width 0.5 dist 35.0 angle 1.1 color white segment g562)(line g564 length 100.0 width 0.5 dist 35.0 angle 1.1 color white segment g562)(line g565 length 100.0 width 0.5 dist 15.0 angle 1.1 color white segment g562)(line g565 length 100.0 width 0.5 dist 15.0 angle 1.1 color white segment g562)(line g563 length 100.0 width 0.5 dist 25.0 angle 1.1 color yellow segment g562)(line g563 length 100.0 width 0.5 dist 25.0 angle 1.1 color yellow segment g562)(segment g550 street A dist oor latdist nil)(segment g550 street A dist oor latdist nil)(line g600 length 100.0 width 0.5 dist -15.0 angle -0.5 color white segment g550)(line g600 length 100.0 width 0.5 dist -15.0 angle -0.5 color white segment g550)(line g601 length 100.0 width 0.5 dist 5.0 angle -0.5 color white segment g550)(line g601 length 100.0 width 0.5 dist 5.0 angle -0.5 color white segment g550)(line g599 length 100.0 width 0.5 dist -5.0 angle -0.5 color yellow segment g550)(line g599 length 100.0 width 0.5 dist -5.0 angle -0.5 color yellow segment g550)(intersection g522 street A cross 1 dist -5.0 latdist nil)(intersection g522 street A cross 1 dist -5.0 latdist nil)(building g431 address 99 street A c1dist 38.2 c1angle -1.4 c2dist 57.4 c2angle -(building g431 address 99 street A c1dist 38.2 c1angle -1.4 c2dist 57.4 c2angle -1.0)1.0)(building g425 address 25 street A c1dist 37.8 c1angle -2.8 c2dist 56.9 c2angle -(building g425 address 25 street A c1dist 37.8 c1angle -2.8 c2dist 56.9 c2angle -3.1)3.1)(building g389 address 49 street 1 c1dist 49.2 c1angle 2.7 c2dist 53.0 c2angle 2.2)(building g389 address 49 street 1 c1dist 49.2 c1angle 2.7 c2dist 53.0 c2angle 2.2)(sidewalk g471 dist 15.0 angle -0.5)(sidewalk g471 dist 15.0 angle -0.5)(sidewalk g474 dist 5.0 angle 1.07)(sidewalk g474 dist 5.0 angle 1.07)(sidewalk g469 dist -25.0 angle -0.5)(sidewalk g469 dist -25.0 angle -0.5)(sidewalk g470 dist 45.0 angle 1.07)(sidewalk g470 dist 45.0 angle 1.07)(stoplight g538 vcolor green hcolor red))(stoplight g538 vcolor green hcolor red))

Hierarchical Structure of Long-Term MemoryHierarchical Structure of Long-Term Memory

conceptsconcepts

skillsskills

Each concept is defined in terms of other concepts and/or percepts.Each concept is defined in terms of other concepts and/or percepts.

Each skill is defined in terms of other skills, concepts, and percepts.Each skill is defined in terms of other skills, concepts, and percepts.

IICARUS CARUS organizes both concepts and skills in a hierarchical organizes both concepts and skills in a hierarchical manner.manner.

Hierarchical Structure of Long-Term MemoryHierarchical Structure of Long-Term Memory

conceptsconcepts

skillsskills

For example, the skill highlighted here refers directly to For example, the skill highlighted here refers directly to the highlighted concepts.the highlighted concepts.

IICARUS CARUS interleaves its long-term memories for concepts and skills.interleaves its long-term memories for concepts and skills.

In addition, a cognitive architecture makes commitments about:In addition, a cognitive architecture makes commitments about:

performance processesperformance processes for: for:

retrieval, matching, and selectionretrieval, matching, and selection

inference and problem solvinginference and problem solving

perception and motor controlperception and motor control

learning processeslearning processes that: that:

generate new long-term knowledge structuresgenerate new long-term knowledge structures

refine and modulate existing structuresrefine and modulate existing structures

Architectural Commitment to ProcessesArchitectural Commitment to Processes

In most cognitive architectures, performance and learning are In most cognitive architectures, performance and learning are tightly intertwined, again reflecting influence from psychology. tightly intertwined, again reflecting influence from psychology.

humans can handle multiple goals with different priorities, humans can handle multiple goals with different priorities, which can interrupt tasks to which attention returns later; which can interrupt tasks to which attention returns later;

conceptual inference, which typically occurs rapidly and conceptual inference, which typically occurs rapidly and unconsciously, is more basic than problem solving; unconsciously, is more basic than problem solving;

humans often resort to means-ends analysis to solve novel, humans often resort to means-ends analysis to solve novel, unfamiliar problems; unfamiliar problems;

mental problem solving requires greater cognitive resources mental problem solving requires greater cognitive resources than execution of automatized skills; than execution of automatized skills;

problem solving often occurs in a physical context, with problem solving often occurs in a physical context, with mental processing being interleaved with execution.mental processing being interleaved with execution.

Ideas about PerformanceIdeas about Performance

IICARUSCARUS embodies these ideas in its performance mechanisms. embodies these ideas in its performance mechanisms.

Cognitive psychology makes clear claims about performance: Cognitive psychology makes clear claims about performance:

IICARUSCARUS’ Functional Processes’ Functional Processes

Long-TermLong-TermConceptualConceptual

MemoryMemory

Short-TermShort-TermBeliefBelief

MemoryMemory

Short-TermShort-TermGoal MemoryGoal Memory

ConceptualConceptualInferenceInference

SkillSkillExecutionExecution

PerceptionPerception

EnvironmentEnvironment

PerceptualPerceptualBufferBuffer

Problem SolvingProblem SolvingSkill LearningSkill Learning

MotorMotorBufferBuffer

Skill RetrievalSkill Retrievaland Selectionand Selection

Long-TermLong-TermSkill MemorySkill Memory

IICARUSCARUS’ Inference-Execution Cycle’ Inference-Execution Cycle

1.1. places descriptions of sensed objects in the perceptual buffer;places descriptions of sensed objects in the perceptual buffer;

2.2. infers instances of concepts implied by the current situation;infers instances of concepts implied by the current situation;

3.3. finds paths through the skill hierarchy from top-level goals;finds paths through the skill hierarchy from top-level goals;

4.4. selects one or more applicable skill paths for execution; selects one or more applicable skill paths for execution;

5.5. invokes the actions associated with each selected path. invokes the actions associated with each selected path.

On each successive execution cycle, the IOn each successive execution cycle, the ICARUSCARUS architecture: architecture:

This framework is very similar to the recognize-act cycle used in This framework is very similar to the recognize-act cycle used in production systems, which in turn borrowed from S-R theories. production systems, which in turn borrowed from S-R theories.

Basic IBasic ICARUSCARUS Processes Processes

conceptsconcepts

skillsskills

Concepts are matched bottom up, starting from percepts.Concepts are matched bottom up, starting from percepts.

Skill paths are matched top down, starting from Skill paths are matched top down, starting from intentions.intentions.

IICARUS CARUS matches patterns to recognize concepts and select skills.matches patterns to recognize concepts and select skills.

IICARUSCARUS Interleaves Execution and Problem Solving Interleaves Execution and Problem Solving

Executed plan

Problem

??

Skill Hierarchy

Primitive Skills

ReactiveExecution

impasse?

ProblemSolving

yesyes

nono

This organization reflects the psychological distinction between automatized This organization reflects the psychological distinction between automatized and controlled behavior. and controlled behavior.

Interleaving Reactive Control and Problem SolvingInterleaving Reactive Control and Problem Solving

Solve(G)Solve(G) Push the goal literal G onto the empty goal stack GS. Push the goal literal G onto the empty goal stack GS. On each cycle, On each cycle, If the top goal G of the goal stack GS is satisfied, If the top goal G of the goal stack GS is satisfied, Then pop GS. Then pop GS. Else if the goal stack GS does not exceed the depth limit, Else if the goal stack GS does not exceed the depth limit, Let S be the skill instances whose heads unify with G. Let S be the skill instances whose heads unify with G. If any applicable skill paths start from an instance in S, If any applicable skill paths start from an instance in S, Then select one of these paths and execute it. Then select one of these paths and execute it. Else let M be the set of primitive skill instances that have not already failed in which G is an effect. Else let M be the set of primitive skill instances that have not already failed in which G is an effect. If the set M is nonempty, If the set M is nonempty, Then select a skill instance Q from M. Then select a skill instance Q from M.

Push the start condition C of Q onto goal stack GS. Push the start condition C of Q onto goal stack GS. Else if G is a complex concept with the unsatisfied subconcepts H and with satisfied subconcepts F,Else if G is a complex concept with the unsatisfied subconcepts H and with satisfied subconcepts F, Then if there is a subconcept I in H that has not yet failed, Then if there is a subconcept I in H that has not yet failed, Then push I onto the goal stack GS. Then push I onto the goal stack GS. Else pop G from the goal stack GS and store information about failure with G's parent. Else pop G from the goal stack GS and store information about failure with G's parent. Else pop G from the goal stack GS. Else pop G from the goal stack GS. Store information about failure with G's parent. Store information about failure with G's parent.

This is traditional means-ends analysis, with three exceptions: (1) conjunctive This is traditional means-ends analysis, with three exceptions: (1) conjunctive goals must be defined concepts; (2) chaining occurs over both skills/operators goals must be defined concepts; (2) chaining occurs over both skills/operators and concepts/axioms; and (3) selected skills are executed whenever applicable. and concepts/axioms; and (3) selected skills are executed whenever applicable.

A Successful Problem-Solving TraceA Successful Problem-Solving Trace

(ontable A T)

(on B A)

(on C B)

(hand-empty)

(clear C)

(unst. C B) (unstack C B) (clear B)

(putdown C T)

(unst. B A) (unstack B A) (clear A)

(holding C) (hand-empty)

(holding B)

A

B

C B

A C

initial stateinitial state

goalgoal

efforts to overcome impasses during problem solving can lead efforts to overcome impasses during problem solving can lead to the acquisition of new skills; to the acquisition of new skills;

learning can transform backward-chaining heuristic search learning can transform backward-chaining heuristic search into more informed forward-chaining behavior; into more informed forward-chaining behavior;

learning is incremental and interleaved with performance; learning is incremental and interleaved with performance;

structural learning involves monotonic addition of symbolic structural learning involves monotonic addition of symbolic elements to long-term memory; elements to long-term memory;

transfer to new tasks depends on the amount of structure transfer to new tasks depends on the amount of structure shared with previously mastered tasks. shared with previously mastered tasks.

Claims about LearningClaims about Learning

Cognitive psychology has also developed ideas about learning: Cognitive psychology has also developed ideas about learning:

IICARUSCARUS incorporates these assumptions into its basic operation. incorporates these assumptions into its basic operation.

IICARUSCARUS Learns Skills from Problem Solving Learns Skills from Problem Solving

Executed plan

Problem

??

Skill Hierarchy

Primitive Skills

ReactiveExecution

impasse?

ProblemSolving

yesyes

nono

SkillLearning

IICARUSCARUS’ Constraints on Skill Learning’ Constraints on Skill Learning

What determines the hierarchical structure of skill memory?What determines the hierarchical structure of skill memory?

The structure emerges the subproblems that arise during The structure emerges the subproblems that arise during problem solving, which, because operator conditions and problem solving, which, because operator conditions and goals are single literals, form a semilattice.goals are single literals, form a semilattice.

What determines the heads of the learned clauses/methods?What determines the heads of the learned clauses/methods?

The head of a learned clause is the goal literal that the The head of a learned clause is the goal literal that the planner achieved for the subproblem that produced it.planner achieved for the subproblem that produced it.

What are the conditions on the learned clauses/methods? What are the conditions on the learned clauses/methods?

If the subproblem involved skill chaining, they are the If the subproblem involved skill chaining, they are the conditions of the first subskill clause.conditions of the first subskill clause.

If the subproblem involved concept chaining, they are the If the subproblem involved concept chaining, they are the subconcepts that held at the subproblem’s outset. subconcepts that held at the subproblem’s outset.

(ontable A T)

(on B A)

(on C B)

(hand-empty)

(clear C)

(unst. C B) (unstack C B) (clear B)

(putdown C T)

(unst. B A) (unstack B A) (clear A)

(holding C) (hand-empty)

(holding B)

A

B

C B

A C

1

skill chainingskill chaining

Constructing Skills from a TraceConstructing Skills from a Trace

(ontable A T)

(on B A)

(on C B)

(hand-empty)

(clear C)

(unst. C B) (unstack C B) (clear B)

(putdown C T)

(unst. B A) (unstack B A) (clear A)

(holding C) (hand-empty)

(holding B)

A

B

C B

A C

1

2

skill chainingskill chaining

Constructing Skills from a TraceConstructing Skills from a Trace

(ontable A T)

(on B A)

(on C B)

(hand-empty)

(clear C)

(unst. C B) (unstack C B) (clear B)

(putdown C T)

(unst. B A) (unstack B A) (clear A)

(holding C) (hand-empty)

(holding B)

A

B

C B

A C

1 3

2

concept chainingconcept chaining

Constructing Skills from a TraceConstructing Skills from a Trace

(ontable A T)

(on B A)

(on C B)

(hand-empty)

(clear C)

(unst. C B) (unstack C B) (clear B)

(putdown C T)

(unst. B A) (unstack B A) (clear A)

(holding C) (hand-empty)

(holding B)

A

B

C B

A C

1 3

2

4

skill chainingskill chaining

Constructing Skills from a TraceConstructing Skills from a Trace

Learned Skills in the Blocks WorldLearned Skills in the Blocks World

(clear (?C)(clear (?C) :percepts :percepts ((block ?D) (block ?C))((block ?D) (block ?C)) :start :start ((unstackable ?D ?C))((unstackable ?D ?C)) :skills :skills ((unstack ?D ?C)))((unstack ?D ?C)))

(clear (?B)(clear (?B) :percepts :percepts ((block ?C) (block ?B))((block ?C) (block ?B)) :start :start ((on ?C ?B) (hand-empty))((on ?C ?B) (hand-empty)) :skills :skills ((unstackable ?C ?B) (unstack ?C ?B)))((unstackable ?C ?B) (unstack ?C ?B)))

(unstackable (?C ?B)(unstackable (?C ?B) :percepts :percepts ((block ?B) (block ?C))((block ?B) (block ?C)) :start :start ((on ?C ?B) (hand-empty))((on ?C ?B) (hand-empty)) :skills :skills ((clear ?C) (hand-empty)))((clear ?C) (hand-empty)))

(hand-empty ( )(hand-empty ( ) :percepts :percepts ((block ?D) (table ?T1))((block ?D) (table ?T1)) :start :start ((putdownable ?D ?T1))((putdownable ?D ?T1)) :skills :skills ((putdown ?D ?T1)))((putdown ?D ?T1)))

Hierarchical skills are Hierarchical skills are generalized traces of generalized traces of successful means-ends successful means-ends problem solvingproblem solving

Cumulative Curves for Two DomainsCumulative Curves for Two Domains

Blocks WorldBlocks World FreeCellFreeCell

Constraints from cognitive psychology can produce very effective Constraints from cognitive psychology can produce very effective approaches to speedup learning. approaches to speedup learning.

Learning Skills for In-City DrivingLearning Skills for In-City Driving

We have trained IWe have trained ICARUSCARUS to drive in a simulated to drive in a simulated in-city environment. in-city environment.

We provide the system We provide the system with tasks of increasing with tasks of increasing complexity. complexity.

Learning transforms the Learning transforms the problem-solving traces problem-solving traces into hierarchical skills. into hierarchical skills.

The agent uses these The agent uses these skills to change lanes, skills to change lanes, turn, and park using turn, and park using only reactive control. only reactive control.

The architecture also supports the transfer of knowledge in that:The architecture also supports the transfer of knowledge in that:

skills acquired later can build on those learned earlier; skills acquired later can build on those learned earlier;

skill clauses are indexed by the goals they achieve. skill clauses are indexed by the goals they achieve.

Transfer of Skills in ITransfer of Skills in ICARUSCARUS

Experimental studies suggest that these lead to effective transfer. Experimental studies suggest that these lead to effective transfer.

Cognitive architectures come with a programming language that:Cognitive architectures come with a programming language that:

includes a syntax linked to its representational assumptionsincludes a syntax linked to its representational assumptions

inputs long-term knowledge and initial short-term elementsinputs long-term knowledge and initial short-term elements

provides an interpreter that runs the specified programprovides an interpreter that runs the specified program

incorporates tracing facilities to inspect system behaviorincorporates tracing facilities to inspect system behavior

Architectures as Programming LanguagesArchitectures as Programming Languages

Such programming languages ease construction and debugging Such programming languages ease construction and debugging of knowledge-based systems.of knowledge-based systems.

Thus, ideas from psychology can support efficient development Thus, ideas from psychology can support efficient development of software for intelligent systems. of software for intelligent systems.

The programming language associated with IThe programming language associated with ICARUSCARUS comes with: comes with:

a syntax for concepts, skills, beliefs, and perceptsa syntax for concepts, skills, beliefs, and percepts

the ability to load and parse such programsthe ability to load and parse such programs

an interpreter for inference, execution, planning, and learningan interpreter for inference, execution, planning, and learning

a trace package that displays system behavior over timea trace package that displays system behavior over time

Programming in IProgramming in ICARUSCARUS

We have used this language to develop adaptive intelligent We have used this language to develop adaptive intelligent agents in a variety of domains. agents in a variety of domains.

An IAn ICARUSCARUS Agent for Urban Combat Agent for Urban Combat

Intellectual PrecursorsIntellectual Precursors

characteristics of human memory characteristics of human memory

human problem solving human problem solving

logical reasoning and inferencelogical reasoning and inference

acquisition of cognitive skillsacquisition of cognitive skills

cognitive architectures (especially ACT, Soar, and Prodigy)cognitive architectures (especially ACT, Soar, and Prodigy)

IICARUSCARUS’ design has been influenced by earlier research on:’ design has been influenced by earlier research on:

We can debate whether IWe can debate whether ICARUSCARUS provides an adequate theory of provides an adequate theory of human behavior, but not its debt to cognitive psychology. human behavior, but not its debt to cognitive psychology.

Directions for Future ResearchDirections for Future Research

progressive deepening in forward-chaining search;progressive deepening in forward-chaining search;

graded nature of categories and category learning;graded nature of categories and category learning;

model-based character of human reasoning; model-based character of human reasoning;

persistent but limited nature of short-term memories; persistent but limited nature of short-term memories;

creating perceptual chunks to reduce these limitations;creating perceptual chunks to reduce these limitations;

storing and retreiving episodic memory traces.storing and retreiving episodic memory traces.

Future work on IFuture work on ICARUSCARUS should incorporate other ideas about: should incorporate other ideas about:

These additions will increase further IThese additions will increase further ICARUSCARUS’ debt to psychology. ’ debt to psychology.

Contributions of IContributions of ICARUSCARUS

includes separate memories for concepts and skills;includes separate memories for concepts and skills;

organizes both memories in a hierarchical fashion;organizes both memories in a hierarchical fashion;

modulates reactive execution with goal seeking; modulates reactive execution with goal seeking;

augments routine behavior with problem solving; andaugments routine behavior with problem solving; and

learns hierarchical skills in a cumulative manner.learns hierarchical skills in a cumulative manner.

IICARUSCARUS is a cognitive architecture for physical agents that: is a cognitive architecture for physical agents that:

These ideas have their roots in cognitive psychology, but they These ideas have their roots in cognitive psychology, but they are also effective in building flexible intelligent agents. are also effective in building flexible intelligent agents.

For more information about the IFor more information about the ICARUSCARUS architecture, see: architecture, see:

http://cll.stanford.edu/research/ongoing/icarus/

End of PresentationEnd of Presentation