the sneps approach to cognitive robotics

82
S.C. Shapiro cse@buffalo The SNePS Approach to Cognitive Robotics Stuart C. Shapiro Department of Computer Science and Engineering and Center for Cognitive Science University at Buffalo [email protected]

Upload: trynt

Post on 25-Feb-2016

32 views

Category:

Documents


0 download

DESCRIPTION

The SNePS Approach to Cognitive Robotics. Stuart C. Shapiro Department of Computer Science and Engineering and Center for Cognitive Science University at Buffalo [email protected]. Outline. Introduction Intensional Representation & Propositions SNePS Connectives and Quantifiers - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

The SNePS Approach to Cognitive Robotics

Stuart C. Shapiro

Department of Computer Science and Engineeringand Center for Cognitive Science

University at Buffalo

[email protected]

Page 2: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary

Page 3: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Goal• A computational cognitive agent that can:

– Understand and communicate in English; – Discuss specific, generic, and “rule-like” information;– Reason;– Discuss acts and plans;– Sense;– Act;– Remember and report what it has sensed and done.

Page 4: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Embodied Cassie

• A computational cognitive agent– Embodied in hardware– or Software-Simulated– Based on SNePS and GLAIR.

Page 5: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

SNePS• Knowledge Representation and Reasoning

– Propositions as Terms• SNIP: SNePS Inference Package

– Specialized connectives and quantifiers• SNeBR: SNePS Belief Revision• SNeRE: SNePS Rational Engine• Interface Languages

– SNePSUL: Lisp-Like– SNePSLOG: Logic-Like– GATN for Fragments of English.

Page 6: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

GLAIR Architecture

Knowledge Level

Perceptuo-Motor Level

Sensory-Actuator Level

NL

VisionSonar

Motion

Proprioception

Grounded Layered Architecture with Integrated Reasoning

SNePS

Page 7: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Interaction with CassieEnglish

(Statement, Question, Command)(Current) Set of Beliefs

[SNePS]

(Updated) Setof Beliefs[SNePS]

Actions[SNeRE]

(New Belief)[SNePS]

English sentence expressingnew belief answering question reporting actions

Answer[SNIP]

GATN Parser

GATN Generator

ReasoningClarification DialogueLooking in World

Reasoning

Page 8: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Cassie, the BlocksWorld Robot

Page 9: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Cassie, the FEVAHR

Page 10: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

FEVAHR/Cassie in the Lab

Page 11: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

FEVAHRWorld Simulation

Page 12: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

UXO Remediation Cassie

CassieCorner flag

NonUXO object

Corner flag

UXO

Batterymeter

Corner flag

Drop-off zone

Field

Safe zoneRechargingStation

Page 13: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Crystal Space Environment

Page 14: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary

Page 15: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Entities, Terms, Symbols, Objects

• Cassie’s mental entity: a person named Bill• SNePS term: B5• Object in world:

Page 16: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Intensional Representation

Intensional entities are distincteven if coreferential.

“The morning star is the evening star.”“George IV wondered if Scott was the author of

Waverly.”

Page 17: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

McCarthy’s Telephone Number Problem

Mary's telephone number is Mike's telephone number. I understand that Mike's telephone number is Mary's

telephone number. Pat knew Mike's telephone number. I understand that Pat knew Mike's telephone number. Pat dialed Mike's telephone number. I understand that Pat dialed Mike's telephone number.

Page 18: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Answering the Telephone Number Problem

Did Pat dial Mary's telephone number? Yes, Pat dialed Mary's telephone number. Did Pat know Mary's telephone number? I don't know.

Page 19: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Representing Propositions

Propositions must befirst-class entities of the domain

Represented by terms.

Page 20: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Discussing Propositions

That Bill is sweet is Mary's favorite proposition.I understand that Mary's favorite proposition is

that Bill is sweet.Mike believes Mary's favorite proposition.I understand that Mike believes that Bill is sweet.

Page 21: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary

Page 22: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Logic for NLU &Commonsense Reasoning

Either Pat is a man or Pat is a woman or Pat is a robot. I understand that Pat is a robot or Pat is a woman

or Pat is a man. Pat is a woman. I understand that Pat is a woman. What is Pat? Pat is a woman and Pat is not a robot

and Pat is not a man.

Page 23: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Representation in FOPL?

Man(Pat) Woman(Pat) Robot(Pat)

Page 24: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Representation in FOPL?

Man(Pat) Woman(Pat) Robot(Pat)but don’t want inclusive or

Page 25: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Representation in FOPL?

Man(Pat) Woman(Pat) Robot(Pat)but don’t want inclusive or

Man(Pat) Woman(Pat) Robot(Pat)+ +T T T

F

TSo don’t want exclusive or either

Page 26: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

andor

andor(i, j){P1, ..., Pn}

True iff at least i, and at most j of the Pi are True

Page 27: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

thresh

thresh(i, j){P1, ..., Pn}

True iff either fewer than i, or more than j of the Pi are True

Note: thresh(i, j) ~andor(i, j)

Page 28: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

or-entailment

{P1, ..., Pn} v=> {Q1, ..., Qn}

True iff for all i, j Pi Qj

Page 29: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

and-entailment

{P1, ..., Pn} &=> {Q1, ..., Qn}

True iff for all jP1 &…& Pn Qj

Page 30: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Numerical entailment

{P1, ..., Pn} i=> {Q1, ..., Qn}

True iff for all jandor(i, n){P1, …, Pn } Qj

Page 31: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Universal Quantifier

all(ū)({R1(ū),..., Rn(ū)} &=> {C1(ū),..., Cm(ū)})

Every ā that satisfiesR1(ū)&…& Rn(ū)

also satisfiesC1(ū),..., Cm(ū)})

Page 32: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Numerical Quantifiers

nexists(i,j,k)(x) ({P1(x),..., Pn(x)}: {Q(x)})}

There are k individuals that satisfyP1(x) ... Pn(x)

and, of them, at least i and at most j also satisfyQ(x)

Page 33: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary

Page 34: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

MENTAL ACTS

• Believe(proposition)• Disbelieve(proposition)

Page 35: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Act Selection

• Do-One({act1 ... actn})

• Snif(if(condition, act), ... if(condition, act) [else(act)])

Page 36: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Act Iteration• Do-All({act1 ... actn})• Sniterate(if(condition, act), ... if(condition, act), [else(act)])• Snsequence(act1, ..., actn)• Cascade(act1, ..., actn)• P-Do-All({act1, ..., act2})

Page 37: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Entity Iteration

WithSome(var, suchthat, do, [else])WithAll(var, suchthat, do, [else])WithSome+(var, suchthat, do, [else])WithNew(vars, thatare, suchthat, do, [else])

Page 38: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Proposition/Act Transformers

• Achieve(proposition)• ActPlan(act, plan)• GoalPlan(proposition, act)• Precondition(act, proposition)• Effect(act, proposition)• WhenDo(proposition, act)• WheneverDo(proposition, act)• IfDo(proposition, act)

Page 39: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary

Page 40: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Conditional Plans If a block is on a support then a plan to achieve

that the support is clear is to pick up the block and then put the block on the table.

all(x, y) ({Block(x), Support(y), On(x, y)} &=> {GoalPlan(Clear(y), Snsequence(Pickup(x), Put(x, Table)))})STRIPS-like representation: No times

Page 41: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Use of Conditional Plan

GoalPlan(Clear(B), Snsequence(Pickup(A), Put(A, Table)))Remember (cache) derived propositions.

Page 42: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Use of Conditional Plan

GoalPlan(Clear(B), Snsequence(Pickup(A), Put(A, Table)))???SNeBR to the rescue!

Page 43: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

A FEVAHR Acting Ruleall(p, o) ({Agent(p), Thing(o)}

&=> {Precondition(Follow(p, o), Near(p, o)),

GoalPlan(Near(p, o), Goto(p, o)),

Precondition(Goto(p, o), Lookat(p, o)),

ActPlan(Lookat(p, o), Find(p, o))})Uses a temporal model.

Page 44: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Acting According to the Rule

Page 45: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Acting According to the Rule

I found a red robot.I am looking at a red robot.

Follow a red robot.

Page 46: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Acting According to the Rule

I went to a red robot.I am near a red robot.I am following a red robot.

I found a red robot.I am looking at a red robot.

Follow a red robot.

Page 47: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

A Plan for Blowing up UXOsall(a)(Agent(a) =>ActPlan(Blowup(a, UXOs), Act(a,Cascade(SearchforUxo(a),WithSome+(obj, Near(a, obj), WithNew({ch ex}, {Charge(ch), Explosion(ex)}, Possess(a, ch), Cascade(Place(a, ch, obj), Hide(a), Waitfor(a, ex), SearchforUxo(a))), goto(a, SafeZone))))))

Page 48: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary

Page 49: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Representation and Use of Indexicals

• Words whose meanings are determined by occasion of use

• E.g. I, you, now, then, here, there• Deictic Center <*I, *YOU, *NOW>• *I: SNePS term representing Cassie• *YOU: person Cassie is talking with• *NOW: current time.

Page 50: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Analysis of Indexicals(in input)

• First person pronouns: *YOU• Second person pronouns: *I• “here”: location of *YOU• Present/Past relative to *NOW.

Page 51: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Generation of Indexicals

• *I: First person pronouns• *YOU: Second person pronouns• *NOW: used to determine tense and aspect.

Page 52: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Come here.

Use of Indexicals 1

Page 53: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Come here.I came to you, Stu.I am near you.

Use of Indexicals 2

Page 54: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Who am I?Your name is ‘Stu’and you are a person.Who have you talked to?I am talking to you.Talk to Bill.I am talking to you, Bill.Come here.

Use of Indexicals 3

Page 55: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Come here.

I found you.I am looking at you.

Use of Indexicals 4

Page 56: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Come here.

I came to you.I am near you.

I found you.I am looking at you.

Use of Indexicals 5

Page 57: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Who am I?

I talked to Stuand I am talking to you.

Your name is ‘Bill’and you are a person.Who are you?I am the FEVAHRand my name is ‘Cassie’.Who have you talked to?

Use of Indexicals 6

Page 58: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary

Page 59: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

A Personal Sense of Time

• *NOW contains SNePS term representing current time.

• *NOW moves when Cassie acts or perceives a change of state.

Page 60: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

B6

Representation of Time

find

lex

action objectB1

!

agentact

event time

NOW

!!before after before after

?????????????

I

Page 61: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Movement of Timet1 t2!before after t3!before after

NOW NOW NOW

Page 62: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Performing a Punctual Actt1 t3!before after

NOW NOW

t2!before after

!

time

event

Page 63: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Performing a Durative Actt1

NOW

!before after t2

!

time

event

NOW

t3!

supintsubint

Page 64: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

The Pacemaker

• PML process periodically increments variable COUNT.

• *COUNT = some PML integer.• Reset to 0 when NOW moves.• Provides bodily “feel” of passing time.

Page 65: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Quantizing Time

Cannot conceptualize fine distinctions in time intervals.

So quantize, e.g. into half orders of magnitude (Hobbs, 2000).

Page 66: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Movement of Time with Pacemaker

NOW COUNT nhom

0

KL

PML

t1 t2q

!before after

time duration!

Page 67: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

The Problem of the Fleeting Now

How can you reason about “now”if it never stands still?

Page 68: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Fleeting Now Example 1

12:15:00: “Is John having lunch now?”12:15:02: Agent walks to John’s office.12:17:00: Agent sees John at his desk, eating.12:19:00: Agent reports “yes”.

Appropriate granularity.

Page 69: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Fleeting Now Example 2

12:15:00: “Is John having lunch now?”Agent knows John is at home without a phone.Agent contemplates driving to John’s home.Don’t bother---inappropriate granularity.

Page 70: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

The Vagueness of “now”I’m now giving a talk.I’m now teaching a course.I’m now visiting Toronto.I’m now living in Buffalo.The agent is now walking to John’s office.The agent is now seeing if John is eating lunch.

Multiple now’s at different granularities.

Page 71: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

NOW-MTF

NOWSemi-lattice of times, all of which contain *NOW,any of which could be meant by “now”Finite---only conceptualized times of conceptualized states

Maximal Temporal Frame based on *NOW

Page 72: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Moving NOW with MTF

NOW

Page 73: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Typical Durations“If the walk light is on now, cross the street.”

Relevant duration is typical duration of walk lights.

“Is John having lunch now?”Relevant duration is typical duration of lunch.

Use quantized typical durations when updating NOW-MTFs.

Page 74: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Using Appropriate Granularity

NOW

Lunch time

Lunch? Lunch!

Yes!

Page 75: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary

Page 76: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Goal

• A computational cognitive agent/robot• That can communicate in natural

language.

Page 77: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Intensional Representation& Propositions

• SNePS terms represent mental entities.• May assert that two entities are coreferential.• Relations/acts may be declared transparent.• Propositions are first-class entities.

Page 78: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

SNePS Connectives and Quantifiers

• Designed logical connectivesand rules of inference

More appropriate for NLU and Commonsense reasoningthan in standard FOPC.

Page 79: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

SNeRE Acting Constructs

• Separate, but CoordinatedSyntax and SemanticsFor Acting and for Reasoning

Page 80: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

Representation and Useof Indexicals

• Use of Deictic Center for parser to interpret indexicals as current referents

• And for generator to generate indexicals from current referents.

Page 81: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

A Personal Sense of Time

• *NOW is current time.• Updated when Cassie acts

or perceives a change of state.• Points into MTF to support vagueness

of “now”.

Page 82: The SNePS Approach to Cognitive Robotics

S.C. Shapiro

cse@buffal

o

For More Information

• Personnel• Manual• Tutorial• Bibliography• ftp’able SNePS source code• etc.• http://www.cse.buffalo.edu/sneps/