the sneps approach to cognitive robotics
DESCRIPTION
The SNePS Approach to Cognitive Robotics. Stuart C. Shapiro Department of Computer Science and Engineering and Center for Cognitive Science University at Buffalo [email protected]. Outline. Introduction Intensional Representation & Propositions SNePS Connectives and Quantifiers - PowerPoint PPT PresentationTRANSCRIPT
S.C. Shapiro
cse@buffal
o
The SNePS Approach to Cognitive Robotics
Stuart C. Shapiro
Department of Computer Science and Engineeringand Center for Cognitive Science
University at Buffalo
S.C. Shapiro
cse@buffal
o
Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buffal
o
Goal• A computational cognitive agent that can:
– Understand and communicate in English; – Discuss specific, generic, and “rule-like” information;– Reason;– Discuss acts and plans;– Sense;– Act;– Remember and report what it has sensed and done.
S.C. Shapiro
cse@buffal
o
Embodied Cassie
• A computational cognitive agent– Embodied in hardware– or Software-Simulated– Based on SNePS and GLAIR.
S.C. Shapiro
cse@buffal
o
SNePS• Knowledge Representation and Reasoning
– Propositions as Terms• SNIP: SNePS Inference Package
– Specialized connectives and quantifiers• SNeBR: SNePS Belief Revision• SNeRE: SNePS Rational Engine• Interface Languages
– SNePSUL: Lisp-Like– SNePSLOG: Logic-Like– GATN for Fragments of English.
S.C. Shapiro
cse@buffal
o
GLAIR Architecture
Knowledge Level
Perceptuo-Motor Level
Sensory-Actuator Level
NL
VisionSonar
Motion
Proprioception
Grounded Layered Architecture with Integrated Reasoning
SNePS
S.C. Shapiro
cse@buffal
o
Interaction with CassieEnglish
(Statement, Question, Command)(Current) Set of Beliefs
[SNePS]
(Updated) Setof Beliefs[SNePS]
Actions[SNeRE]
(New Belief)[SNePS]
English sentence expressingnew belief answering question reporting actions
Answer[SNIP]
GATN Parser
GATN Generator
ReasoningClarification DialogueLooking in World
Reasoning
S.C. Shapiro
cse@buffal
o
Cassie, the BlocksWorld Robot
S.C. Shapiro
cse@buffal
o
Cassie, the FEVAHR
S.C. Shapiro
cse@buffal
o
FEVAHR/Cassie in the Lab
S.C. Shapiro
cse@buffal
o
FEVAHRWorld Simulation
S.C. Shapiro
cse@buffal
o
UXO Remediation Cassie
CassieCorner flag
NonUXO object
Corner flag
UXO
Batterymeter
Corner flag
Drop-off zone
Field
Safe zoneRechargingStation
S.C. Shapiro
cse@buffal
o
Crystal Space Environment
S.C. Shapiro
cse@buffal
o
Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buffal
o
Entities, Terms, Symbols, Objects
• Cassie’s mental entity: a person named Bill• SNePS term: B5• Object in world:
S.C. Shapiro
cse@buffal
o
Intensional Representation
Intensional entities are distincteven if coreferential.
“The morning star is the evening star.”“George IV wondered if Scott was the author of
Waverly.”
S.C. Shapiro
cse@buffal
o
McCarthy’s Telephone Number Problem
Mary's telephone number is Mike's telephone number. I understand that Mike's telephone number is Mary's
telephone number. Pat knew Mike's telephone number. I understand that Pat knew Mike's telephone number. Pat dialed Mike's telephone number. I understand that Pat dialed Mike's telephone number.
S.C. Shapiro
cse@buffal
o
Answering the Telephone Number Problem
Did Pat dial Mary's telephone number? Yes, Pat dialed Mary's telephone number. Did Pat know Mary's telephone number? I don't know.
S.C. Shapiro
cse@buffal
o
Representing Propositions
Propositions must befirst-class entities of the domain
Represented by terms.
S.C. Shapiro
cse@buffal
o
Discussing Propositions
That Bill is sweet is Mary's favorite proposition.I understand that Mary's favorite proposition is
that Bill is sweet.Mike believes Mary's favorite proposition.I understand that Mike believes that Bill is sweet.
S.C. Shapiro
cse@buffal
o
Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buffal
o
Logic for NLU &Commonsense Reasoning
Either Pat is a man or Pat is a woman or Pat is a robot. I understand that Pat is a robot or Pat is a woman
or Pat is a man. Pat is a woman. I understand that Pat is a woman. What is Pat? Pat is a woman and Pat is not a robot
and Pat is not a man.
S.C. Shapiro
cse@buffal
o
Representation in FOPL?
Man(Pat) Woman(Pat) Robot(Pat)
S.C. Shapiro
cse@buffal
o
Representation in FOPL?
Man(Pat) Woman(Pat) Robot(Pat)but don’t want inclusive or
S.C. Shapiro
cse@buffal
o
Representation in FOPL?
Man(Pat) Woman(Pat) Robot(Pat)but don’t want inclusive or
Man(Pat) Woman(Pat) Robot(Pat)+ +T T T
F
TSo don’t want exclusive or either
S.C. Shapiro
cse@buffal
o
andor
andor(i, j){P1, ..., Pn}
True iff at least i, and at most j of the Pi are True
S.C. Shapiro
cse@buffal
o
thresh
thresh(i, j){P1, ..., Pn}
True iff either fewer than i, or more than j of the Pi are True
Note: thresh(i, j) ~andor(i, j)
S.C. Shapiro
cse@buffal
o
or-entailment
{P1, ..., Pn} v=> {Q1, ..., Qn}
True iff for all i, j Pi Qj
S.C. Shapiro
cse@buffal
o
and-entailment
{P1, ..., Pn} &=> {Q1, ..., Qn}
True iff for all jP1 &…& Pn Qj
S.C. Shapiro
cse@buffal
o
Numerical entailment
{P1, ..., Pn} i=> {Q1, ..., Qn}
True iff for all jandor(i, n){P1, …, Pn } Qj
S.C. Shapiro
cse@buffal
o
Universal Quantifier
all(ū)({R1(ū),..., Rn(ū)} &=> {C1(ū),..., Cm(ū)})
Every ā that satisfiesR1(ū)&…& Rn(ū)
also satisfiesC1(ū),..., Cm(ū)})
S.C. Shapiro
cse@buffal
o
Numerical Quantifiers
nexists(i,j,k)(x) ({P1(x),..., Pn(x)}: {Q(x)})}
There are k individuals that satisfyP1(x) ... Pn(x)
and, of them, at least i and at most j also satisfyQ(x)
S.C. Shapiro
cse@buffal
o
Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buffal
o
MENTAL ACTS
• Believe(proposition)• Disbelieve(proposition)
S.C. Shapiro
cse@buffal
o
Act Selection
• Do-One({act1 ... actn})
• Snif(if(condition, act), ... if(condition, act) [else(act)])
S.C. Shapiro
cse@buffal
o
Act Iteration• Do-All({act1 ... actn})• Sniterate(if(condition, act), ... if(condition, act), [else(act)])• Snsequence(act1, ..., actn)• Cascade(act1, ..., actn)• P-Do-All({act1, ..., act2})
S.C. Shapiro
cse@buffal
o
Entity Iteration
WithSome(var, suchthat, do, [else])WithAll(var, suchthat, do, [else])WithSome+(var, suchthat, do, [else])WithNew(vars, thatare, suchthat, do, [else])
S.C. Shapiro
cse@buffal
o
Proposition/Act Transformers
• Achieve(proposition)• ActPlan(act, plan)• GoalPlan(proposition, act)• Precondition(act, proposition)• Effect(act, proposition)• WhenDo(proposition, act)• WheneverDo(proposition, act)• IfDo(proposition, act)
S.C. Shapiro
cse@buffal
o
Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buffal
o
Conditional Plans If a block is on a support then a plan to achieve
that the support is clear is to pick up the block and then put the block on the table.
all(x, y) ({Block(x), Support(y), On(x, y)} &=> {GoalPlan(Clear(y), Snsequence(Pickup(x), Put(x, Table)))})STRIPS-like representation: No times
S.C. Shapiro
cse@buffal
o
Use of Conditional Plan
GoalPlan(Clear(B), Snsequence(Pickup(A), Put(A, Table)))Remember (cache) derived propositions.
S.C. Shapiro
cse@buffal
o
Use of Conditional Plan
GoalPlan(Clear(B), Snsequence(Pickup(A), Put(A, Table)))???SNeBR to the rescue!
S.C. Shapiro
cse@buffal
o
A FEVAHR Acting Ruleall(p, o) ({Agent(p), Thing(o)}
&=> {Precondition(Follow(p, o), Near(p, o)),
GoalPlan(Near(p, o), Goto(p, o)),
Precondition(Goto(p, o), Lookat(p, o)),
ActPlan(Lookat(p, o), Find(p, o))})Uses a temporal model.
S.C. Shapiro
cse@buffal
o
Acting According to the Rule
S.C. Shapiro
cse@buffal
o
Acting According to the Rule
I found a red robot.I am looking at a red robot.
Follow a red robot.
S.C. Shapiro
cse@buffal
o
Acting According to the Rule
I went to a red robot.I am near a red robot.I am following a red robot.
I found a red robot.I am looking at a red robot.
Follow a red robot.
S.C. Shapiro
cse@buffal
o
A Plan for Blowing up UXOsall(a)(Agent(a) =>ActPlan(Blowup(a, UXOs), Act(a,Cascade(SearchforUxo(a),WithSome+(obj, Near(a, obj), WithNew({ch ex}, {Charge(ch), Explosion(ex)}, Possess(a, ch), Cascade(Place(a, ch, obj), Hide(a), Waitfor(a, ex), SearchforUxo(a))), goto(a, SafeZone))))))
S.C. Shapiro
cse@buffal
o
Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buffal
o
Representation and Use of Indexicals
• Words whose meanings are determined by occasion of use
• E.g. I, you, now, then, here, there• Deictic Center <*I, *YOU, *NOW>• *I: SNePS term representing Cassie• *YOU: person Cassie is talking with• *NOW: current time.
S.C. Shapiro
cse@buffal
o
Analysis of Indexicals(in input)
• First person pronouns: *YOU• Second person pronouns: *I• “here”: location of *YOU• Present/Past relative to *NOW.
S.C. Shapiro
cse@buffal
o
Generation of Indexicals
• *I: First person pronouns• *YOU: Second person pronouns• *NOW: used to determine tense and aspect.
S.C. Shapiro
cse@buffal
o
Come here.
Use of Indexicals 1
S.C. Shapiro
cse@buffal
o
Come here.I came to you, Stu.I am near you.
Use of Indexicals 2
S.C. Shapiro
cse@buffal
o
Who am I?Your name is ‘Stu’and you are a person.Who have you talked to?I am talking to you.Talk to Bill.I am talking to you, Bill.Come here.
Use of Indexicals 3
S.C. Shapiro
cse@buffal
o
Come here.
I found you.I am looking at you.
Use of Indexicals 4
S.C. Shapiro
cse@buffal
o
Come here.
I came to you.I am near you.
I found you.I am looking at you.
Use of Indexicals 5
S.C. Shapiro
cse@buffal
o
Who am I?
I talked to Stuand I am talking to you.
Your name is ‘Bill’and you are a person.Who are you?I am the FEVAHRand my name is ‘Cassie’.Who have you talked to?
Use of Indexicals 6
S.C. Shapiro
cse@buffal
o
Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buffal
o
A Personal Sense of Time
• *NOW contains SNePS term representing current time.
• *NOW moves when Cassie acts or perceives a change of state.
S.C. Shapiro
cse@buffal
o
B6
Representation of Time
find
lex
action objectB1
!
agentact
event time
NOW
!!before after before after
?????????????
I
S.C. Shapiro
cse@buffal
o
Movement of Timet1 t2!before after t3!before after
NOW NOW NOW
S.C. Shapiro
cse@buffal
o
Performing a Punctual Actt1 t3!before after
NOW NOW
t2!before after
!
time
event
S.C. Shapiro
cse@buffal
o
Performing a Durative Actt1
NOW
!before after t2
!
time
event
NOW
t3!
supintsubint
S.C. Shapiro
cse@buffal
o
The Pacemaker
• PML process periodically increments variable COUNT.
• *COUNT = some PML integer.• Reset to 0 when NOW moves.• Provides bodily “feel” of passing time.
S.C. Shapiro
cse@buffal
o
Quantizing Time
Cannot conceptualize fine distinctions in time intervals.
So quantize, e.g. into half orders of magnitude (Hobbs, 2000).
S.C. Shapiro
cse@buffal
o
Movement of Time with Pacemaker
NOW COUNT nhom
0
KL
PML
t1 t2q
!before after
time duration!
S.C. Shapiro
cse@buffal
o
The Problem of the Fleeting Now
How can you reason about “now”if it never stands still?
S.C. Shapiro
cse@buffal
o
Fleeting Now Example 1
12:15:00: “Is John having lunch now?”12:15:02: Agent walks to John’s office.12:17:00: Agent sees John at his desk, eating.12:19:00: Agent reports “yes”.
Appropriate granularity.
S.C. Shapiro
cse@buffal
o
Fleeting Now Example 2
12:15:00: “Is John having lunch now?”Agent knows John is at home without a phone.Agent contemplates driving to John’s home.Don’t bother---inappropriate granularity.
S.C. Shapiro
cse@buffal
o
The Vagueness of “now”I’m now giving a talk.I’m now teaching a course.I’m now visiting Toronto.I’m now living in Buffalo.The agent is now walking to John’s office.The agent is now seeing if John is eating lunch.
Multiple now’s at different granularities.
S.C. Shapiro
cse@buffal
o
NOW-MTF
NOWSemi-lattice of times, all of which contain *NOW,any of which could be meant by “now”Finite---only conceptualized times of conceptualized states
Maximal Temporal Frame based on *NOW
S.C. Shapiro
cse@buffal
o
Moving NOW with MTF
NOW
S.C. Shapiro
cse@buffal
o
Typical Durations“If the walk light is on now, cross the street.”
Relevant duration is typical duration of walk lights.
“Is John having lunch now?”Relevant duration is typical duration of lunch.
Use quantized typical durations when updating NOW-MTFs.
S.C. Shapiro
cse@buffal
o
Using Appropriate Granularity
NOW
Lunch time
Lunch? Lunch!
Yes!
S.C. Shapiro
cse@buffal
o
Outline• Introduction• Intensional Representation & Propositions• SNePS Connectives and Quantifiers• SNeRE Acting Constructs• Example Plans • Representation and Use of Indexicals• A Personal Sense of Time• Summary
S.C. Shapiro
cse@buffal
o
Goal
• A computational cognitive agent/robot• That can communicate in natural
language.
S.C. Shapiro
cse@buffal
o
Intensional Representation& Propositions
• SNePS terms represent mental entities.• May assert that two entities are coreferential.• Relations/acts may be declared transparent.• Propositions are first-class entities.
S.C. Shapiro
cse@buffal
o
SNePS Connectives and Quantifiers
• Designed logical connectivesand rules of inference
More appropriate for NLU and Commonsense reasoningthan in standard FOPC.
S.C. Shapiro
cse@buffal
o
SNeRE Acting Constructs
• Separate, but CoordinatedSyntax and SemanticsFor Acting and for Reasoning
S.C. Shapiro
cse@buffal
o
Representation and Useof Indexicals
• Use of Deictic Center for parser to interpret indexicals as current referents
• And for generator to generate indexicals from current referents.
S.C. Shapiro
cse@buffal
o
A Personal Sense of Time
• *NOW is current time.• Updated when Cassie acts
or perceives a change of state.• Points into MTF to support vagueness
of “now”.
S.C. Shapiro
cse@buffal
o
For More Information
• Personnel• Manual• Tutorial• Bibliography• ftp’able SNePS source code• etc.• http://www.cse.buffalo.edu/sneps/