Natural Language Processing

Spatial Semantic Hierarchy

&

Narrative Maps

Vladimir Kulyukin

www.vkedco.blogspot.com

Outline

Spatial Semantic Hierarchy (SSH)

SSH Levels

Axioms & Inferences

Deduction, Induction, Abduction

Narrative Maps

Extraction of SSHs from Narrative Maps

Background

Background

Spatial Semantic Hierarchy (SSH) was

discovered and developed by Benjamin

Kuipers and his students

SSH is a model of knowledge of

environments that consists of multiple

interacting representations

These representations are quantitative and

qualitative

Types of Spatial Knowledge

Large-scale space – space that exceeds the

agent’s sensory horizon

Visual space – immediate environment that the

agent can explore by gaze

Graphical space – spatial layouts and relations

among symbols expressed graphically (e.g., on

paper or tablet)

The term cognitive map refers to human

knowledge of large-scale space

Why Study Spatial Knowledge?

Spatial knowledge is fundamental to

commonsense knowledge

We use spatial knowledge daily to navigate

We use spatial knowledge represent concepts

graphically

We use spatial knowledge of the real world to

organize virtual worlds (e.g., the concept of

computer desktop)

SSH Levels

SSH Levels

SSH consists of five levels: 1) sensory; 2)

control; 3) causal; 4) topological; and 5)

metrical

Kuipers organizes these levels in a lattice

of nodes where each node corresponds to a

representation with its own ontology (aka

conceptualization in the terminology of this

course), axioms, and inference rules

Relations among SSH Levels

Level X is dependent on level Y when X

“presupposes, or is defined in terms of, or

is inferred from, knowledge in the

representation at” Y

Level X receives information from level Y

when knowledge stored and/or computed at

Y is accessible to X

Organization of SSH Lattice Nodes

SSH lattice nodes are organized along two

dimensions: qualitative vs. quantitative

(horizontal) and ontological (vertical)

The horizontal level indicates that spatial

knowledge can be either qualitative or

quantitative

The vertical level organizes nodes in terms of

ontological dependencies

Sensory Level

Sensory level is the interface to the agent’s

sensorimotor system

Sensors can be continuous (camera, laser,

sonar, etc.) or discrete (digital compass,

odometer, RFID reader, Wi-Fi receiver, etc.)

Distinction continuous vs. discrete is

arbitrary because a continuous sensor’s

output can be made discrete

Control Level

Control level is a set of control laws that

bind the agent and its environment in a

dynamic system within some uniform

segment of that environment

Each control law has conditions for its

appropriateness and termination

There are two broad classes of control

laws: hill-climbing & trajectory following

Causal Level

Causal level discretizes the continuous

world and the agent’s actions in terms of

sensory views, actions, and relations among

views and actions

This is similar to the STRIPS action

semantics of the PDDL operators that we

have investigated before

Plans constructed at the causal level are

executed on the control level

Quick Review of PDDL

PDDL Problem Specification

Knowledge engineers must do two things to do

problem solving describe in PDDL: 1) describe a

domain and 2) describe a problem

The description of the domain is placed into a

domain file

The description of the problem is placed into

a problem file

Domain Definition

(define (domain <DOMAIN NAME>)

<REQUIREMENT>*

<PREDICATE>*

<ACTION>*

)

<REQUIREMENT>* is a statement that specifies requirements

(e.g., :typing or :equal)

<PREDICATE>* is a sequence of predication specifications

<ACTION>* is a sequence of action specifications

Problem Definition

(define (problem <PROBLEM NAME>)

<DOMAIN NAME>

<OBJECT STATEMENT>

<INITIAL STATE DESCRIPTION>

<GOAL DESCRIPTION>

)

<DOMAIN NAME> is a statement that references the domain in which the

problem must be solved

<OBJECT STATEMENT> is a sequence of object constants

<INITIAL STATE DESCRIPTION> is a sequence of predicates that describe the

initial state of the world

<GOAL DESCRIPTION> is a sequence of predicates that describe the goal state

of the world

STRIPS Semantics of PDDL Actions

STRIPS (Stanford Research Institute Planning System) is an

AI Planner developed by Richard Fikes and Nils Nilson in

1971

A STRIPS operator has preconditions (a set of predicates

that must be true in the current state of the world for the

operator to be considered application) and postconditions

(a set of predicates that will be true in the state of the

world that results from the operator’s application)

A PDDL action also has preconditions and postconditions

called effects

STRIPS Semantics of PDDL Actions

(:action <ACTION_NAME>

:parameters (?x1 ?x2 …. ?xn)

:precondition (<PREDICATE STATEMENT>*)

:effect (<PREDICATE STATEMENT>*)

)

SYMBOLIC VARIABLES

THAT BIND TO OBJECT

CONSTANTS

PREDICATE STATEMENTS THAT

MUST BE TRUE IN THE CURRENT

STATE OF THE WORLD FOR THE

ACTION <ACTION_NAME> TO

EXECUTE

PREDICATE STATEMENTS THAT WILL BE

TRUE IN THE WORLD AFTER THE ACTION

IS EXECUTED

Topological Level

Topological level describes the world in terms of

places, paths, regions, and their connectivity, order,

and containment

SSH makes a claim that a topological network map is

more effective for planning that the flat causal model

SSH makes another claim that “the ability to plan and

act is not dependent on the availability of quantitative

spatial knowledge”

The latter claim is very interesting, because, if it is

true, planning can be done without direct contact with

the world

Metrical Level

Metrical level is a global geometric map of the

environment with a single frame of reference

Kuipers does say that quantitative geometric

information is also present at each SSH level:

local analog maps at control level; action

magnitudes at causal level; headings and

distances at topological level

Smaller local frames can be linked into a global

frame of reference

Deeper Dive into SSH Levels

Control Level

Uniform Segments

Environment is divided into uniform segments each

of which has its own control law (e.g., follow middle

line, follow right wall, follow left wall, etc.)

The agent is assumed to use only sensory input to

execute control laws

The agent receives a continuous time series of

sensory values and outputs a continuous time series of

motor signals

A control law is a relation b/w sensory inputs and

motor outputs

Control Laws as Differential Equations

The agent, the environment, and a given

control law is a dynamic system

This system can be modeled by a differential

equation

The system’s behavior is described by a

solution to the equation

Hill-Climbing vs. Trajectory Following

A hill-climbing control law brings the agent into

a locally distinctive state

A hill-climbing control law terminates when a

distinctiveness measure (e.g., distance) reaches

a local maximum

A trajectory-following control law brings the

agent from one distinctive state to the

neighborhood of the next

Low-Level Details of Control Laws

In Sections 2.1, 2.2., 2.3, 2.4, and 2.5 of his

article “The Spatial Semantic Hierarchy,” Kuipers

discusses low-level details of control laws

These are fascinating and insightful but are

peripheral to the NLP problem of automating SSH

acquisition from narrative maps that we are

investigating here

Hence, we will skip them in this presentation

Guarantees of Control Level

Guarantees of control level are more

interesting to us, because they specify what we

can assume about the agent’s physical abilities

There are two broad guarantees:

1) After a hill-climbing law terminates at a

distinctive state, at least one trajectory

following law is applicable (no dead ends)

2) After a trajectory-following law terminates

at least one hill-climbing law is applicable

Causal Level

Action Abstraction Schema

A sequence of control laws can be abstracted into an

action that starts at a given sensory view V and ends

at another sensory view V’

This abstraction is called the schema <V, A, V’>

Situation calculus is a suitable formalism of the

causal level

Causal level, unlike control level, consists of

discrete states

The agent performs a sequence of discrete actions

that result in state transitions

View

A view is a description of the sensory input

vector s(t) = [s1(t), …, sn(t)]

My guess is that this definition is deliberately

vague to give the knowledge engineer a lot of

elbow space to play with various representations

For example, a description can specify a Wi-Fi

cluster or the color histogram of an image take

by the robot’s camera

Actions, Schemas, Routines

An action is a sequence of one or more control

laws

An action is initiated at a locally distinctive

state specified by one view description and

terminates at another locally distinctive state

specified by another view description

Actions are specified by schemas <V, A, V’>

A routine is a set of schemas indexed by initial

view

Declarative & Procedural Schema Interpretations

The first interpretation (declarative) means that a view V is

observed in situation s0 and the view V’ holds in the result of

executing action A in situation s0

The second interpretation (procedural) means that if a

view V is observed, then execute action A right away (now)

ℎ𝑜𝑙𝑑𝑠 𝑉, 𝑠0 && ℎ𝑜𝑙𝑑𝑠 𝑉′, 𝑟𝑒𝑠𝑢𝑙𝑡 𝐴, 𝑠0

ℎ𝑜𝑙𝑑𝑠 𝑉, 𝑛𝑜𝑤 ⇒ 𝑑𝑜 𝐴, 𝑛𝑜𝑤

Turns & Travels

At the causal level, all actions are classified into two categories:

turn and travel

This categorization may be too restrictive: SSH article states that

one can construct environments for which these categories break

down

A claim is made, however, that these actions are sufficient for office

spaces and street networks

A turn is an action that leaves the agent in the same place; a travel

is an action that takes the agent from one place to another

𝑇𝑟𝑎𝑣𝑒𝑙 δ, ΔΘ , 𝑤ℎ𝑒𝑟𝑒 δ, ΔΘ 𝑎𝑟𝑒 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒 𝑎𝑛𝑑 𝑜𝑟𝑖𝑒𝑛𝑡𝑎𝑡𝑖𝑜𝑛 𝑐ℎ𝑎𝑛𝑔𝑒, 𝑟𝑒𝑝𝑠𝑒𝑐𝑡𝑖𝑣𝑒𝑙𝑦

𝑇𝑢𝑟𝑛 α , 𝑤ℎ𝑒𝑟𝑒 α 𝑖𝑠 𝑎𝑛 𝑟𝑜𝑡𝑎𝑡𝑖𝑜𝑛 𝑎𝑛𝑔𝑙𝑒

Routines

A routine is a sequence of schemas

A routine is indexed by its initial view, i.e., the view of

the 1st schema

A routine represents a behavior that moves the agent from

one distinctive state to another distinctive state

Figure 8 in “The Spatial Semantic Hierarchy” seems to

imply that distinctive states are described by views

A routine can be viewed as a description of a behavior or

as a procedure for executing that behavior in the world

Complete & Partial Schemas in Routines

A schema of the form <V, A, V’> is complete

A schema of the form <V, A, nil> is partial

If a routine is defined in terms of complete

schemas, the agent can both execute and

describe it

If a routine is defined in terms of partial

schemas, the agent can only execute it in the

world but not describe it

Complete & Adequate Routines

Suppose 𝑉0𝐴0𝑉1𝐴1𝑉2𝐴2, … , 𝑉𝑛−1𝐴𝑛−1, 𝑉𝑛 is an

alternating sequence of views and actions

A routine R is complete from 𝑉0 to 𝑉𝑛 if R has a

complete schema < 𝑉𝑖 , 𝐴𝑖 , 𝑉𝑖+1 > for each

0 < 𝑖 < 𝑛

A routine R is adequate from 𝑉0 to 𝑉𝑛 if it

contains either a complete schema

< 𝑉𝑖 , 𝐴𝑖 , 𝑉𝑖+1 > or a partial schema < 𝑉𝑖 , 𝐴𝑖 , 𝑉𝑖+1 >

for each 0 < 𝑖 < 𝑛

Complete & Adequate Routines

Adequate routines support situated action, i.e., behavior

that can be executed only when the agent is placed

(“situated”) in the world

Complete routines support both situated action and

cognitive manipulation, i.e., the agent can both execute

and describe the behavior

Example: if the agent can only navigate a route, the

navigation routine is adequate; if the agent can both

navigate and describe it, the navigation routine is complete

Deduction, Induction, & Abduction

Deduction

Given the implication A B and the truth of A,

infer B

Example:

- Implication: if the agent’s Wi-Fi classifier

classifies the input signal at cluster C, the agent

is at location L

- Truth of antecedent: the agent’s Wi-Fi classifier

classifies the input signal at cluster C

- Inference: the agent is at cluster location L

Induction

Infer the consequent B from the antecedent A if B

has so far always followed A

Example: you observe a swan and note that its

color is white; you observe another swan and note

that its color is white; you observe another n swans

and they are all white; you conclude that if a bird is

a swan (A), then its color is white (B)

Inductive inferences are always congruent with

agents’ experience but are not always be true: there

are, in fact, black swans

Abduction

Infer A as a possible antecedent of the observed

consequent B

In the literature on abduction, A is sometimes

referred to as a possible explanation of B

Example: you observe that your neighbor’s lot is

wet on a dry day (B) and infer that your neighbor has

watered the lawn (A); chances are your inference is

true but there is a chance that it is wrong: there may

be a leaking sprinkler

Topological Level

Elements of Topological Level

A place is a zero-dimensional part of the environment

A path is a one-dimensional subspace (e.g., a street in a

city)

There are two directions along a path: dir = +1 and dir =-1

These directions may be loosely interpreted and forward

and backward

A travel action moves the agent from one place on a path

to another place on a path

A turn action keeps the agent in the same place

A region is a 2D subset of the environment

A region may be abstracted into a place

Abduction

Given a sequence of views and actions, the

agent infers places, paths, and regions by

abduction

The agent postulates a minimal set of places,

paths, and regions consistent with the views and

actions

Abduced elements may or may not be sufficient

to explain the sequence of observed views and

actions

Topological Relations

Given a sequence of views and actions, the

agent infers places, paths, and regions by

abduction

The agent postulates a minimal set of places,

paths, and regions consistent with the views and

actions

Abduced elements may or may not be sufficient

to explain the sequence of observed views and

actions

Topological Relations

𝑎𝑡 𝑣𝑖𝑒𝑤, 𝑝𝑙𝑎𝑐𝑒 − 𝑣𝑖𝑒𝑤 is seen at 𝑝𝑙𝑎𝑐𝑒

𝑎𝑙𝑜𝑛𝑔 𝑣𝑖𝑒𝑤, 𝑝𝑎𝑡ℎ, 𝑑𝑖𝑟 − 𝑣𝑖𝑒𝑤 is seen along 𝑝𝑎𝑡ℎ in direction 𝑑𝑖𝑟

𝑜𝑛 𝑝𝑙𝑎𝑐𝑒, 𝑝𝑎𝑡ℎ − 𝑝𝑙𝑎𝑐𝑒 is on 𝑝𝑎𝑡ℎ

𝑜𝑟𝑑𝑒𝑟 𝑝𝑎𝑡ℎ, 𝑝𝑙𝑎𝑐𝑒1, 𝑝𝑙𝑎𝑐𝑒2, 𝑑𝑖𝑟 − the order on path from 𝑝𝑙𝑎𝑐𝑒1 to 𝑝𝑙𝑎𝑐𝑒2 is 𝑑𝑖𝑟

𝑟𝑖𝑔ℎ𝑡𝑂𝑓 𝑝𝑎𝑡ℎ, 𝑑𝑖𝑟, 𝑟𝑒𝑔𝑖𝑜𝑛 − 𝑝𝑎𝑡ℎ, facing direction 𝑑𝑖𝑟, has 𝑟𝑒𝑔𝑖𝑜𝑛 on its right

𝑙𝑒𝑓𝑡𝑂𝑓 𝑝𝑎𝑡ℎ, 𝑑𝑖𝑟, 𝑟𝑒𝑔𝑖𝑜𝑛 − 𝑝𝑎𝑡ℎ, facing direction 𝑑𝑖𝑟, has 𝑟𝑒𝑔𝑖𝑜𝑛 on its left

𝑖𝑛 𝑝𝑙𝑎𝑐𝑒, 𝑟𝑒𝑔𝑖𝑜𝑛 − 𝑝lace is in 𝑟𝑒𝑔𝑖𝑜𝑛

Topological Axioms

𝑜𝑟𝑑𝑒𝑟 𝑝𝑎𝑡ℎ, 𝐴, 𝐵, 𝑑𝑖𝑟 → 𝑜𝑛 𝐴, 𝑝𝑎𝑡ℎ & 𝑜𝑛(𝐵, 𝑝𝑎𝑡ℎ)

┐𝑜𝑟𝑑𝑒𝑟 𝑝𝑎𝑡ℎ, 𝐴, 𝐴, 𝑑𝑖𝑟

𝑜𝑟𝑑𝑒𝑟 𝑝𝑎𝑡ℎ, 𝐴, 𝐵, +1 ↔ 𝑜𝑟𝑑𝑒𝑟(𝑝𝑎𝑡ℎ, 𝐵, 𝐴, −1)

𝑜𝑟𝑑𝑒𝑟 𝑝𝑎𝑡ℎ, 𝐴, 𝐵, 𝑑𝑖𝑟 &𝑜𝑟𝑑𝑒𝑟 𝑝𝑎𝑡ℎ, 𝐵, 𝐶, 𝑑𝑖𝑟 → 𝑜𝑟𝑑𝑒𝑟(𝑝𝑎𝑡ℎ, 𝐴, 𝐶, 𝑑𝑖𝑟)

∃𝛼 𝑎𝑙𝑜𝑛𝑔 𝑉, 𝑝𝑎𝑡ℎ, 𝑑𝑖𝑟 & 𝑉, 𝑡𝑢𝑟𝑛, 𝛼 , 𝑉′ & 𝑎𝑙𝑜𝑛𝑔 𝑉′, 𝑝𝑎𝑡ℎ, −𝑑𝑖𝑟

Abducing Places & Paths from Views & Actions

Every view is observed at some place.

∀𝑣𝑖𝑒𝑤 ∃𝑝𝑙𝑎𝑐𝑒 𝑎𝑡 𝑣𝑖𝑒𝑤, 𝑝𝑙𝑎𝑐𝑒

Abducing Places & Paths from Views & Actions

If the agent turns, it does not change its place: 𝑉, 𝑡𝑢𝑟𝑛 𝛼 , 𝑉′ → ∃𝑝𝑙𝑎𝑐𝑒 𝑎𝑡 𝑉, 𝑝𝑙𝑎𝑐𝑒 &𝑎𝑡 𝑉′, 𝑝𝑙𝑎𝑐𝑒

Abducing Places & Paths from Views & Actions

If the agent travels a non-zero distance, then the first and second view exist at two distinct places. 𝑉, 𝑡𝑟𝑎𝑣𝑒𝑙 𝛼 , 𝑉′ & 𝛿 ≠ 0 → ∃𝑝𝑙𝑎𝑐𝑒1, 𝑝𝑙𝑎𝑐𝑒2 𝑝𝑙𝑎𝑐𝑒1 ≠ 𝑝𝑙𝑎𝑐𝑒2 & 𝑎𝑡 𝑉, 𝑝𝑙𝑎𝑐𝑒1 & 𝑎𝑡 𝑉′, 𝑝𝑙𝑎𝑐𝑒2

Abducing Places & Paths from Views & Actions

If the agent travels, then there are a path and direction such that the 1st view V exists on that path in that direction and the 2nd view V’ exists on that path in the same direction. 𝑉, 𝑡𝑟𝑎𝑣𝑒𝑙 𝛼 , 𝑉′ → ∃𝑝𝑎𝑡ℎ, 𝑑𝑖𝑟 𝑎𝑙𝑜𝑛𝑔(𝑉, 𝑝𝑎𝑡ℎ, 𝑑𝑖𝑟) & 𝑎𝑙𝑜𝑛𝑔 𝑉′, 𝑝𝑎𝑡ℎ, 𝑑𝑖𝑟

Abducing Places & Paths from Views & Actions

If the agent travels, then there are a path and a direction with two places such that the first place has the first view, the second place has the second view, both views exist along the path and can be ordered along the same direction. 𝑉, 𝑡𝑟𝑎𝑣𝑒𝑙 δ , 𝑉′ & 𝛿 ≠ 0 → ∃𝑝𝑙𝑎𝑐𝑒1, 𝑝𝑙𝑎𝑐𝑒2, 𝑝𝑎𝑡ℎ, 𝑑𝑖𝑟 𝑎𝑡(𝑉, 𝑝𝑙𝑎𝑐𝑒1) & 𝑎𝑡 𝑉′, 𝑝𝑙𝑎𝑐𝑒2 & 𝑎𝑙𝑜𝑛𝑔(𝑉, 𝑝𝑎𝑡ℎ, 𝑑𝑖𝑟 & 𝑜𝑟𝑑𝑒𝑟(𝑝𝑎𝑡ℎ, 𝑝𝑙𝑎𝑐𝑒1, 𝑝𝑙𝑎𝑐𝑒2, 𝑑𝑖𝑟 )

Regions, Boundaries, Abstractions

Regions are sets of places

Places are grouped into regions because

1) they are located on one side of a specific

boundary;

2) they share a 2D metrical frame;

3) they are abstracted to the same place in a

higher-level topological map

Regions, Boundaries, Abstractions

SSH supports upward and downward mapping

Upward mapping: multiple places at a lower level map to a

single place/region at a higher level

Downward mapping: a single place at a higher level map to

multiple places at a lower level

An abstraction region is the set of places in a more detailed

map abstracted to a particular place

Example: a corridor can be abstracted to a single place in a

higher-level map

Topological Level Uses

Topological level of representation supports

various problem-solving methods

It can be searched as a graph (DFS, BFS)

Distance measures, when and if they are

available, support A* and Dijkstra

Topological level can support goals and sub-

goals

Metrical Level

Global Metrical Mapping

An agent may have a global single frame of reference (2D

or 3D)

Many useful state of knowledge cannot be expressed

numerically in terms of real numbers (orientation error)

Storage of large global frame of references may present

problems

Global frame of reference can be split into a patchwork of

local frame of references

Narrative Maps

Narrative Map

Narrative map consists of two verbal descriptions:

verbal route descriptions and verbal surveys of a given

environment

The origins of narrative maps vary from blogs,

forums, books, O&M instructors

Question: To what extent can narrative maps be

mined/parsed to extract various levels of SSH?

Route Description 01

Exit Northbound M 4 or M 104 bus on Broadway between 120th and

121st streets. Walk straight to the inside guideline of the sidewalk and

turn right. Trail along the left and in 50 feet you will reach the corner of

120th and Broadway.

source: http://www.clickandgomaps.com/

Route Description 02

Turn left and continue trailing. In 50 feet, you will reach the first left side

opening. Pass this opening and continue straight. In 200 feet, you will

reach the second left side opening, a driveway that leads to Thorndike.

Take this left turn.

source: http://www.clickandgomaps.com/

Route Description 03

Pass through the second set of double doors and you will be in the

main lobby of the Thorndike building. At 1:00, 40 feet away is the

disability services main office.

source: http://www.clickandgomaps.com/

Route Description 04

As you continue along this bypass lane, you will notice a rough gravel

driveway near its endpoint. Five feet after crossing this driveway, there

is a mailbox that sticks out into the walking path. Continue following

along the left side edge, and in 800 to 1000 feet, you will reach a

second bypass lane.

source: http://www.clickandgomaps.com/

Route Description 05

At any point, turn and cross to the South side of route 83 and then turn

East with the grass line on your right. It will be easiest for both you and

Pixie to walk along the dirt or grass to the right of the pavement. The

first intersection you reach will be 173rd street. Cross and turn right to

proceed towards your home.

source: http://www.clickandgomaps.com/

Thoughts on Automated SSH Extraction

from

Narrative Maps

Tentative Suggestions

Represent landmarks in terms of SSH: place, path,

region

Represent actions that the agent can execute on

route (CD primitive acts are a possibility)

Develop a parser that parses narrative maps into

landmarks and actions

Develop an SSH compiler that takes graphs produced

by the parser and creates various levels of SSH

Conceptual Analyzer

CA: CD Parsing

Natural Language Input

CD Graphs (aka CDs) Inference Engine

Modified and/or New CDs

LTM

PTRANS

PTRANS – transfer of the physical location of an

object

Examples:

1) GO is an PTRANS of oneself to a place

2) PUT is an PTRANS of an object to a place

The robot went to the lab.

The robot put the block on the table.

PROPEL

PROPEL – application of a physical force to an object;

this primitive applies whenever any force is applied

Examples:

PUSH, PULL, KICK, THROW have the PROPEL primitive

The robot pushed the chair to the wall.

This is an instance of PROPEL by the robot to the chair

that caused a PTRANS of the chair from its current

location to the wall.

MOVE

MOVE – the movement of a body part of an

agent/animal by that agent/animal

Examples:

KICK, HAND have the MOVE primitive

The boy kicked the ball.

This is an instance of MOVE by the boy of his foot to

the ball that causes a PTRANS of the ball from its

current location to some unknown location.

GRASP

GRASP – the grasping of an object by an actor

Examples:

HOLD, GRAB have the GRASP primitive

The robot picked up the ball from the floor.

This is an instance of GRASP by the robot of the ball to

the ball that causes a PTRANS of the ball from the

floor into the robot’s gripper. This is also an instance of

MOVE by the robot of its gripper to the ball.

INGEST

INGEST – the taking of an object by an animal/agent to the inside of that

animal agent

Examples:

EAT, DRINK, SMOKE, BREATHE have the INGEST primitive

The robot charged.

John ate an apple.

These are instances of INGEST. The first sentence is an INGEST by the

robot of electricity inside the robot’s batter. The second sentence is an

instance of INGEST by John of the apple to John’s stomach.

MTRANS

MTRANS – the transfer of mental information within one animal/agent or between/among

animals/agents.

CD Theory partitions the agent’s memory into two components: CP (conscious processor

where current mental manipulation occurs) and LTM (long-term memory where things are

stored)

Examples:

TELL, INFORM, SEE, FORGET have the MTRANS primitive

Mary told the robot how to get to the lab.

The robot told Mary which rooms it had cleaned.

Both sentences are instances of MTRANS. Mary does an MTRANS of a route from some

location to the lab. The robot does an MTRANS of the rooms it had cleaned to Mary.

SPEAK

SPEAK – the production of sounds by an animal/agent.

Examples:

SHOUT, PURR, BEEP have the SPEAK primitive

The robotic car beeped twice.

Mary yelled at John.

ATTEND

ATTEND – the focusing of a sense organ by an animal/agent

toward a stimulus.

Examples:

ATTEND(EAR) – LISTEN

ATTEND(EYE) – SEE

ATTEND(NOSE) – SMELL

ATTEND(SKIN) – TOUCH

The robot detected a door.

John saw an exit.

A Field Study of the SSH Topological Level in

Blind Navigation of Modern Supermarket

Background

We investigated the utility of verbal route

instructions in a longitudinal study of blind shopping in

supermarkets

In our system, called ShopTalk, verbal route

directions were generated from a manually

constructed topological map (inspired by the

topological level of the SSH) of the supermarket’s

locomotor space

ShopTalk 1.0

Field Study Results

Ten visually impaired participants were able to

detect environmental cues needed to make sense of

the generated verbal instructions (provided at

beginning of experiment)

The participants used their O&M skills to localize and

orient themselves in the store, without any wearable

or environment-embedded sensors

Field Study Results

A key finding was that verbal route directions were

sufficient for our sample of independent travelers to

navigate this supermarket reliably

The more they used the system, the less they

requested verbal route directions

This finding suggests that the stores (and other

dynamic and complex environments) may not need to

be instrumented with any external sensors, such as

RFID tags, Wi-Fi routers, IR transmitters, etc.

References & Reading Suggestions

B. Kuipers. (2000). “The Spatial Semantic Hierarchy.”

Artificial Intelligence 119, pp. 191-233.

R. Schank, C. Riesbeck W. A. (1981) Inside Computer

Understanding. Lawrence Erlbaum & Associates.

J. Nicholson, V. Kulyukin, D. Coster. (2009). “ShopTalk:

Independent Blind Shopping Through Verbal Route

Directions and Barcode Scans.” The Open Rehabilitation

Journal, ISSN: 1874-9437 Volume 2, 2009, DOI

10.2174/1874943700902010011.