A paper by Kevin Gold and Brian Scassellati

Presented by Paul Dilley

Deictic Pronoun Learning and Mirror Self-Identification

Paper’s focusTwo milestones in human reasoning about the

self:

1. Mirror self-identification

2. Using deictic pronouns such as “I” and “You”

Building a robot to do these – in order to answer questions about the two

milestones

• Few animals recognise themselves in the mirror

• Social intelligence needed to do so – assumption

Bonbo

s Chimpan

zee

sOran

gutan

s

Gorilla

s Human

s

Dolphin

s

Orca

sElep

han

tsMag

pie

sBarn

owls

PigsMirror self-recognition

Mirror Test (Mirror Test.mp4)

Mark placed on being secretly

Mirror placed in front of being

It moves limbs to the mark?

Requires “Mirror Test.mp4” to be in the same directory:

Or watch the embedded movie in a media player:

Click here if the movie does not play

Fin!

How is the mirror test done?

The mirror test and “I”Problems:• Does the mirror test = Social intelligence?

• How different is the intelligence of using “I” vs. touching a mark, during a mirror test?

“I” and “You” are known as deictic pronouns

“Dik-tik”

Build a robot?The paper author’s built a robot that can

perform self mirror recognition and apply deictic pronouns to the reflection

Help answer questions on previous slide about human intelligence

Provide an insight for intelligent robot building

How it works

Audio is captured using adual-channel microphone Robot uses cameras to capture images

Face recognition

Motion Detection

Colour Detection

Self Recognition ModuleFinds motion that coincides with motor

control commands

Speech recognition

Sound localisation

Word learning

How it worksWord learning

Identify already understood words

Match agents with the properties of understood words

Eventual learning of “I” and “you”

Property TypesActions – “speaking”

Being the target of actions – “addressee”

1. No grammatical parsing, sentences are treated as simple collections of words

2. Each word learnt with confidence adds an associated property to a list of properties to look for in the

environment

3. Each agent is then matched

The process is done using chi-squares

Self IdentificationExploratory arm movements + studying the

time delay for motion detection

These experimental values are put onto a distribution with 95% falling between 0.5 to 1 seconds

Future movements in this time frame had the moving object labelled as “self”

Who has the ball?The robot goes into speaker mode and

prepares to address the questioner

Nico searches for utterances of learnt words, e.g. “got”

This is linked to an associated action “hasBall”

Nico then searches for a property that is unassociated with “hasBall” but true for the associated agent

Implementation

The robot learnt words-property pairs observing

two people play a game of catch with a dataset of 50

utterances

“You” became associated with the addressee and

“I” with the speaker

ImplementationNico was placed in front of a mirror, with the ball

either at his base or being held by the experimenter

Nico correctly answered “I got the ball” 16 out f 20

times.

The incorrect answer “You got...” was given twice,

and the robot misunderstood twice

ImplementationNico got confused when the experimenter

was holding the ball

He moved within Nico’s arm test confusing the bot

“You got the ball” was only correct 11/20 times

When the bot was told to stop fidgeting the result was much better: 18/20 times correct

Conclusions

Bot learns “I” and “you” from observation and applies them correctly

It’s use of “I” for a mirror image is an exciting step for

robotics

It does not represent “self-awareness” from a

scientific point of view – but is useful for

understanding behaviours

Importantly, the bot can associate words with the

actions in the environment – rather than just visual

properties

The authors believe: Applying words with functions and roles is more useful than with

superficial appearance

The authors believe: Allowing robots to

map human actions and properties onto

robotic states in general would be the next step for research

Recognising in the mirrorMost crucially: Nico could recognise its own feedback as self-generated – but a pre-programmed kinetic model, when the robot knows where its arm is, would probably suffice. It would require less training

The authors conclude:• The mirror test might be a better test of recognising feedback

• It’s not that useful: “why waste neurons on it” – but it has benefits to humans, but may not be as significant to intelligence as previously thought

• But asking a robot “Who is that in the mirror?” and answers such as “I” do touch on several different aspects of human intelligence

What do I think about the paper?

Most great apes can pass the mirror test by

adult hood

Thanks for listening!