Situation Models and Embodied Language Processes

Franz Schmalhofer

University of Osnabrück / Germany

1) Memory and Situation Models

2) Computational Modeling of Inference Processes

3) What Memory and Language are for

4) Neural Correlates

5) Integration of Behavioral Experiments and Neural Correlates (ERP; fMRI) by Formal Models

Northwest of Germany

Schloß (University Building Osnabrück)

Cognitive Science in Osnabrück

Bachelor Degree:

Cognitive Psychology,

Computational Linguistics

Computer Science and

Artificial Intelligence

Neurobiology

Neuroinformatics

Mathematics

Philosophy of Mind

Masters Degree:

PhD Program:

Townhall of Osnabrück (early 17-th century negotiations for a European peace)

Central Europe: 1618 - 1648

Arbitrary and Perceptual Symbols

Steckenpferd

Hobby Horse

Franz Schmalhofer

EDUCATION

• 1978 Diplom Psychology

Universität Regensburg

• 1982 PhDUniversity of Colorado

• 1996 HabilitationUniversität Heidelberg

EMPLOYMENT

• Universität Heidelberg und Freiburg, 1982-1987

• McGill University/Montreal, 1987/88

• University of Colorado/USA, visiting, 1989

• German Research Center for Artificial Intelligence (DFKI) Kaiserslautern, 1989-2000

• Cognitive Science, University of Osnabrück; (Cognitive Psychology), since 2000

Introduction

• Please introduce yourself: Name, University, City, Country

• What you have studied so far

• What do you expect from this course

Situation models and Embodied Language Processes

1. Memory and Situation models 2. Computational Modeling of

Inference Processes3. What memory and language are

for4. Neural correlates of language

processes5. Integration of behavioral

experiments, computational models, neural correlates (ERP; fMRI)

• Analyze data from memory experiments

• Do a C-I model

• Embodied language processes; extend C-I model (marker passing);

• Discuss ERP and fMRI

• Levels approach, material structuring, analysis of dependent measures

Lectures Small group meetings

Memory and Situation Models

1. Osnabrück, Maps, Cognitive Science

2. Ebbinghaus and serial position effects

3. Serial position curves , Aktinson & Shiffrin as cognitive architecture

4. False memory, Loftus, Bransford & Franks

5. Referenced situation important, not so much the stimuli

Analyze false memory;

• Enter data; one participant per line

• Merge data

• Calculate means and standard deviations

Herrmann Ebbinghaus (1850 - 1909)

• Born in Barmen (near Bonn)

• studied in Bonn, Halle, Berlin,

• went abroad and tutored in France and England

• 1880 „Privatdozent“ at the University of Berlin

• 1885 „Über das Gedächtnis“

• then professor at Breslau and Halle

• What made Ebbinghaus famous?

Nonsense Syllables and Learning to Criterion

• Ebbinghaus used only himself as subject

• but strictly followed his experimental prescriptions

• commit CVC-trigrams (nonsense syllables) such as „gid, var, mon“ to memory

• in comparison to such syllables, memorizing a poem was approximately nine times faster

Number ofsyllables ina series

Number ofrepetitionsrequired forerrorlessreproduction

Probableerror

7 1

12 16.6 +

- 1.116 30.0 +

- 0.424 44.0 +

- 1.736 55.0 +

- 2.8

A natural science with statistical laws

Ebbinghaus, (1885/1913):• „When in repeated cases I memorised series of

syllables of a certain length to the point of their first possible reproduction, the times (or number of repetitions) necessary differed greatly from each other, but the mean values derived from them had the character of genuine constants of natural science (p. 52)

• logarithmic function of forgettingRubin & Wenzel (1996) One hundred years of forgetting.

Psychological Bulletin.• Ebbinghaus‘ results confirmed by many data sets

Human memory

• Human intelligence as adaptive behavior– for computational reasons,

memory is a necessary condition for being adaptive

• Memory system– Is it possible to identify

subsystems of memory?

– How do they interact?

• Memory contents– what information is stored?

– How is the information organized?

– How is the information retrieved

• Performance factors– Under what circumstances do

we remember less? more?

Memory in common experience

A phone numberremembered just

for dialing

A phone numberthat we have

forgotten

A well-learned phone number we

are currently thinking about

Well-learned phonenumbers we are

not currently thinking about

yes no

Memory content durable?

Memorycontent

currentlyavailable?

yes

no

Discuss

• Do the introspectively compelling properties of availability and durability provide an argument for postulating the existence of memory subsystems?

• What type of evidence would be needed to establish the existence of functionally different memory subsystems?

Classroom demonstration

dog

chair

coat

tree

mail

table

flower

pants

sofa

mouse

horse

video

salad

shoe

• Instruction– 14 words are going to be

presented to you each for 1 s.

– try to remember as many as possible

– after the presentation, write down the words you remember in any order you like

dog

chair

coat

tree

mail

table

flower

pants

sofa

mouse

horse

video

salad

shoe

Serial position effect

3 6 9 12 15serial position

recall idealized diagram!

20%

40%

60%

80%

Interpretation

3 6 9 12 15serial position

recall idealized diagram!

20%

40%

60%

80%short-termmemoryeffect

long-termmemoryeffect

Variation A of the experiment

nice

rainy

fast

bright

noble

round

blond

sunny

green

high

cheap

small

soft

tired

• Instruction– 14 words are going to be

presented to you

– try to remember as many as possible

– after the presentation, count backward from 315 for 20 sec

– then write down the words you remember

nice

rainy

fast

bright

noble

round

blond

sunny

green

high

cheap

small

soft

tire

Serial position effect: exp. A

3 6 9 12 15serial position

recall idealized diagram!

20%

40%

60%

80%

Cognitive Architecture: Memory model of Atkinson & Shiffrin (1971)

Sensory registers

visual..

auditory

Long term store

unlimited capacity

Response output

Enviromental input

Short term store

Temporyworking memory(limited capacity)

Multi-component working memory

• Baddeley (1992)– Phonological loop

– Visuo-spatial sketchpad

– Central executive

Visuo-spatial sketch pad

Phonological loop

Central executive

Experimental/Empirical Research Strategy in Cognitive Science

Human cognition as an object ofinvestigation

Computational theory

Data Hypothesis aboutspecific situations

modeling

prediction

experiment

derivation

revise

Forgetting

• Underwood (1957) reviewed forgetting over 24-hour retention interval– 80% forgotten, if 15 or more lists had been previously

learned

– 20- 25% if no earlier list had been learned

• Der Spiegel 14/1993: Wissenszwerge unter Druck: „Und von dem was bei Hörern, Lesern und Sehern ankommt, vergessen sie innerhalb von 24 Stunden mehr als 80 Prozent“. P. 150

Two-process theory of memory

• Recall = Search + Decision (i.e. retrieval + recognition)

• Recognition = Decision

• Whereas Recall involves two fallible stages, recognition involves only one

• Bahrick (1970): Pr (recall) = Pr (retrieval) * Pr (recog.)yielded good prediction of empirical results

Recognition and Recall Experiment

Four Stages• 1) Pairs of words are presented (cue + to-be-

remembered word); study second word (e.g. BLACK of word-pair; the cues (train) need not be remembered but might be helpful– train - BLACK

• 2) Freely associate to the cues which will be presented; write the words on the left side of a sheet of paper– bed - ?

• 3) Recognition task• 4) Recall task

Experiment by Tulving & Thomson (1973)

• 1) Study a word list with weak associates: pretty - BLUE

• 2) Write down what comes to mind from strong associates of the target word:– sky - ? (BLUE)

• 3) Recognition test (recall in the presence of the target):– BLUE; did it occur on list 1? Only 24 % yes answers

• 4) Cued recall test:– pretty - ? BLUE; 63 % of the words are correctly recalled

Classroom demonstration

BED

REST

AWAKE

TIRED

DREAM

WAKE

SNOOZE

BLANKET

DOZE

SLUMBER

SNORE

NAP

PEACE

YAWN

• Instruction– You will be presented with

three lists of English words

– Afterwards you will perform numerical tasks

– and your memory will be tested.

DROWSY

Classroom demonstration

STEAL

ROBBER

CROOK

BURGLAR

MONEY

COP

BAD

WOOD

CUSHION

SWIVEL

STOOL

SITTING

ROCKING

BENCH

ROB

JAIL

GUN

VILLAIN

CRIME

BANK

BANDIT

CRIMINAL

TABLE

SIT

LEGS

SEAT

COUCH

DESK

RECLINER

SOFA

Solve arithmetic problems

• 83 * 20 =

• 70 * 19 =

• 29 * 31 =

• Memory test

Recognition test on April, 21, 2002 in class; 45 participants

NO YES How confident are you: (1= not at all, 7=very)

objectively

DREAM 5 40 7 yes

FORK 45 0 7 no

WEATHER 45 0 6 no

BRACELET 45 0 5 no

CHAIR 10 35 4 lure

ROBBER 20 25 7 yes

STOOL 5 40 7 yes

TRAFFIC 45 0 6 no

Recognition test (continued)

NO YES How confident? objectively

SNOOZE 0 45 7 yes

COUCH 5 40 6 yes

RADIO 43 2 6 no

JAIL 10 35 6 yes

SLEEP 5 40 6 lure

SAND 45 0 7 no

BLANKET 25 20 6 yes

THIEF 42 3 5 lure

Bransford, Barclay & Franks (1972)

• 1a. Three turtles rested on a floating log, and a fish swam beneath them.

• 1b. Three turtles rested on a floating log, and a fish swam beneath it.

• 2a. Three turtles rested beside a floating log, and a fish swam beneath them.

• 2b. Three turtles rested beside a floating log, and a fish swam beneath it.

• Recognition test:

– When 1a was heard, false alarms on 1b

– When 2a was heard, only rare false alarms on 2b

Practical problems of memory

• Eyewitness testify that they recognize a person as having caused the accident or having committed the crime

• adults recover „repressed memories“ of sexual and/or physical abuse they suffered in childhood

• Can we trust their memory, when the event occurred a day, a month, a year, many years ago?

• Are these so called recovered memories genuine or are they false memories (i.e. the event never happened)

Influence of post-event informationLoftus & Palmer (1974)

• Memory for an incidence can be systematically distorted by questioning that occurs subsequently

• Film with multiple car accidents is shown.

• Participants describe what has happened.

• Questions– A) No question about car speed (control group).

– B) About how fast were the cars going when they hit each other?

– C) About how fast were the cars going when they smashed into each other?

• One week later all participants were asked: Did you see broken glass?

Estimated speed of car

Incorrect memories

Memory is suggestible

• Source misattribution: Believing sth that never happened

• Misinformation acceptance: additional information becoming part of a genuine experience

• Overconfidence in memory: Misremember what we have experienced

•Repeated exposure to misinformation

•Imagining that sth happened increases memory that it did happen

•Misinformation effects occur even when participants are warned beforehand

Individual differences (Tomes & Katz 1997)

• People accepting misinformation, tend to have

– Poor general memory

– High scores on imagery vividness

– High empathy scores

Bransford & Johnson (1972)

• The procedure is actually quite simple. First you arrange items into different groups. Of course one pile may be sufficient depending on how much there is to do. If you have to go somewhere else due to lack of facilities, that is the next step; otherwise, you are pretty well set. It is better not to overdo things. That is, it is better to do too few things at once than too many. In the short run this may not seem important, but complications can easily arise. A mistake can be expensive as well.

Recall and comprehension of „washing clothes“ story (Bransford & Johnson, 1972)

Summary of lecture (Memory)

• Empirical Precison: Forgetting and the differentiation betweeen short term and long term effects

• Formal Specification: The use and usefulness of computational models

• Experimental Demonstrations:– Distinction between explicit (verbal tasks) and implicit memories (behavioral

tasks) – Context effects (encoding specificity, mood congruency, transfer appropriate

processing) in recall (action oriented) and recognition (perception oriented)– „False memories“ and the distortions of memories, but also the improvement of

memory by topical information

• References: – Any recent text book on Cognitive Psychology (e.g. Eysenck & Keane, 2000,

Kellogg, 2003, or Sternberg, 2003)