the role of background knowledge in sentence processing raluca budiu july 9, 2001 thesis committee:...

The Role of Background Knowledge in Sentence Processing

Raluca Budiu

July 9, 2001

Thesis Committee:

John Anderson, Chair

Jaime Carbonell

David Plaut

Lynne Reder, Department of Psychology

Thesis Defense, July 9, 2001 2

Ambiguity of Language

My mouse behaves erratically lately.-- From an e-mail to CS facilities

Could you pass me the salt?

That's the sun of the egg. -- Child speaking about the yolk of a fried egg


Language and Noise

• Communication channels are noisy

• People make mistakesWe understand how unfair the death penalty is.

-- George W. Bush, speaking of death tax

• Listeners ignore semantic inconsistenciesWhen an aircraft crashes, where should the

survivors be buried?


Insight from this Research

Flexibility: stretching words’

meanings

Reliability: ignoring noise &

semantic inconsistencies


The Sentence-Processing Model

Model

Priorknowledge

Sentence

Noah took two animals of each kind on the ark

Napoleon was defeated at Waterloo in 1815

Plato was Socrate’s student

Sentenceinterpretation


Main Contribution

A model of language comprehension that:• Offers a unified explanation of several complex

linguistic phenomena

• Is incremental (on line)

• Is as fast as humans

• Uses prior knowledge and sentence context to understand vague words

• Is based on the ACT-R theory (Anderson & Lebiere, 1998)


ACT-R

• A cognitive architecture based on production systems

• A rigorous framework for building, running and testing computational models

• Based on verified assumptions about human cognition (e.g., memory properties, attention)

• Produces quantitative predictions about human behavior (e.g., accuracy and latency in a task)


Research Methodology

Human subjects

Experiment

Match?

Quantitative measures Quantitative measures

ComputationalACT-R model

predictions

no


Evaluation of the Model

The model• Can comprehend

– Literal or metaphoric, distorted or undistorted sentences

– Isolated or in-discourse sentences

• Can explain patterns of text recall• Compares well with people on psycholinguistic

experiments• Is fast, accurate, and scalable


Outline

• IntroductionThe sentence-processing model

– Evaluation

• Comprehension of sentences in discourse– Evaluation

• Scalability

• Future work and conclusions


The Sentence-Processing Model

Model

Background

knowledge

(words + thematic roles)

Noah took two animals of each kind on the arkNapoleon was defeated at Waterloo

in 1815

Plato was Socrate’s student

SentenceinterpretationInput sentence


Propositional Representation

take

arkanimals

Noah

Ark Prop

agent verb

place-obliquepatie

nt

Parent Ark PropChild animalsType patient

Noah took the animals on the ark


Associations


Napoleon was defeated at WaterlooNoah is Lamech’s son

PatriarchNoah

Moses

Napoleon

take

Noah

& Activation


Noah is Lamech’s son




Noah

Searchtook the animals on the arkNoahNoah

Noah

Napoleon was defeated at Waterloo

Patriarch

Moses

Napoleon

take


Noah

Matchtook the animals on the arkNoahNoah

NoahNoah took the animals on the ark



Patriarch

Moses

Napoleon

take

Noah is Lamech’s sonNoah


Noah

Matchtook the animals on the arkNoahNoah




Patriarch

Moses

Napoleon

take

Noah is Lamech’s son is

took

take


Noah

Searchtook the animals on the arkNoahNoah




Patriarch

Moses

Napoleon

take

took

take

Noah took the animals on the arkNoah took the animals on the ark


Final Interpretation



Napoleon was defeated at WaterlooNoah is Lamech’s son

PatriarchNoah

Moses

Napoleon

take


Failures of Comprehension

burnt offerings on the altar

Lamech Prop

Noah offered

Lamech Prop

No interpretation

word offeredrole verbinterpretation Lamech Prop

Bug


Summary of the Model

Read word

Bug

no

Integration

en

d o

f se

nte

nce

Interpretation?

yes

Search

no

Match?

yes


Answering True/False Queries

• False = a bug OR no final interpretation found

• True = no bug AND final interpretation found


Outline

• Introduction The sentence-processing model

– Empirical evaluation• Moses illusion

• Metaphor-position effects

• Comprehension of sentences in discourse– Evaluation

• Scalability• Future work and conclusions


Moses Illusion

• How many animals of each kind did Moses take on the ark?

• Good vs. bad distortions

How many animals of each kind did Adam take on the ark?


Moses-Illusion Data

Illusion rates for good and bad distortions (Ayers, Reder & Anderson, 1996)

• Percent correct distortions in the gist task (Ayers et al., 1996)

• Reading times in the literal and gist task (Reder & Kusbit, 1991)


Illusion rates (Ayers et al., 1996) and results of simulation

0

10

20

30

40

50

60

Undistorted Good distortions Bad distortions

Illu

sio

n R

ate

(%

)

Humans Model


Simulation of Moses Illusion

take

ark

animalsNoah

Ark Prop

agent

verb

place-oblique

patient

Moses

Adam

How many animals did Moses take on the ark

Zoo Prop Zoo Prop

Ark Prop

Adam

Zoo PropNo interpretation

Bug


Metaphor Comprehension

Effects of position on metaphor understanding (Gerrig & Healy, 1983)

• Metaphor-familiarity effects (Budiu & Anderson, 1999)

• Understanding metaphoric/literal sentences in context (Budiu & Anderson, 2000)


Metaphor Position

Stars Prop

Stars Prop

Container Prop Container Prop

Stars Prop

Drops of molten silver filled the night sky.

The night sky was filled with drops of molten silver.

4.30 s4.30 s

3.68 s3.68 s

ModelModel

3.53 s3.53 s

4.21 s4.21 s

HumansHumans


Outline

• IntroductionThe sentence-processing model

– Evaluation

Comprehension of sentences in discourse– Evaluation

• Scalability



Sentences in DiscourseCreate background knowledge from discourse propositions

King Lear had three daughtersGoneril and Regan declare their grand love

King Lear decided to divide his kingdom

Cordelia is disinherited

…Cordelia refuses to make an insincere speech

Cordelia marries the king of France

King Lear’sstory


Novel Sentences Use a partially matching interpretation to relate

to discourse

Cordelia marries the king of France

Prop 5Cordelia is disinherited

No interpretationProp 5No interpretation

word marriedInterpretation Prop 5 ……

Bug

Integration

<end>

Prop 5


Outline

• Introduction

• The sentence-processing model– Evaluation

• Comprehension of sentences in discourseEvaluation

• Scalability



Metaphor in Discourse

Experiments Metaphor vs. LiteralReading Time

Ortony et al., 1978Inhoff et al., 1984Shinjo & Myers, 1987Keysar, 1990

Gibbs, 1990Onishi & Murphy, 1993 slower

same

Comprehension

shallow

deepAnswer true/false

Comprehension(of novel sentences)

OurExperiments


Metaphoric Sentences in Context

During history seminars, a massive young man always yawned and never paid any attention to the discussions. He was a very good linebacker who had been all-state in football. The seminar always came after his training sessions, so he was very tired.

The bear slept quietlyThe bear yawned in class

Read new:

The athlete slept quietlyThe athlete yawned in class

True or false:


Metaphoric Sentences in Context

The bear yawned in class

True or false:

Find interpretationBug

Reevaluate bug

The athlete yawned in class

Find interpretation

The athlete slept quietly <end>

Interpretation No interpretation

Bug-basedintegration

The bear slept quietly

Read new:

No interpretation

bear

Bug


Outline

• Introduction

• The sentence-processing model• Evaluation

• Comprehension of sentences in discourse• Evaluation

Scalability



Computational Constraints

• Speed

• AccuracyScalability

- Word databaseSentence database


Scalability Test

• 436 noun-verb-noun sentences (Brown corpus via PennTreebank project)

• 999 distinct words

• One word repeated in at most 9 propositions• Associations based on LSA similarity measures

(Landauer & Dumais, 1997)

• Test for comprehension of a known sentence


Model PerformanceExperiment Accuracy Switches/word

Metaphor position

Moses illusion — literal

— gist > 94% < 0.72

Metaphor verification

Metaphor comprehension

Text Memory


Summary

• A model of sentence comprehension with a strong associative mechanism to speed up the search of an interpretation

• It offers a unified explanation for a variety of empirical psycholinguistic data

• It is scalable

• It is implemented in ACT-R


Future Work

• Extend the model to other empirical phenomena (e.g., priming, text inference, lexical ambiguity)

• Identify the ACT-R assumptions that are fundamental

• Eliminate some of the limitations


Conclusions

• Context can help the comprehension of metaphoric or semantically-flawed sentences

• Semantic associations between words are a powerful mechanism that allows fast and flexible comprehension

• “Peripheral” language phenomena can shedlight on deep cognitive processes


Limitations of the Model• No syntactic processing• Atomic word-phrases (e.g., drops of molten silver)

• Rudimentary discourse processing• Cannot account for sentences containing similar words

(e.g., George W.Bush is the son of George Bush)

• Relationship between discourse and background knowledge

• Similarities not from ratings• No thematic-role cues


Metaphor in Discourse

Experiments Metaphor vs. LiteralReading Time

Ortony et al., 1978Inhoff et al., 1984Shinjo & Myers, 1987Keysar, 1990

Gibbs, 1990Onishi & Murphy, 1993 slower

same

Comprehension

shallow

deep

the role of background knowledge in sentence processing raluca budiu july 9, 2001 thesis committee:...

Documents