pdp models of morphology psych 419/719 april 3, 2001
TRANSCRIPT
PDP Models of Morphology
Psych 419/719
April 3, 2001
What is Morphology?
• Derives from MORPH, meaning “to change”
• Defines the rules of a language governing how words can be changed into new words.
Different Kinds of Morphology
• Inflectional– Plural (DOG->DOGS)– Past tense (BUG->BUGGED)– 3rd person singular (LIKE->LIKES)
• Derivational:– GOVERN->GOVERNMENT– POMPOUS->POMPOSITY
• Compound Words:– Houseboat, boathouse
• Infixing
The General Character...
Sound
Meaning
Cat
Meows, purrs
s+
plural+
Exceptions are Legion..
• Plural: MICE, MEN, GEESE, KNIVES• Past tense: RAN, LIT• Derivational: there’s GOVERN in
GOVERNMENT, but no DEPART in DEPARTMENT
• Compounds: A bookcase is a container for books, a suitcase is a container for suits, but a staircase isn’t really a container for stairs, a pocketbook isn’t a book, a bookworm isn’t a worm, etc.
Despite This, We Can Generalize
• WUGS are either more than one WUG, or the act of the verb WUG (e.g., “Bob wugs his car”)
• GLORPED is doing GLORP in the past
• ESTRANGEMENTALITY is the processes of ESTRANGEMENT
The Standard Account• Part of your knowledge of your language is
knowing the rules that govern such transformations
• Rules operate over standard linguistic units – Stems, affixes, suffixes, etc.
• Knowledge of rules is independent of knowledge of mapping from sound to meaning
• The exceptions must be memorized by a separate system
What PDP Networks Are Like
• They can memorize exceptions, but attend to statistical regularities as well.
• They’re good at such tasks, but not as good at partitioning training into qualitatively different categories, like rules and exceptions
The PDP Story
• You learn to associate word forms to their meanings
• Regularities are easy to learn– So you pick up on them,– And languages evolve to use them.
• “Morphology” is simply attending to these regularities
Non-Morphological Regularities• Onomatopoeia - where the sound of a word has
something to do with its meaning– whistle, whir, whip, whiz– boom, bang, clang
• Words with similar sounds mean similar things– glitter glisten, gleam– sparkle, sputter, sprite
• Common Latin root– include, exclude, preclude
.. Leads to Different Interpretations
• On the standard account, these kinds of regularities are not “morphological” and as such are not handled by the morphological system
• On the PDP account, they’re just another regularity for the sound to meaning system to learn. Less predictable, but still there.
Other Effects in Morphology
• Claim: for some items, you regularize the inflection if the meaning is far from the original meaning– The batter flied out to left field– Maple Leafs vs. Timber Wolves
• Sometimes the reverse:– The suspect has been held up in the apartment
for 6 hours (c.f. HOLED/HOLD)
U-Shaped Learning
• Initially, children seem to learn word forms whether they are regular or exception
• At some point, performance on exceptions drops.. Children regularize them (saying “eated” for “ate”)
• Ultimately, performance recovers
The Standard Account of U-Shaped learning
• Initially children are memorizing word forms.
• Then, they infer the rule for morphology
• This results in interference between memorized forms and rule-generated forms– Competition between two systems
• Eventually, rule is learned, exceptions are put in “exception box”
The PDP Account of U-Shaped Learning
• A single homogenous system is learning the task
• Interference between regular and irregular forms is not competition between two separate, atomic systems
• … but rather, results from the normal dynamics of learning in a single system
The Facts of U-Shaped Learning
• When you look closely at children’s performance, there isn’t a global “switch” from good to poor performance on all exceptions– As might be predicted by standard account
• Rather, there are micro-U trends by item over development
• And by the way, it doesn’t happen that often...
Micro-U Shaped Learning
• This falls out naturally from the PDP account.
• Not quite as straightforward for the standard account.
Review: Rumelhart & McClelland’s Past Tense Model
• Created a two layer network to map uninflected forms to inflections.– Initially, introduced high frequency (mostly
exception) words in training.– Then, switched to whole training set
• One system learned exceptions and regulars
• Demonstrated U-Shaped learning at a global scale
Criticisms of R&M Past Tense Model
• The frequency shift was bogus
• The representation was poor
• The errors it made were implausible
• Didn’t account for semantic effects
• The U-Shaped learning it modeled isn’t what actually happens
• Wrong theory: people aren’t just inflecting base forms when making the past tense in normal language
Another Try:Plunkett & Marchman ‘91
• Used more reasonable phonological representation
• Did not introduce explicit frequency shift
• Found that parameters for type/token frequency to induce best match to people was that of actual English
• … But, had very few items (500), and did not master vocabulary early in training
One More Time:Plunkett and Marchman ‘93
• Increased training set size gradually, one item at a time.
• Two conditions:– Add new item when existing ones mastered.
Result: Got Stuck– Add new item when certain amount of time
passed. Result: Much better
Stem To Inflected Word:Summary
• Models were able to reproduce U-Shaped learning with a high degree of fidelity to what children do
• Provided an account of effect of vocabulary size on interference and generalization that is absent in standard account
A New Attack:Marslen-Wilson and Tyler
• Looked at priming effects in lexical decision– You get a prime such as BAKE, then have to
make lexical decision on BAKER
• Crossed semantic overlap with phonological overlap
Marslen-Wilson’s Manipulation• Semantic, Phonological, Morphological related
– bake / baker
• Phonological but not semantic– corn / corner
• Semantic but not phonological– cook / baker
• Result: only morphological condition primed reliably
• Conclusion: Morphology is special
The Counter to Marslen-Wilson:Gonnerman ‘98
• Marslen-Wilson observed weak (ns) priming in other conditions
• Maybe just phonological or semantic isn’t enough; need both
• M-W’s semantic primes weren’t very closely related
• Should get graded priming if items sufficiently related, and enough subjects
Gonnerman’s Results
• If semantic relatedness is high, get priming even with no morphological relationship– Jubilee - jubilant, fork - spoon
• Priming effects are in fact graded: semantic overlap gives some priming, semantic and phonological gives more
• No effect of morphology independent of semantic and phonological overlap!
The Gonnerman & Devlin Model
• Maps forms onto meaning• Manipulated phonological
and semantic overlap• Results broadly replicated
that of empirical study
Word Form
Meaning
Brain Damage Can YieldDouble Dissociations
• Some patients exhibit an impairment in generating irregular past tenses – Say runned instead of ran
• While others have an impairment in generating novel past tenses– Can’t say glorped for the past tense of glorp
• Taken as evidence (Ullman and colleagues) for two systems in morphology
The Patients
• Those impaired on rules:– Patients with Parkinson’s disease, or left
inferior cortex (including Broca’s area)
• Those impaired on exceptions:– Patients with damage to Wernike’s area, either
from Alzheimer’s disease or lesion
The Two Accounts
• The “rules” are in the left inferior frontal area; Broca’s or the basal ganglia
• The “exceptions” are part of declarative memory, near Wernike’s area
• Phonological knowledge is needed more for rule-like items
• Semantic knowledge is needed more for exception processing
Standard Account PDP Account
The Joanisse & Seidenberg Model
• Four tasks:– Speaking
– Hearing
– Repeating
– Past tense formation
Semantics Speech in
Speech Out
Modeled rule impairmentwith phonological damage
And exception impairmentwith semantic damage
Results of Simulation
• Phonological damage impaired rule performance more than exceptions
• Semantic damage impaired exceptions more than rule performance
Another Angle: Why Do WeGeneralize At All?
• In English, the regulars are by far more common than exceptions. Easy to decide it’s the default.
• But: in German, there are several forms of the past tense. The “default” is low in frequency– But: Other forms are phonologically conditioned!
The Case of the Rats Eaters
• People can form compound nouns, by glueing two words together (rat eater, pig farmer, etc).
• It has been observed that people don’t like to use plurals as the head of a compound– rat eater (ok)– mice eater (ok)– rats eater (bad)
The Standard Account: The Level-Ordering Hypothesis
• Rules are applied at different levels of representation
• Each level doesn’t have access to input to previous levels– Irregular inflections stored in lexicon– Compounding applies to items in lexicon– Regular inflection happens after compounding
Problems With This Account
• People use regular compounds in plurals all the time– parks department, weapons inspector, pilots
union, communications industry, compounds research
• If compounds are generated by rule, acceptability shouldn’t be frequency sensitive– But it is
The PDP-Inspired Alternative
• There are cues to what is acceptable• Account hinges on what makes a good
modifier– Children learn that modifiers generally aren’t
semantically or phonologically plural– Consider adjectives: red balloons, not reds
balloons– Irregulars are semantically but not phonologically
plural, so not great, but better than pure plurals
Next Time: Reading and Dyslexia
• Optional reading on the class web page
• No class April 10