emergence of semantics from experience
DESCRIPTION
Emergence of Semantics from Experience. Jay McClelland Department of Psychology and Center for Mind, Brain, and Computation Stanford University. 1. language. The Parallel Distributed Processing Approach to Semantic Cognition. - PowerPoint PPT PresentationTRANSCRIPT
Emergence of Semantics from Experience
Jay McClelland
Department of Psychology andCenter for Mind, Brain, and Computation
Stanford University
1
• Representation is a pattern of activation distributed over neurons within and across brain areas.
• Bidirectional propagation of activation underlies the ability to bring these representations to mind from given inputs.
• The knowledge underlying propagation of activation is in the connections.
• Experience affects our knowledge representations through a gradual connection adjustment process
language
The Parallel Distributed Processing Approach to Semantic Cognition
2
Distributed Representations:and Overlapping Patterns for Related
Concepts
dog goat hammer
dog goat hammer
3
Emergence of Meaning in Learned Distributed Representations
• Learned distributed representations that capture important aspects of meaning emerge through a gradual learning process in simple connectionist networks
• The progression of learning captures several aspects of cognitive development:– Differentiation of Concepts– Illusory Correlations– Overgeneralization– And many other things
4
5
The Rumelhart Model
6
The Training Data:
All propositions true of items at the bottom levelof the tree, e.g.:
Robin can {grow, move, fly}
7
Target output for ‘robin can’ input
8
aj
ai
wij
neti=ajwij
wki
Forward Propagation of Activation
9
k ~ (tk-ak)
wij
i ~ kwki
wki
aj
Back Propagation of Error ()
Error-correcting learning:
At the output layer: wki = kai
At the prior layer: wij = jaj
…
ai
10
11
12
Experience
Early
Later
LaterStill
13
Why Does the Model Show Progressive Differentiation?
• Learning is sensitive to patterns of coherent covariation
• Coherent Covariation:
– The tendency for properties of objects to co-vary in clusters
• Figure shows attribute loadings on the principal dimensions of covariation. These capture:
– 1. Plants vs. animals– 2. Birds vs. fish– 3. Trees vs. flowers
• Same color = features that covary
• Diff color = anti-covarying features
14
Trajectories of Concept Representations During Differentiation
15
Illusory Correlations
16
A typical property thata particular object lackse.g., pine has leaves
An infrequent,atypical property
17
Overgeneralization of Frequent Names to Similar Objects
“dog”
“goat”“tree”
18
19
Other Applications of the Model
• Expertise effects
• Conceptual reorganization
• Effects of language and culture
• Effects of brain damage:
– Loss of differentiation– Overgeneralization in
object naming– Illusory correlations
camel swan 20
Conclusion
• We represent objects using patterns of activity over neuron-like processing units
• These patterns depend on connection weights learned through experience
• Differences in experience lead to differences in conceptual representations.
21