knowledge: representation and manipulation

REPRESENTA

TION A

ND

MANIPULA

TION O

F

KNOWLE

DGE IN M

EMORY:

IMAGES A

ND

PROPO

SITIO

NS

CH

AP

TE

R 7

CO

GN

I TI V

E P

SY C

HO

L OG

Y

CHAPTER OUTLINE

Mental Representation of Knowledge

Mental Manipulation of Images

Synthesizing Images and Propositions

Spatial Cognition and Cognitive Maps

MENTAL R

EPRESENTA

TION

OF KNOW

LEDGE

KNOWLEDGE REPRESENTATION

• The form for what you know in your mind about things, ideas, events, and so on, in the outside world.


• Introspectionist Approach•We can ask people to describe their own knowledge representations and knowledge-representation processes: What do they see in their minds when they think of the Statue of Liberty, for example?

•Unfortunately, none of us has conscious access to our own knowledge-representation processes and self-reported information about these processes is highly unreliable (Pinker, 1985).


• Rationalist Approach•We try to deduce logically how people represent knowledge.

• In classic epistemology—the study of the nature, origins, and limits of human knowledge—philosophers distinguished between two kinds of knowledge structures.

1. Declarative knowledge – refers to facts that can be stated, such as the date of your birth, the name of your bestfriend, or the way a rabbit looks.


2. Procedural knowledge – refers to knowledge of procedures that can be implemented. Examples are the steps involved in tying your shoelaces, adding a column of numbers, or driving a car.

The distinction is between knowing that and knowing how (Ryle, 1949).


• Two main sources of empirical data on knowledge representation:

•Standard laboratory experiment•Neuropsychological studies


• In experimental work, researchers indirectly study knowledge representation because they cannot look into people’s minds directly.

• They observe how people handle various cognitive tasks that require the manipulation of mentally represented knowledge.


• In neuropsychological studies, researchers typically use one of two methods:

• They observe how the normal brain responds to various cognitive tasks involving knowledge representation.• They observe the links between various deficits in knowledge representation and associated pathologies in the brain.


• In the following sections, we explore some of the theories researchers have proposed to explain how we represent and store knowledge in our minds:

• First, we consider what the difference is between images and words when they are used to represent ideas in the outside world.

• Then we learn about mental images and the idea that we store some of or knowledge in the form of images.


• In the following sections, we explore some of the theories researchers have proposed to explain how we represent and store knowledge in our minds:

•Next, we explore the idea that knowledge is stored in the form of both words and images (dual-code theory).

• Finally, we consider an alternative—propositional theory—which suggests that we actually use an abstract form of knowledge encoding that makes use of neither words nor mental images.


•Knowledge can be represented in different ways in your mind: It can be stored as

•Mental picture•Words•Abstract propositions

COMMUNICATING KNOWLEDGE: PICTURES VS. WORDS

• External representation – ideas in the outside world, such as a book.

•Some ideas are better and more easily represented in pictures, whereas others are better represented in words.

• Example: “What is the shape of the egg?” “What is justice?”

The picture is relatively analogous (similar) to the real-world object it represents. The picture shows concrete attributes, such as shape and relative size.


• Symbolic representation – the relationship between the word and what it represents is simply arbitrary (based on random choice or personal whim, rather than any reason or system.)• Capture some kinds of information but not other kinds of information.

• In forming words, the sounds or letters also must be sequenced according to rules (e.g, “c-a-t,” not “a-c-t’ or “t-a-c”).

• In forming sentences, the words also must be sequenced according to rules. • “The cat is under the table.”• “Table under cat the is.”


• Pictures, aptly capture concrete and spatial information in a manner analogous to whatever they represent. They convey all features simultaneously.• In general, any rules for creating or understanding pictures pertain to the analogous relationship between the picture and what it represents.

•Words, handily capture abstract and categorical information in a manner that is symbolic of whatever the words represent. • Representations in words usually convey information sequentially.

PICTURES IN YOUR MIND: MENTAL IMAGERY

• Internal representation •How do we represent what we know in our minds.

• Imagery – is the mental representation of things that are not currently seen or sensed by the sense organs.

PICTURES IN YOUR MIND: MENTAL IMAGERY• In our minds we often have images for objects, events, and settings.

•Mental imagery even can represent things that you have never experienced.

•Mental images even may represent things that do not exist at all outside the mind of the person creating the image.


• Imagery may involve mental representations in any of the sensory modalities, such as hearing, smell, or taste.

• Imagine the sound of fire alarm, your favorite song, or your nation’s anthem. Now, imagine the smell of a rose, of fried bacon, or of an onion. Finally, imagine the taste of a lemon, pickle, or your favorite candy.


•Most research on imagery has focused on visual imagery, such as representations of objects or settings that are not presently visible to the eyes.

•Most of us are more aware of visual imagery than of other forms of imagery.

•We use visual image to solve problems and to answer questions involving objects. •Which is darker red—a cherry or an apple?•How many windows are there in your house?•How do you get from your home to your first class of the day?


• Using guided-imagery techniques for controlling pain and for strengthening immune responses and otherwise promoting health.• Imagine being at a beautiful beach and feeling very comfortable, letting your pain fade into the background.• Imagine the cells of your immune system successfully destroying all the bad bacteria in your body.


• Such techniques are also helpful in overcoming psychological problems, such as phobias and other anxiety disorders.

•Design engineers, biochemists, physicists, and many other scientists and technologists use imagery to think about various structures and processes and to solve problems in their chosen field.


• Research also indicates that the use of mental images can help to improve memory.

•Mental imagery also is used in other fields such as occupational therapy. Using these techniques, patients with brain damage train themselves to complete complex tasks.


• In what form do we represent images in our minds?• All images of everything we ever sense may be stored as exact copies of physical images.• But realistically, to store every observed physical image in the brain seems impossible.

• Amazingly, learning can indeed take place just by using mental images.

DUAL-CODE THEORY: IMAGES AND SYMBOLS

• Dual-code theory – we use both pictorial and verbal codes for representing information in our minds.

• According to Paivio, mental images are analog codes.

• Analog codes – resemble the objects they are representing.

• Example: trees and rivers might be represented by analog codes

movement of the hands on an analog clock are analogous to the passage of time.

The mental images we form in our minds are analogous to the physical stimuli we observe.


• In contrast, our mental representations for words chiefly are represented in a symbolic code.

• Symbolic code is a form of knowledge representation that has been chosen arbitrarily to stand for something that does not perceptually resemble what is being represented.


• A symbol may be anything that is arbitrarily designated to stand for something other than itself.•We recognize that the numeral “9” is a symbol for the concept of “nineness.” It represents a quantity of nine of something.•Concepts like justice and peace are best represented symbolically.

DUAL-CODE THEORY: IMAGES AND SYMBOLS• Paivio, consistent with his dual-code theory, noted that verbal information seems to be processed differently than pictorial information.

• Participants more easily recalled the pictures when they were allowed to do so in any order. But they more readily recalled the sequence in which the words were presented than the sequence for the pictures, which suggests the possibility of two different systems for recall of words versus pictures.

ACTIVIT

Y #1


• Analogical and Symbolic Representation of Cats• To get an intuitive sense of how you may use each of the two kinds

of representations, think about how you mentally represent all the facts you know about cats. Use your mental definition of the word cat and all the inferences you may draw from your mental image of a cat. Which kind of representations is more helpful for answering the following questions:

• Is a cat’s tail long enough to reach the tip of the cat’s nose if the cat is stretching to full length?• Do cats like to eat fish?• Are the back legs and the front legs of a cat exactly the same size

and shape?• Are cats mammals?• Which is wider—a cat’s nose or a cat’s eye?

• Which kind of mental representations were most valuable for answering each of these qustions?

ACTIVIT

Y #2


•Dual Coding• Look at the list of words your friends and family members recalled in the demonstration in Chapter 6. Add up the total number of recollections for every other word (i.e., book, window, box, hat, etc.—the words in odd-numbered positions in the list). Now add up the total number of recollections for the other words (i.e., peace, run, harmony, voice, etc.—the words in even-numbered positions on the list).


•Dual Coding•Most people will recall more words from the first set than from the second set. This is because the first set is made up of words that are concrete, or those words that are easily visualized. The second set of words is made up of words that are abstract, or not easily visualized.

• This is a demonstration of the dual-coding hypothesis (or its more contemporary version, the functional-equivalence hypothesis).

STORING KNOWLEDGE AS ABSTRACT CONCEPTS: PROPOSITIONAL THEORY

• Propositional theory suggests that we do not store mental representations in the form of images or mere words.

•We may experience our mental representations as images, but these images are epiphenomena—secondary and derivative phenomena that occur as a result of other more basic cognitive processes.

• According to propositional theory, our mental representations (sometimes called “mentalese”) more closely resemble the abstract form of a proposition.

STORING KNOWLEDGE AS ABSTRACT CONCEPTS: PROPOSITIONAL THEORY

• A proposition is the meaning underlying a particular relationship among concepts.

WHAT IS A PROPOSITION?

• UNDER ( CAT, TABLE) - logical expression

• [Relationship between elements] ([Subject element], [Object element])

• Verbal representation:• “The table is above the cat”• “The cat is beneath the table.”• “Above the cat is the table.”

• Logicians have devised a shorthand means, called “predicate calculus,” of expressing the underlying meaning of a relationship. It attempts to strip away the various superficial differences in the ways we describe the deeper meaning of a proposition.

USING PROPOSITIONS

•We may use propositions to represent any kind of relationship, including actions, attributes, spatial positions, class membership, or almost any other conceivable relationship. The possibility for combining propositions into complex propositional representational relationships makes the use of such representations highly flexible and widely applicable.

PROPOSITIONAL REPRESENTATIONS OF UNDERLYING MEANINGS

• Type of Relationship• actions

•Representation in Words•A mouse bit a cat.

• Propositional Representation•Bite [action](mouse[agent of action], cat [object]

• Imaginal Representation


• Type of Relationship• attributes

•Representation in Words•Mice are furry.

• Propositional Representation• [external surface characteristic] (furry [attribute], mouse [object]



• Type of Relationship•Spatial positions

•Representation in Words•A cat is under the table.

• Propositional Representation• [vertically higher position] (table, cat)



• Type of Relationship•Class or category membership

•Representation in Words•A cat is an animal.

• Propositional Representation• [categorical membership] (animal [category], cat [member])


ACTIVIT

Y #3

LIMITATIONS OF MENTAL IMAGE

•Mental Images•Quickly glance at this figure and then cover it with your hand. Imagine the figure you just saw. Does it contain a parallelogram?

ACTIVIT

Y #4

CAN MENTAL IMAGES BE AMBIGUOUS?(a) Look closely at the rabbit, then cover it with your hand and

recreate it in your mind. Can you see a different animal in this image just by mentally shifting your perspective?

(b) What animal do you observe in this figure? Create a mental image of this figure, and try to imagine the front end of this animal as the back end of another animal and the tail end of this animal as the front end of another animal.

(c) Observe the animal in this figure, and create a mental image of the animal; cover the figure, and try to reinterpret your mental image as a different kind of animal (both animals probably are facing in the same direction).

CAN MENTA

L IMAGES

BE

AMBIGUOUS?

AMBIGUOUS FIGURE

• Can be interpreted in more than one way.•Often used in studies of perception.

•Without looking back at the figure, can you determine the alternative interpretation of Figure 7.6 (a)?

AMBIGUOUS FIGURE

The alternative interpretation of the rabbit is a duck. In this interpretation, the rabbit’s ears are the duck’s bill.

LIMITATIONS OF PROPOSITIONAL THEORY

• The mental reinterpretation of ambiguous figures involves two manipulations:1. Mental realignment of the reference frame. This

realignment would involve a shift in the positional orientations of the figures on the mental page or screen on which the image is displayed.

2. Mental reconstrual (reinterpretation) of parts of the figure.


• Across experiments, 20% to 83% of participants were able to reinterpret ambiguous figures, using one or more of the following hints:1. Implicit reference-frame hint. Participants were

shown another ambiguous figure involving realignment of the reference frame; ex. A hawk’s head/a goose’s tail, and a hawk’s tail/a goose’s head.

2. Explicit reference-frame hint. Participants were asked to modify the reference frame by considering either “the back of the head of the animal they had already seen as the front of the head of some other animal” (conceptual hint); “the front of the thing you were seeing as the back something else” (abstract hint).


• Across experiments, 20% to 83% of participants were able to reinterpret ambiguous figures, using one or more of the following hints:

3. Attentional hint. Participants were directed to attend to regions of the figure were realignments or reconstruals were to occur.

4. Construals from “good” parts. Participants were asked to construe an image from parts determined to be “good” (according to both objective [geometrical] and empirical [inter-rater agreement] criteria), rather than parts to be “bad” (according to similar criteria.)

• Additionally, some spontaneous reinterpretation of mental images for ambiguous figure may occur. This is particularly likely for images of figures that may be reinterpreted without realigning the reference frame.

• Figure (c), which may be a whole snail or an elephant’s head, or possibly even a bird, a helmet, a leaf, or a seashell.

THE INFLUENCE OF SEMANTIC LABELS

• Visual images can be distorted through verbal information.

• Participants were asked to view figures that were labeled. When they recalled the images, they were distorted in the direction of the meaning of the images.

• Semantic (verbal) information (e.g., labels for figures) tends to distort recall of visual images in the direction of the meaning of the images.

• Recall differs based on the differing labels given for the figures.

THE

INFL

UENCE O

F

SEMANTIC L

ABELS

MENTAL M

ANIPULA

TIONS

OF IM

AGES

FUNCTIONAL-EQUIVALENCE HYPOTHESIS

• Although visual imagery is not identical to visual perception, it is functionally equivalent to it.

• Functionally equivalent things are strongly analogous to each other—they can accomplish the same goals.

• This view essentially suggests that we use images rather than propositions in knowledge representations for concrete objects that can be pictured in the mind.

PRINCIPLES OF VISUAL IMAGERY

• According to the functional-equivalence hypothesis, we represent and use visual imagery in a way that is functionally equivalent (strongly analogous) to that for physical percepts.

•Ronald Finke has suggested several principles of visual imagery that may be used to guide research and theory development.

PRINCIPLES OF VISUAL IMAGERY

1. Our mental transformations of images and our mental movements across images correspond to those of physical objects and percepts.

2. The spatial relations among elements of a visual image are analogous to those relations in actual physical space.

3. Mental images can be used to generate information that was not explicitly stored during encoding.

4. The construction of mental images is analogous to the construction of visually perceptible figures.

5. Visual imagery is functionally equivalent to visual perception in terms of the processes of the visual system used for each.

NEUROSCIENCE AND FUNCTIONAL EQUIVALENCE

• Schizophrenia provides an interesting example of the similarities between perception and imagery.

• Auditory hallucinations are experiences of “hearing” that occur in the absence of actual auditory stimuli.

• Evidence from other researchers reveals that during auditory hallucinations there is abnormal activation of the auditory cortex.

• In sum, it is believed that auditory hallucinations occur at least in part because of malfunctions of the auditory imaging system and problematic perception processes.

• These results suggest that there is indeed functional equivalence between what our senses perceive and what we create in our minds.

MENTAL ROTATION

• Involves rotationally transforming an object’s visual mental image.

MENTAL ROTATION

• For which of these pairs of figures does the figure on the right show an accurate rotation of the figure on the left?

• The forms were rotated from 0 to 180 degrees. • Picture plane (2-D space clockwise or counterclockwise)•Depth (3-D space)•Distracter form

MENTAL ROTATION • There was no significant difference between rotations in the picture plane and rotations in depth.

• To rotate objects at larger angles of rotation takes longer.

•Mental rotation may be an automatic process. • Such automatic processes may be a sign of more effective visuospatial skills because increased speed is associated with increased accuracy in spatial memory.

• Aging affects some aspects of visual imagery more than others.

INTELLIGENCE AND MENTAL ROTATION

• Primary Mental Abilities test of Louise and Thelma Thurstone (1962) requires mental rotation of two-dimensionally pictured objects in the picture plane.

• It has identified the mental representations and cognitive processes that underlie adaptations to the environment and thus, ultimately, that constitute human intelligence.

NEUROSCIENCE AND MENTAL ROTATION• Mental rotation also indicates that the motor cortex (areas in the posterior frontal cortex) is activated during this task. The areas associated with hand movement were particularly active during the mental rotation task.

• Areas of the cerebral cortex have representations that resemble the 2-D spatial arrangements of visual receptors in the retina of the eye. These mappings may be construed as relatively depictive of the visual arrays in the real world.

• The same brain areas involved in perception also are involved in mental rotation tasks.

GENDER AND MENTAL ROTATION• In young children, there is no gender differences either in performance or in neurological activation.

• There seem differences in the activation of the parietal regions between men and women. There is less parietal activation for women than for men completing the same mental rotation task.

• In women, spatial tasks involve both sides of the brain, whereas in men, the right side dominates this function.

• The differences in brain activation may mean that men and women use different strategies to solve mental rotation problems.

•Women have a proportionally greater amount of gray matter in the parietal lobe than do men, which is associated with a performance disadvantage for mental rotation tasks for the women (as they need increased effort to complete the tasks).

ZOOMING IN ON MENTAL IMAGES: IMAGE SCALING

•We use mental images the same way we use our actual perceptions.•When you look at a building from afar, you won’t be able to see as many details as when you are close by, and you may not be able to see things as clearly.

• In general, seeing details of large objects is easier than seeing such details of small ones. We respond more quickly to questions about large objects we observe than to questions about small ones we observe.

ACTIVIT

Y #5

IMAGE SCALING

• Find a large bookcase (floor to ceiling, if possible; if not, observe the contents of a large refrigerator with an open door). Stand as close to the bookcase as you can while still keeping all of it in view. Now, read the smallest writing on the smallest book in the bookcase. Without changing your gaze, can you still see all of the bookcase? Can you read the title of the book farthest from the book on which you are focusing your perception? Depending on what you want to see (a detail like a book title or the whole shelf), you may have to zoom in and out of what you see. When you look at a small detail, it will be hard to perceive the whole shelf, and vice versa. The same is true for mental images.

IMAGE SCANNING

• Look at the rabbit and the fly in figure 7.10. Close your eyes and picture them both in your mind. Now in your imagination, look only at the fly and determine the exact shape of the fly’s head. Do you notice yourself having to take time to zoom in to “see” the detailed features of the fly? If you are like most people, you are able to zoom in on your mental images to give the features or objects a larger portion of your mental screen, much as you might physically move toward an object you wanted to observe more closely.• Now, look at the rabbit and the elephant and picture them both in your mind. Next, close your eyes and look at the elephant. Imagine walking toward the elephant, watching it as it gets closer to you. Do you find that there comes a point when you can no longer see the rabbit or even all of the elephant? If you are like most people, you will find that the image of the elephant will appear to overflow the size of your image space. To “see” the whole elephant, you probably have to mentally zoom out again.

ZOOMING IN ON DETAILS

MENTAL SCANNING: AN IMAGINARY ISLAND

REPRESENTATIONAL NEGLECT

• A person asked to imagine a scene and then describe it ignores half of the imagined scene.

SYNTH

ESIZIN

G IMAGES

AND PROPO

SITIO

NS

JOHNSON-LAIRD’S MENTAL MODELS

•Mental models are knowledge structures that individuals construct to understand and explain their experiences.

LEFT BRAIN OR RIGHT BRAIN: WHERE IS INFORMATION MANIPULATED?

• The right hemisphere appears to represent and manipulate visuospatial knowledge in a manner similar to perception.• Also represents knowledge in a manner that is analogous to our physical environment.

• The left hemisphere appears to be more proficient in representing and manipulating verbal and other symbol-based knowledge.• Also has the ability to manipulate imaginal components and symbols and to generate entirely new information.

TWO KINDS OF IMAGES: VISUAL VERSUS SPATIAL

•Visual imagery refers to the use of images that represent visual characteristics such as colors and shapes.

• Spatial imagery refers to images that represent spatial features such as depth dimensions, distances, and orientations.

SPATI

AL COGNIT

ION A

ND

COGNITIV

E MAPS

SPATIAL COGNITION

•Deals with the acquisition, organization, and use of knowledge about objects and actions in two- and three-dimensional space.

COGNITIVE MAPS

• Are internal representations of our physical environment, particularly centering on spatial relationships.

COGNITIVE MAPS

• Rats• Edward Tolman•He argued for the importance of the mental representations that give rise to behavior.

• The ability of rats to learn a maze.

COGNITIVE MAPS

• Bees•Nobel Prize-winning German scientist•Studied the behavior of bees when they return to their hive after having located a source of nectar.

COGNITIVE MAPS

• Pigeons•Noted for their excellent cognitive maps.• These birds are known for their ability to return to their home from distant locations.

COGNITIVE MAPS

•Hippocampus is involved in the formation of cognitive maps in humans as well.

• Left hippocampus plays a pivotal role in map formation.• Also crucial for the perception of landmarks within the environment.

•Right hippocampus is involved in sensitivity to global features of the environment.

COGNITIVE MAPS• Humans seem to use three types of

knowledge when forming and using cognitive maps:

1. Landmark knowledge is information about particular features at a location and which may be based on both imaginal and propositional representations.

2. Route-road knowledge involves specific pathways for moving from one location to another. It may be based on both procedural knowledge and declarative knowledge.

3. Survey knowledge involves estimated distances between landmarks, much as they might appear on survey maps. It may be represented imaginally or propositionally.

RULES OF THUMB FOR USING MENTAL MAPS: HEURISTICS•When we use landmark, route-road, and survey knowledge, we sometimes use rules of thumb that influence our estimations of distance. These rules of thumb are cognitive strategies termed heuristics.

• Allow people to solve problems and make judgments quickly and efficiently, but they are also prone to errors.

RULES OF THUMB FOR USING MENTAL MAPS: HEURISTICS• Right-angle bias: people tend to think of intersections (street

crossings) as forming 90-degree angles more often than the intersections really do.

• Symmetry heuristic: people tend to think of shapes (states or countries) as being more symmetrical than they really are.

• Rotation heuristic: when representing figures and boundaries that are slightly slanted (oblique), people tend to distort the images as being either more vertical or more horizontal than they really are.

• Alignment heuristic: people tend to represent landmarks and boundaries that are slightly out of alignment by distorting their mental images to be better aligned than they really are.

• Relative-position heuristic: the relative positions of particular landmarks and boundaries is distorted in mental images in ways that more accurately reflect people’s conceptual knowledge about the contexts in which the landmarks and boundaries are located, rather than reflecting the actual spatial configuration.

• Sometimes, to get between places, we need to imagine the route we will need to traverse.

•Mental imagery provides a key basis for this adaptation.

• Thus, our imagery abilities are potential keys to our survival and to what makes us intelligent in our everyday lives.

1.What are some practical applications of having two codes for knowledge representation? Give an example applied to your own experiences, such as applications to studying for examinations. (10 pts)

2.Create a cognitive map of your house from your school campus using the three types of knowledge. (10 pts)a. Landmarkb. Route-roadc. survey

1. What is a concept?2. What is category?3. What is the difference between prototypes and

examplars?4. What is the theory-based view of meaning?5. What are the components of a semantic network?6. What is a schema?7. Why do we need scripts?8. What is procedural knowledge?9. What are the different kinds of nondeclarative knowledge?10.What are the two types of priming?11.What is the ACT-R model?12.How is procedural knowledge represented in the ACT-R

model?13.What is parallel processing?14.How does a connectionist network represent knowledge?15.What is domain specificity?

knowledge: representation and manipulation

Documents