Chapter 2:Chapter 2:Visual PerceptionVisual Perception

PowerPoint by Glenn E. Meyer, Trinity UniversityPowerPoint by Glenn E. Meyer, Trinity University

©2004 Prentice Hall ©2004 Prentice Hall

Cognition – 2/eCognition – 2/eDr. Daniel B. WillinghamDr. Daniel B. Willingham

22©2004 Prentice Hall

What Makes Visual Perception Hard?What Makes Visual Perception Hard?

Vision is hard as the pattern of light that Vision is hard as the pattern of light that falls on your retina is consistent with with falls on your retina is consistent with with many different scenes.many different scenes.

For example, is the For example, is the figure to the left a figure to the left a square or a cube square or a cube face on? Maybe it is face on? Maybe it is the base of a four the base of a four sided pyramid?sided pyramid?

33©2004 Prentice Hall

Visual Perception Hard - Continued?Visual Perception Hard - Continued?We will have to deal with ambiguities.We will have to deal with ambiguities.This is done by the visual system making assumptions.This is done by the visual system making assumptions.So what is the visual system for?So what is the visual system for?

1) It identifies objects1) It identifies objects

2) It helps us navigate in the world2) It helps us navigate in the world

44©2004 Prentice Hall

Visual Perception Hard - Continued?Visual Perception Hard - Continued?Chief problem of the visual system is the Chief problem of the visual system is the inverse inverse

projection problemprojection problem : :The problem of recovering three-dimensional shape from a two-dimensional The problem of recovering three-dimensional shape from a two-dimensional projection, like the projection on the retina. A two dimensional project may projection, like the projection on the retina. A two dimensional project may represent different three dimensional objects. represent different three dimensional objects.

Can you see the Necker Cube on the Left Can you see the Necker Cube on the Left as either of the cubes below?as either of the cubes below?

OROR

??

Thus, visual system must deal with Thus, visual system must deal with indeterminacy in indeterminacy in shapeshape and and orientationorientation..

55©2004 Prentice Hall

Visual Perception Hard - Continued?Visual Perception Hard - Continued? The Second Important Problem - The Second Important Problem - Surface FeaturesSurface Features The Visual System must deal with an object’s surface features: color, how dark or light it is, etc.The Visual System must deal with an object’s surface features: color, how dark or light it is, etc. How does the visual system deal with luminance, the amount of light the eye receives?How does the visual system deal with luminance, the amount of light the eye receives? There are indeterminacies inThere are indeterminacies in

Light SourceLight Source ReflectanceReflectance ShadowShadow

The Mach Card as an Example of The Mach Card as an Example of Indeterminacy?Indeterminacy?

Is the gray part a gray surface of a two part object or is a shadow of the white part?

66©2004 Prentice Hall

Visual Perception Hard - Continued?Visual Perception Hard - Continued? The Third Important Problem - The Third Important Problem - Object size and distance are Object size and distance are

indeterminate in a two dimensional representation.indeterminate in a two dimensional representation. For example, the images of the sun and moon seem the same size but For example, the images of the sun and moon seem the same size but

they are at different distances and are different sizes. Could you tell they are at different distances and are different sizes. Could you tell which is what by visual inspection alone?which is what by visual inspection alone?

Is Face Perception Special?Is Face Perception Special?

• Prosopagnosia - syndrome that affects face recognitionProsopagnosia - syndrome that affects face recognition

• A sheep farmer lost his ability to recognize faces but still could A sheep farmer lost his ability to recognize faces but still could recognize his sheep (MacNeil and Warrington, 1983)recognize his sheep (MacNeil and Warrington, 1983)

• However, imaging of the fusiform gyrus suggests that the loss is really However, imaging of the fusiform gyrus suggests that the loss is really one of visual expertise (Gauthier, et al, 1999)one of visual expertise (Gauthier, et al, 1999)

• The issue is still debated.The issue is still debated.

77©2004 Prentice Hall

How Are Visual Ambiguities How Are Visual Ambiguities Resolved?Resolved?

• ShapeShape• BrightnessBrightness• Distance and SizeDistance and Size• Top-Down Influences in VisionTop-Down Influences in Vision• An Alternative: An Alternative:

The Ecological ApproachThe Ecological Approach

88©2004 Prentice Hall

ShapeShape• The visual system uses assumptions in processing objectsThe visual system uses assumptions in processing objects• Object orientations are assumed Object orientations are assumed notnot to be unusual to be unusual• Likelihood principle (Helmhotz):Likelihood principle (Helmhotz):

Suggestion that among the many ways of interpreting an ambiguous visual Suggestion that among the many ways of interpreting an ambiguous visual stimulus, the visual system will interpret it as the stimulus that is most stimulus, the visual system will interpret it as the stimulus that is most likely to occur in the worldlikely to occur in the worldExample – lines parallel in a 2-dimensional representation are likely to be Example – lines parallel in a 2-dimensional representation are likely to be parallel in the 3-dimensional world parallel in the 3-dimensional world

• Frame of Reference:Frame of Reference:The position or orientation of an object is defined relative to something The position or orientation of an object is defined relative to something else. For example, which object is a square and which is a diamond?else. For example, which object is a square and which is a diamond?

Palmer, et al 1988Palmer, et al 1988

99©2004 Prentice Hall

BrightnessBrightness• Assumptions:Assumptions:

Surfaces are uniformly coloredSurfaces are uniformly colored Gradual changes could be caused by shadowGradual changes could be caused by shadow

• Three Factors Contribute to LuminanceThree Factors Contribute to Luminance Light SourceLight Source Shading Shading Shadow – see Photo 2.1 for a fine exampleShadow – see Photo 2.1 for a fine example

• Visual system needs to analyze complex scenes to find Visual system needs to analyze complex scenes to find simple meaningful components. This is depicted in:simple meaningful components. This is depicted in:

• Adelson's (1998) illusion in Fig. 2.5Adelson's (1998) illusion in Fig. 2.5• Gilchrist’s (1997) demonstration of the effects of local Gilchrist’s (1997) demonstration of the effects of local

contrast tested contrast tested binocularlybinocularly & & monocularlymonocularly – see Fig. 2.6 – see Fig. 2.6

1010©2004 Prentice Hall

Distance Distance andand Size Size• How can we determine the true size and distance of objects? Through How can we determine the true size and distance of objects? Through

the use of cues.the use of cues.• Three Cues in the Visual SystemThree Cues in the Visual System

Accommodation – lens shape changes as you focus on objectsAccommodation – lens shape changes as you focus on objects Convergence – angle of the eyes as you focus on objectsConvergence – angle of the eyes as you focus on objects Stereopsis – based on Stereopsis – based on retinal disparityretinal disparity, the difference in position of an object’s , the difference in position of an object’s

image on each retina image on each retina o For example – see Fig 2.8 and the discussion of the For example – see Fig 2.8 and the discussion of the correspondence problemcorrespondence problem. . Random Random

stereogramsstereograms are a prime example. are a prime example.

o Correspondence problemCorrespondence problemTo use disparity as a cue to depth, one must match up the left and right retinal images. To use disparity as a cue to depth, one must match up the left and right retinal images. The correspondence problem refers to the difficulty that retinal images may contain The correspondence problem refers to the difficulty that retinal images may contain many possible matchesmany possible matches

o Random dot stereograms Random dot stereograms Special stimuli with no cues to depth except retinal disparity. Based on shifting patterns Special stimuli with no cues to depth except retinal disparity. Based on shifting patterns of random dot elements to the left or right in two versions of identical dot matrices. of random dot elements to the left or right in two versions of identical dot matrices. Developed by Julesz.Developed by Julesz.

1111©2004 Prentice Hall

Distance and Size - ContinuedDistance and Size - Continued

Cues in the EnvironmentCues in the Environment Familiar Size Familiar Size

Using one’s knowledge of the typical size of an object as a cue to the likely Using one’s knowledge of the typical size of an object as a cue to the likely size and distance of an object. size and distance of an object.

For example, if a child appears larger than an adult, it is likely that the For example, if a child appears larger than an adult, it is likely that the child is closer to the observer. child is closer to the observer.

Pictorial Cues – Cues to distance that can be used in 2-dimensional Pictorial Cues – Cues to distance that can be used in 2-dimensional picturespictures::

o Occlusion: An object that occludes another is closerOcclusion: An object that occludes another is closer o Texture Gradient :A field is assumed to have a uniform texture gradient, so if Texture Gradient :A field is assumed to have a uniform texture gradient, so if

more detail is visible in part of the field, it is assumed to be closer more detail is visible in part of the field, it is assumed to be closer o Linear Perspective: Parallel lines converge in the distance, so the closer they are to Linear Perspective: Parallel lines converge in the distance, so the closer they are to

converging, the farther away the locationconverging, the farther away the location o Relative Height: Objects higher in the picture plane are farther awayRelative Height: Objects higher in the picture plane are farther away – – Photo 2.2Photo 2.2

o Atmospheric Perspective: Objects in the distance look less distinct with a bluish Atmospheric Perspective: Objects in the distance look less distinct with a bluish tinge as they are viewed through dust & water particles that scatter light – Photo tinge as they are viewed through dust & water particles that scatter light – Photo

2.22.2

1212©2004 Prentice Hall

Top-Down Influences in VisionTop-Down Influences in Vision• Bottom-up processing (as Bottom-up processing (as

previously discussed) can’t previously discussed) can’t handle all of vision.handle all of vision.

• Top down processingTop down processing is is needed in which needed in which conceptual knowledge conceptual knowledge influences the processing influences the processing or interpretation of lower-or interpretation of lower-level perceptual processes.level perceptual processes.

• For Example – What is For Example – What is ThisThis and is this and is this 1515??

Parsing ProblemParsing Problem

• One important issue is the parsing One important issue is the parsing problem.problem.

• For some ambiguous figures, it For some ambiguous figures, it seems impossible to identify the seems impossible to identify the figure without knowing what its figure without knowing what its parts are, but its parts cannot be parts are, but its parts cannot be identified unless one knows what identified unless one knows what the figure is.the figure is.

• Palmer suggests it is resolved Palmer suggests it is resolved through using top-down and through using top-down and bottom -up information bottom -up information simultaneously simultaneously

What are these from? What are these from? Click on them!Click on them!

1313©2004 Prentice Hall

It’s a Face!It’s a Face!

Click on the face to go back if you want to!Click on the face to go back if you want to!

1414©2004 Prentice Hall

An Alternative: The Ecological ApproachAn Alternative: The Ecological Approach• Previously, the discussion was about the computational Previously, the discussion was about the computational

approachapproach Dominant approach, it assumes that information provided in Dominant approach, it assumes that information provided in

environment is impoverished. Cognitive system must do environment is impoverished. Cognitive system must do computation to derive the richness of environment.computation to derive the richness of environment.

• Gibson proposes the Gibson proposes the Ecological ApproachEcological Approach Emphasizes that the environment has rich sources of information in Emphasizes that the environment has rich sources of information in

it and that the computations the visual system needs to perform are it and that the computations the visual system needs to perform are probably not that extensiveprobably not that extensive. . Two examples are:Two examples are:

1)1) Object Size – Gibsonians suggest the use of Object Size – Gibsonians suggest the use of EyeheightEyeheight over the over the previously discussed cues. Eyeheight is based on The height of the previously discussed cues. Eyeheight is based on The height of the observer’s eyes from the groundobserver’s eyes from the ground. . Wraga (1999 a & b) support this.Wraga (1999 a & b) support this.

2)2) Distance for Navigation – How do we catch a baseball. Through Distance for Navigation – How do we catch a baseball. Through computation or using a simple environmental cue. McBeath, et al computation or using a simple environmental cue. McBeath, et al (1995) found that people catch a fly ball by running so the (1995) found that people catch a fly ball by running so the trajectory of the ball looks like a straight line.trajectory of the ball looks like a straight line.

1515©2004 Prentice Hall

What is Visual Perception For?What is Visual Perception For?

• Identifying ObjectsIdentifying Objects• NavigationNavigation

There seem to be separate psychological and There seem to be separate psychological and physiological systems for these two processesphysiological systems for these two processes

1616©2004 Prentice Hall

Identifying ObjectsIdentifying Objects

• Core Question: What does the memory Core Question: What does the memory representation that supports the recognition of representation that supports the recognition of objects look like?objects look like?

• Two Families of AnswersTwo Families of Answers• Viewer-centered presentationsViewer-centered presentations

A mental representation of what an object looks like relative to A mental representation of what an object looks like relative to the observer. the observer.

• Object- centered representationObject- centered representation A mental representation of what an object looks like relative to A mental representation of what an object looks like relative to

the object itself. The representation can support recognition of the object itself. The representation can support recognition of the object when it is viewed from any perspective. the object when it is viewed from any perspective.

1717©2004 Prentice Hall

IdentifyingIdentifying Objects - Continued Objects - Continued• Viewer-Centered TheoriesViewer-Centered Theories

• Template TheoryTemplate Theory – an older theory – an older theory A simple template matching theory of object recognition says that you A simple template matching theory of object recognition says that you

compare what you see to templates stored in memorycompare what you see to templates stored in memory Problem is you need a tremendous number of templatesProblem is you need a tremendous number of templates

• Feature-matching TheoriesFeature-matching Theories A theory of visual object identification proposing a memory representation A theory of visual object identification proposing a memory representation

of an object is list of featuresof an object is list of features Example: The letter Example: The letter TT is conceptualized as having two features, a is conceptualized as having two features, a

horizontal line horizontal line ____ and a vertical line | . and a vertical line | . Theory have advantages over Templates Theory have advantages over Templates

Can handle transformations betterCan handle transformations better Supported by neurophysiological discover of seeming “edge” and Supported by neurophysiological discover of seeming “edge” and

“line” detectors by Hubel and Wiesel (1979)“line” detectors by Hubel and Wiesel (1979) DisadvantageDisadvantage – feature theories have problems with natural objects – feature theories have problems with natural objects

1818©2004 Prentice Hall

Identifying Objects - ContinuedIdentifying Objects - Continued• Object Centered TheoriesObject Centered Theories

• In case, you don’t remember. A mental representation of what an object In case, you don’t remember. A mental representation of what an object looks like relative to the object itself. The representation can support looks like relative to the object itself. The representation can support recognition of the object when it is viewed from any perspective. recognition of the object when it is viewed from any perspective.

• Relies on ability to recognize object parts as they are relative to each Relies on ability to recognize object parts as they are relative to each other. Possible processes and related theories are:other. Possible processes and related theories are: Good continuation: Points that can be interpreted connecting a Good continuation: Points that can be interpreted connecting a

straight or smoothly curving line will be interpreted that way rather straight or smoothly curving line will be interpreted that way rather than as connecting sharply angled lines.than as connecting sharply angled lines. See likelihood principle also.See likelihood principle also.

Do you the figure below as two crossed lines Do you the figure below as two crossed lines ( X )( X ) or as the two angles or as the two angles ( > <) ( > <) to the right?to the right?

1919©2004 Prentice Hall

Identifying Objects - ContinuedIdentifying Objects - Continued• Object Centered Theories - Object Centered Theories - ContinuedContinued

• Applications of sophisticated versions of this principle, based on Applications of sophisticated versions of this principle, based on boundaries properties (convexities, maximum curvature) can be boundaries properties (convexities, maximum curvature) can be applied to applied to object center feature theoriesobject center feature theories (Hoffman and Richards) (Hoffman and Richards)

• Influential version is Biederman’s Recognition by Components Influential version is Biederman’s Recognition by Components (1987) using Geons (36 basic shapes – ex. cylinders, bricks, etc.) (1987) using Geons (36 basic shapes – ex. cylinders, bricks, etc.) that act like a visual alphabet. They are easily distinguished. that act like a visual alphabet. They are easily distinguished. Pro – Biederman found that obscuring vertices impairs objects Pro – Biederman found that obscuring vertices impairs objects

recognition while obscuring other parts of objects has a lesser recognition while obscuring other parts of objects has a lesser effect.effect.

Which is easiest to recognize as a cup? The left or right?Which is easiest to recognize as a cup? The left or right?

2020©2004 Prentice Hall

Identifying Objects - ContinuedIdentifying Objects - Continued• Object Centered Theories - Object Centered Theories - ContinuedContinued

Con – Biederman – Not all natural objects can be decomposed into geons. What about a shoe?Con – Biederman – Not all natural objects can be decomposed into geons. What about a shoe?

• Viewer Center Theories - Viewer Center Theories - AgainAgain Compromise position suggested – we store multiple viewer centered representations of objects Compromise position suggested – we store multiple viewer centered representations of objects

and apply transformationsand apply transformations Supporting work:Supporting work:

Shepard and Cooper’s (1986) mental rotation of letters suggests transformations are usedShepard and Cooper’s (1986) mental rotation of letters suggests transformations are used Tarr (1995) found if subjects are trained with multiple views of the standard mental Tarr (1995) found if subjects are trained with multiple views of the standard mental

rotation block figures, they access the best view for the taskrotation block figures, they access the best view for the task

• Object vs. Viewer – who wins?Object vs. Viewer – who wins?• Suggested that the mind uses two methods of recognitionSuggested that the mind uses two methods of recognition

Decomposition-into-parts for objects that have telltale geonsDecomposition-into-parts for objects that have telltale geons Multiple Views for objects that are closely relatedMultiple Views for objects that are closely related

2121©2004 Prentice Hall

NavigationNavigation• What/where hypothesis:What/where hypothesis:

Visual system segregates analysis of what objects are (object Visual system segregates analysis of what objects are (object

recognition) and where they are (spatial location).recognition) and where they are (spatial location).• Ungerleider and Mishkin (1982) tested monkeys on two tasks:Ungerleider and Mishkin (1982) tested monkeys on two tasks:

Location task (called the landmark task) – Disrupted by Parietal Location task (called the landmark task) – Disrupted by Parietal LesionsLesions

Object Identity task (matching task) – Disrupted by Temporal Object Identity task (matching task) – Disrupted by Temporal LesionsLesions

• What/how Hypothesis (Goodale & Milner, 1992):What/how Hypothesis (Goodale & Milner, 1992):Alternative to the what/ where hypothesis, this proposal holds that the Alternative to the what/ where hypothesis, this proposal holds that the visual system segregates analysis of what objects are (object recognition visual system segregates analysis of what objects are (object recognition and location) and how to manipulate them (visual information dedicated and location) and how to manipulate them (visual information dedicated to the motor system).to the motor system).

2222©2004 Prentice Hall

Navigation - ContinuedNavigation - Continued• What/how Hypothesis (Goodale & Milner, 1992) - continuedWhat/how Hypothesis (Goodale & Milner, 1992) - continued• Haffenden and Goodale (1998) – Fooling one system and not the Haffenden and Goodale (1998) – Fooling one system and not the

otherother• Subjects reach for the center object in both displays.Subjects reach for the center object in both displays.• Do they look the same size? One looks biggerDo they look the same size? One looks bigger• When subjects reach do they alter their grasp based on the When subjects reach do they alter their grasp based on the

perceptual effect?perceptual effect?

Answer: Subjects don’t alter their grip. Answer: Subjects don’t alter their grip. The “what” system was fooled but the “how” The “what” system was fooled but the “how” system wasn’t. See a similar result by Proffitt, system wasn’t. See a similar result by Proffitt, et al (1995) with hill slopes – Fig. 2.22et al (1995) with hill slopes – Fig. 2.22

2323©2004 Prentice Hall

Functional Imaging and the Functional Imaging and the Representation of SpaceRepresentation of Space

• Ungerleider and Mishkin (1982): spatial info was the Ungerleider and Mishkin (1982): spatial info was the

province of dorsal stream, parietal lobeprovince of dorsal stream, parietal lobe

• Goodale & Milner (1992): spatial info in both streamsGoodale & Milner (1992): spatial info in both streams

• Ventral – layout of objects in spaceVentral – layout of objects in space

• Dorsal – object’s position for acting on those objectsDorsal – object’s position for acting on those objects

• Epstein & Kanwisher (1988): Ventral stream contains area Epstein & Kanwisher (1988): Ventral stream contains area

sensitive to spatial aspects of local environment – sensitive to spatial aspects of local environment –

parahippocampal place area (PPA) as seen in Box 2-2parahippocampal place area (PPA) as seen in Box 2-2