Chapter 4: Object RecognitionChapter 4: Object Recognition

•• What do various disorders of shape recognition tell us What do various disorders of shape recognition tell us about object recognition?about object recognition?

-- ApperceptiveApperceptive visual visual agnosiaagnosia (Ch. 2)(Ch. 2)

-- Associative visual Associative visual agnosiaagnosia

-- Perceptual categorization deficitPerceptual categorization deficit

•• What do What do neuroimagingneuroimaging studies tell us about object studies tell us about object recognition?recognition?

•• The computational interpretation: What does cognitive The computational interpretation: What does cognitive evidence suggest about constraints on the nature of the evidence suggest about constraints on the nature of the neural representations underlying object recognition?neural representations underlying object recognition?

Object RecognitionObject Recognition

•• Dr. Harley has filled Dr. Harley has filled

you in on the you in on the

neuroscience side of neuroscience side of

the miraclethe miracle……

Farah (2000) – Fig. 4.1

Visual Visual AgnosiasAgnosias

•• Visual Visual AgnosiaAgnosia: A blanket : A blanket term for impaired visual object term for impaired visual object recognition following brain recognition following brain damage when elementary damage when elementary visual functions (acuity, visual visual functions (acuity, visual fields) are adequate. fields) are adequate.

•• ApperceptiveApperceptive visual visual agnosiaagnosia: : Inability to group the local Inability to group the local features into a coherent features into a coherent perceptual representation (Ch perceptual representation (Ch 2)2)

•• Associative visual Associative visual agnosiaagnosia: : Inability to recognize visually Inability to recognize visually presented objects, despite presented objects, despite having a coherent perceptual having a coherent perceptual representation representation

Visual Visual AgnosiasAgnosias: Copying and : Copying and

drawing from memory.drawing from memory.

Apperceptive agnosia Associative agnosia

Copying

Memory

Copying

Associative visual Associative visual agnosiaagnosia: Criteria: Criteria

•• Patients can form percepts (do perceptual Patients can form percepts (do perceptual grouping) unlike grouping) unlike apperceptiveapperceptive agnosiaagnosia patients.patients.

•• They can see an object well enough to describe They can see an object well enough to describe its appearance, to draw it, or to succeed in a its appearance, to draw it, or to succeed in a same/different test of appearances.same/different test of appearances.

•• They have difficulty recognizing visually They have difficulty recognizing visually presented objects (canpresented objects (can’’t name or sort objects by t name or sort objects by category)category)

•• They can demonstrate knowledge of objects They can demonstrate knowledge of objects from other sensory modalitiesfrom other sensory modalities

Associative Associative agnosiaagnosia behavior: An behavior: An

impairment of impairment of shape perceptionshape perception

•• They can copy complex objects. They can copy complex objects. BUT their copying behavior is BUT their copying behavior is abnormal abnormal –– HJA took 6 hours to do HJA took 6 hours to do the cathedral.the cathedral.

•• They are very sensitive to visual They are very sensitive to visual quality of stimuli. They recognize quality of stimuli. They recognize real objects better than line real objects better than line drawings; face recognition of drawings; face recognition of unfamiliar people is impaired by unfamiliar people is impaired by changing lighting conditions.changing lighting conditions.

•• They make visual shape recognition They make visual shape recognition errors; they might call a baseball errors; they might call a baseball bat, a paddle, knife, bat, a paddle, knife, basterbaster, etc. , etc.

Associative Visual Associative Visual AgnosiaAgnosia

•• The human analog of the ITThe human analog of the IT--lesionedlesioned

monkeys.monkeys.

–– They fail to recognize objects because they They fail to recognize objects because they

fail to represent shape normally.fail to represent shape normally.

•• Slow, slavish copyingSlow, slavish copying

•• Sensitivity to visual quality of the stimuliSensitivity to visual quality of the stimuli

•• Visual shape errorsVisual shape errors

Regions of brain damage associated Regions of brain damage associated

with different types of visual with different types of visual agnosiaagnosia

•• ApperceptiveApperceptive AgnosiaAgnosia: :

Diffuse damage to the Diffuse damage to the

occipital lobe and occipital lobe and

surrounding areassurrounding areas

•• Associative Associative AgnosiaAgnosia: :

OccipitotemporalOccipitotemporal

regions of both regions of both

hemisphereshemispheres

From Banich (2004)

Perceptual categorization deficitPerceptual categorization deficit

•• Difficulty recognizing Difficulty recognizing

objects viewed from objects viewed from

unusual perspectives or unusual perspectives or

uneven illumination uneven illumination

conditions (Warrington & conditions (Warrington &

Taylor, 1973)Taylor, 1973)

•• Initially assumed to Initially assumed to

reflect an impairment of reflect an impairment of

viewpointviewpoint--invariant object invariant object

recognitionrecognition

BUT: Is Perceptual Categorization BUT: Is Perceptual Categorization

Deficit really about loss of shape Deficit really about loss of shape

representations?representations?

•• Not impaired in the real world.Not impaired in the real world.

•• Not impaired under all viewing conditions per se.Not impaired under all viewing conditions per se.

•• Mainly impaired when matching an USUAL to an Mainly impaired when matching an USUAL to an

UNUSUAL view; normal people have similar, less UNUSUAL view; normal people have similar, less

serious, type of difficulty.serious, type of difficulty.

•• Associated with unilateral RH lesions, usually in Associated with unilateral RH lesions, usually in

parietal cortex, not parietal cortex, not inferotemporalinferotemporal cortexcortex

Perceptual categorization deficitPerceptual categorization deficit

•• Demonstrates value of examining Demonstrates value of examining

evidence carefully and trying to link it with evidence carefully and trying to link it with

what is known to see whether patterns fit what is known to see whether patterns fit

or not.or not.

•• In this case, the pattern differs greatly In this case, the pattern differs greatly

from other visual from other visual agnosiasagnosias. .

Functional Functional NeuroimagingNeuroimaging Studies: Studies:

•• Goal of PET, Goal of PET, fMRIfMRI studiesstudies: To localize the : To localize the

psychological psychological process(esprocess(es) of interest ) of interest

•• Research designResearch design: Measure brain activity in at : Measure brain activity in at

least two conditions: a control (baseline) least two conditions: a control (baseline)

condition and an experimental condition.condition and an experimental condition.

–– Psychological process localized by subtracting brain Psychological process localized by subtracting brain

activity in the control (baseline) condition from brain activity in the control (baseline) condition from brain

activity in the experimental condition.activity in the experimental condition.

PET Imaging (PET Imaging (Posner & Posner & RaichleRaichle, 1994, 1994))

Upper row: Control PET scan (resting while looking at static fixation point) is subtracted from looking at a flickering checkerboard stimulus positioned 5.5°from fixation point.Middle row: Subtraction procedure produces a somewhat different image for each of 5 subjects.Bottom row: The 5 images are averaged to eliminate noise, producing the image at the bottom.

Where in the brain does visual Where in the brain does visual

recognition occur? recognition occur?

•• Visual recognition Visual recognition

localized to the localized to the

posterior half of the posterior half of the

brain brain –– really?really?

•• Whoops!!! Why did Whoops!!! Why did

these studies fail to these studies fail to

localize visual localize visual

recognition?recognition?

•• Poor methodology!Poor methodology!

NeuroimagingNeuroimaging methods: The methods: The

subtraction techniquesubtraction technique

•• Using the subtraction technique effectively Using the subtraction technique effectively

requires that the baseline and experimental requires that the baseline and experimental

conditions differ only in the process of interest.conditions differ only in the process of interest.

–– Requires careful logical analysis of the task to Requires careful logical analysis of the task to

determine the best comparison conditions.determine the best comparison conditions.

•• If there are many differences between the If there are many differences between the

baseline and experimental conditions, it is baseline and experimental conditions, it is

difficult to interpret the results of the study.difficult to interpret the results of the study.

NeuroimaginingNeuroimagining Studies of Object Studies of Object

Recognition: Comparing baseline and Recognition: Comparing baseline and

experimental conditionsexperimental conditions

Sequence of trial eventsSequence of trial events

–– Fixation pointFixation point

–– Stimulus presentationStimulus presentation

–– Task (two types)Task (two types)

Passive Viewing TaskPassive Viewing Task::

Are stimuli comparable Are stimuli comparable

(e.g., size, complexity)(e.g., size, complexity)

Mind is not a vacuumMind is not a vacuum

Active Viewing TaskActive Viewing Task::

What is task?What is task?

What is response?What is response?

SergentSergent et al (1992a): et al (1992a):

Active viewing task.Active viewing task.

•• Baseline conditionBaseline condition

-- View fixation pointView fixation point

•• Experimental Experimental conditioncondition

-- View fixation pointView fixation point

-- See line drawing of See line drawing of objectobject

-- Decide whether the Decide whether the object is living or nonobject is living or non--livingliving

-- Make Yes/No Make Yes/No responseresponse

Localizing Visual RecognitionLocalizing Visual Recognition

•• Is human visual recognition supported by Is human visual recognition supported by

one general purpose recognition system or one general purpose recognition system or

are there specialized modules for are there specialized modules for

recognizing objects, faces, printed words?recognizing objects, faces, printed words?

•• Good functional Good functional neuroimagingneuroimaging studies do studies do

exist to test these possibilities and will be exist to test these possibilities and will be

discussed in Chapters 5 & 6.discussed in Chapters 5 & 6.

Constraints on the nature of shape Constraints on the nature of shape

representations in ITrepresentations in IT: : Coordinate systemsCoordinate systems

•• Cannot be simple viewerCannot be simple viewer--centered or centered or

environmentallyenvironmentally--centeredcentered

–– IT cells respond to a given shape over IT cells respond to a given shape over

changes in the position, size, and picture changes in the position, size, and picture

plane orientation of objectplane orientation of object

–– Impairments of IT Impairments of IT lesionedlesioned monkeys: monkeys:

suggest they have lost abstract shape suggest they have lost abstract shape

representation representation

Constraints: Coordinate systems cont.Constraints: Coordinate systems cont.

•• Two possibilities:Two possibilities:

1. Object1. Object--centered centered

coordinate systemcoordinate system

2. A cluster of multiple 2. A cluster of multiple

viewerviewer--centered object centered object

representations plus representations plus

the ability to transform the ability to transform

one representation to one representation to

another as necessary another as necessary

(a la Multiple Views (a la Multiple Views

model of model of TarrTarr, 1995)., 1995).

Constraints: Coordinate systems cont.Constraints: Coordinate systems cont.

•• FarahFarah prefers the Multiple Viewerprefers the Multiple Viewer--Centered Centered representation option because:representation option because:–– Position, size, and orientation invariance shown by IT Position, size, and orientation invariance shown by IT

cells is not perfect; consistent with some views being cells is not perfect; consistent with some views being better learned than others.better learned than others.

–– IT neurons can selectively learn arbitrary associations IT neurons can selectively learn arbitrary associations between pairs of stimuli, a prerequisite for deriving between pairs of stimuli, a prerequisite for deriving invariance from viewerinvariance from viewer--centered representations.centered representations.

–– Consequently, because cell activity takes time to Consequently, because cell activity takes time to decay and because seeing different views of an object decay and because seeing different views of an object tends to be clustered in time, the correlation tends to be clustered in time, the correlation (association) between several different (association) between several different retinotopicretinotopicviews of the same object could be learned. views of the same object could be learned.

Constraints: Constraints: Primitives and OrganizationPrimitives and Organization

•• PrimitivesPrimitives

–– Cannot be contoursCannot be contours

–– Could be either surfaceCould be either surface--based (2D) or based (2D) or

volumevolume--based (e.g., 3D based (e.g., 3D geongeon--like parts).like parts).

•• OrganizationOrganization

–– Hierarchical? We really donHierarchical? We really don’’t know much t know much

about how multipart objects are representedabout how multipart objects are represented

Constraints: ImplementationConstraints: Implementation

Symbolic Model Symbolic Model vsvs Neural netNeural net

•• Is a perceptual representation created and then Is a perceptual representation created and then compared to a memory representation? (Implies compared to a memory representation? (Implies that the memory representation can be destroyed, that the memory representation can be destroyed, yet the perceptual representation remains intact). yet the perceptual representation remains intact). Unlikely, since no such evidence exists. Unlikely, since no such evidence exists.

•• Does the input get coded and recoded in a Does the input get coded and recoded in a succession of neural nets as it is processed through succession of neural nets as it is processed through the visual system? (Implies that impairment in test the visual system? (Implies that impairment in test of object memory is associated with perceptual of object memory is associated with perceptual impairment). Yes, this seems true in associative impairment). Yes, this seems true in associative agnosiaagnosia..

Constraints: ImplementationConstraints: Implementation

•• Distributed Distributed

representation more representation more

likely than local likely than local

representationrepresentation

–– Single unit recording Single unit recording

–– Graceful degradation Graceful degradation

following damage to following damage to

ITIT