104

Object Recognition

-Segregation of function

-Visual hierarchy

-What and where (ventral and dorsal streams)

-Single cell coding and ensemble coding

-Distributed representations of object categories

-Face recognition

-Object recognition as a computational problem

Higher Perceptual Functions

105

Segregation of function exists already in theearly visual system:

M channel (magnocellular): from M-type retinalganglion cells to magnocellular LGN layers to layer IVBof V1; wavelength-insensitive in LGN, orientationselectivity in V1 (“simple cells”), binocularity anddirection selectivity in layer IVB; processing visualmotion.

P channel (parvocellular): from P-type retinal ganglioncells to parvocellular LGN layers to interblob regions oflayer III in V1; many cells in LGN show coloropponency, cells in interblob regions of V1 have strongorientation selectivity and binocularity (“complexcells”), channel is also called P-IB; processing visualobject shape.

Functional Segregation

106

Segregation of function can also be found atthe cortical level:

- within each area: cells form distinct columns.

- multiple areas form the visual hierarchy …

Functional Segregation

107

The Visual Hierarchy

van Essen and Maunsell, 1983

108van Essen et al., 1990

The Visual Hierarchy

109

-functional segregation of visual features into separate(specialized) areas.-increased complexity and specificity of neuralresponses.- columnar groupings, horizontal integration withineach area.-larger receptive fields at higher levels.-visual topography is less clearly defined at higherlevels, or disappears altogether.-longer response latencies at higher levels.- large number of pathways linking each segregatedarea to other areas.- existence of feedforward, as well as lateral andfeedback connections between hierarchical levels.

The Visual Hierarchy

110

The Architecture of Visual Cortex

Mishkin and Ungerleider, 1983

Lesion studies in the macaque monkey suggest that there aretwo large-scale cortical streams of visual processing:

Dorsal stream (“where”)

Ventral stream (“what”)

111

What and Where

Mishkin and Ungerleider, 1983

Object discrimination task

Landmark discrimination task

Bilateral lesion of the temporallobe leads to a behavioral deficitin a task that requires thediscrimination of objects.

Bilateral lesion of the parietallobe leads to a behavioral deficitin a task that requires thediscrimination of locations(landmarks).

112

The Architecture of Visual Cortex

motion

colorform

Lateral views of the macaque monkey brain

113

Single Cells and Recognition

What is the cellular basis for visual recognition (visuallong-term memory)?

1. Where are the cellular representationslocalized?

2. What processes generate theserepresentations?

3. What underlies their reactivation during recalland recognition?

114

Single Cells and Recognition

Visual recognition involves the inferior temporal cortex(multiple areas). These areas are part of a distributednetwork and are subject to both bottom-up (feature driven)and top-down (memory driven) influences.

Miyashita and Hayashi, 2000

115

Single Cells and Recognition

Characteristics of neural responses in IT:

1. Object-specific (tuned to object class), selectivefor general object features (e.g. shape)

2. Non-topographic (large RF)3. Long-lasting (100’s ms)

Columnar organization (“object feature columns”)Specificity has often rather broad range

(distributed response pattern)

116

Distributed Representations

Are there specific, dedicated modules (or cells) foreach and every object category?

No. – Why not?

117

Distributed Representations

Evidence → feature based and widely distributedrepresentation of objects across (ventral) temporalcortex.

What is a distributed representation?

118

Distributed Representations

Experiments conducted by Ishai et al.:

Experiment 1:1. fMRI during passive viewing2. fMRI during delayed match-to-sample

Experiment 2:1. fMRI during delayed match-to-sample with

photographs2. fMRI during delayed match-to-sample with line

drawings

Three categories: houses, faces, chairs.

119

Distributed Representations

Findings:

Experiment 1:Consistent topography in areas that most strongly

respond to each of the three categories.Modules?No - Responses are distributed (more so for non-face

stimuli)

Experiment 2:Are low-level features (spatial frequency, texture etc.)

responsible for the representation?No – line drawings elicit similar distributions of responses

120

Distributed Representations

From Ishai et al., 1999

121

Distributed Representations

From Ishai et al., 1999

houses

faces

chairs

122

Face recognition achieves a very high level ofspecificity – hundreds, if not thousands ofindividual faces can be recognized.

Face Recognition

Visual agnosia specific to faces: prosopagnosia.

High specificity of face cells → “gnostic units”,“grandmother cells”

Many face cells respond to faces only – andshow very little response to other object stimuli.

123

Face Recognition

Typical neural responses in the primate inferior temporalcortex:

Desimone et al., 1984

124

Face Recognition

Face cells (typically) do not respond to:1. “jumbled” faces2. “partial” faces3. “single components” of faces (although some

face-component cells have been found)4. other “significant” stimuli

Face cells (typically) do respond to:1. faces anywhere in a large bilateral visual field2. faces with “reduced” feature content (e.g. b/w,

low contrast)

Face cell responses can vary with: facialexpression, view-orientation

125

Face Recognition

Face cells are (to a significant extent) anatomicallysegregated from other cells selective forobjects. They are found in multiple subdivisionsacross the inferior temporal cortex (in particularin or near the superior temporal sulcus)

126

Face Recognition

Faces versus objects in a recent fMRI study (Halgren etal. 1999)

127

Object Recognition:Why is it a Hard Problem?

Objects can be recognized over huge variations inappearance and context!

Ability to recognize objects in a great number ofdifferent ways:object constancy (stimulus equivalence)

Sources of variability:- Object position/orientation- Viewer position/orientation- Illumination (wavelength/brightness)- Groupings and context- Occlusion/partial views

128

Object Recognition:Why is it a Hard Problem?

Examples for variability:field of view

Translation invariance

Rotation invariance

129

Object Recognition:Why is it a Hard Problem?

More examples for variability:field of view

Size invariance

Color

130

Object Recognition:Why is it a Hard Problem?

Variability in visual scenes:

field of view

Partial occlusionand presence of other objects

131

Object Recognition: Theories

Representation of visual shape (set of locations):

Viewer-centered coordinate systems:frame of reference: viewerexample: retinotopic coordinates, head-centeredcoordinateseasily accessed, but very unstable …

Environment-centered coordinate systems:locations specified relative to environment

Object-centered coordinate systems:intrinsic to or fixed to object itself (frame of reference:object)less accessible

132

Object Recognition: Theories

A taxonomy:

1. Template matching models (viewer-centered,normalization stage and matching)

2. Prototype models

3. Feature analysis model

4. Recognition by components (object-centered)

133

Object Recognition: Geons

Theory proposed by Irv Biederman.

Objects have parts.

Objects can be described as configurations of a(relatively small) number of geometricallydefined parts.

These parts (geons) form a recognition alphabet.24 geons for four basic properties that areviewpoint-invariant.

134

Object Recognition: Geons

How geons are constructed:

135

Object Recognition: Geons

Irv Biederman, JCN, 2001

Geons in IT?

136

How does Invariance Develop?

137

Deficits of feature perception (such asachromatopsia) generally do not cause an inabilityto recognize objects.

Failure of knowledge or recognition = “agnosia”.(visual agnosia)

In visual agnosias, feature processing and memoryremain intact, and recognition deficits are limitedto the the visual modality. Alertness, attention,intelligence and language are unaffected.

Other sensory modalities (touch, smell) maysubstitute for vision in allowing objects to berecognized.

Higher Perceptual Functions: Agnosias

138

Apperceptive agnosia: perceptual deficit, affectsvisual representations directly, components ofvisual percept are picked up, but can’t beintegrated, effects may be graded, oftenaffected: unusual views of objects

Associative agnosia: visual representations areintact, but cannot be accessed or used inrecognition. Lack of information about thepercept. “Normal percepts stripped of theirmeaning” (Teuber)

This distinction introduced by Lissauer (1890)

Two Kinds of Agnosias

139

Apperceptive Agnosia

Diagnosis: ability to recognize degraded stimuliis impaired

A AFarah: Many “apperceptive agnosias” are“perceptual categorization deficits” …

140

Apperceptive Agnosia

Studies by E. Warrington:

Laterality in recognition deficits: patients withright-hemispheric lesions (parietal, temporal)showed lower performance on degraded imagesthan controls or left-hemispheric lesions.

Hypothesis: object constancy is disrupted (notcontour perception)

Experiment: Unusual views of objects – patientswith right-hemispheric lesions show acharacteristic deficit for these views.

141

Apperceptive Agnosia

Is “perceptual categorization deficit” a generalimpairment of viewpoint-invariant objectrecognition?

1. Patients are not impaired in everyday life(unlike associative agnosics).

2. They are not impaired in matching different“normal” views of objects, only “unusual views”.

3. Impairment follows unilateral lesions, notbilateral (as would be expected if visual shaperepresentations were generally affected).

142

Associative Agnosia

Patients do well on perceptual tests (degradedimages, image segmentation), but cannotaccess names (“naming”) or other information(“recognition”) about objects. Agnosics fail toexperience familiarity with the stimulus.

When given names of objects, they can(generally) give accurate verbal descriptions.

Warrington’s analysis places associativeagnosia in left hemisphere.

143

Associative Agnosia

Associative agnosics can copydrawings of objects butcannot name them (evidencefor intactness of perceptualrepresentations…) but…

144

Agnosia Restricted to SpecificCategories

Specific deficits in recognizing living versus non-livingthings.

Warrington and Shallice (1984): patients with bilateraltemporal lobe damage showed loss of knowledge aboutliving things (failures in visual identification and verbalknowledge).

Their interpretation: distinction between knowledgedomains – functional significance (vase-jug) versussensory properties (strawberry-raspberry).

Evolutionary explanation…

145

Agnosia Restricted to SpecificCategories

Another view: Damasio (1990)

Many inanimate objects are manipulated byhumans in characteristic ways.

Interpretation: inanimate objects will tend toevoke kinesthetic representations.

Agreeing with Warrington, difficulty is not due tovisual characteristics or visual discriminability.

146

Agnosia Restricted to SpecificCategories

Yet another view: Gaffan and Heywood (1993)

Presented images (line drawings) of animate andinanimate to normal humans and normalmonkeys, tachistoscopically (20 ms). Bothsubject groups made more errors in identifyinganimate vs. inanimate objects.

Interpretation: Living things are more similar toeach other than non-living things → “category-specific agnosia”

147

How is Semantic KnowledgeOrganized?

Category-based systemProperty-based system

Network model by Farah and McClelland (1991).

148

Prosopagnosia

Is face recognition “special”?Anatomical localizationFunctional independence

Associative visual agnosia (prosopagnosia): Lostability to recognize familiar faces.

Affects previous experience as well as(anterograde component) newly experiencedfaces.

Patients can recognize people by their voice,distinctive clothing, hairstyle etc.

149

Prosopagnosia

What is special about faces:

1. Higher specificity of categorization2. Higher level of expertise3. Higher degree of visual similarity4. Evolutionary significance

Can face and object recognition be dissociated?

Neuropsychological evidence suggests, yes (studyby McNeil and Warrington)

Also, remember Ishai et al. (object category map)