Visual Perception: what do we want to explain?

How do we get visual information from the world and use it to controlbehavior? What neural processes underlie visually guided behavior?

Traditional sub-areas - visual sensitivity color visionspatial visiontemporal visionbinocular vision/ depth

perceptiontexture perceptionmotion perceptionsurfaces, segmentationobject perceptionattention

Visual control of movement

eye movementsreachingattention

Traditional sub-areas - visual sensitivity color visionspatial visiontemporal

visionbinocular

vision/ depth perceptiontexture

perceptionmotion

perceptionsurfaces,

segmentationobject

perceptionattention

Visual control of movement

eye movements

reachingattention

Readings, Class 1:

Gazzaniga et al Chapter 5 pp 177-188

Kandell et al Chapters 27, 28

Class 2: Squire et al Ch 46, 47

Sources:Kandel, Schwartz & Jessel Principles of Neural Science McGraw-Hill 4th edGazzaniga, Ivry, Mangun Cognitive Neuroscience Norton, 3rd edSquire, Berg, Bloom, du Lac, Ghosh, Spitzer Fundamental Neuroscience 3rd ed Academic PressRosenbaum Human Motor Control 2nd ed Academinc Press

Eye movements

Visual Projections

Signals from each eye areadjacent in LGN but remainsegregated in different layers.Convergence occurs in V1.

Two kinds of cells in retina projectto different layers in LGN

M=magno=bigP=parvo=small

Major transformations of the light signal in the retina:

1. Temporal filtering – reduced response to high temporal frequencies – Temporal integration – a strong 1 msec flash is equivalent to a weaker 50 msec flash.

2. Light adaptation – sensitivity regulation - adjustment of operating range to mean light level. (Light level 1010 range, ganglion cells, 102 range.)

3. Anatomical organization of photoreceptors provides high acuity in fovea with rapid fall-off in the periphery.

4. Convergence of photoreceptors onto ganglion cells also leads to acuity limitations in the peripheral retina.

5. Organization of 3 cone photoreceptors into color opponent signals (Luminance, Red-Green, Yellow-Blue)

Magno and parvo cells have different spatial and temporal sensitivities.

Function of the differentM and P pathways isunclear.

Retinotopic Organization and Cortical Magnification

The brain uses more physical space for signals from the fovea thanthe periphery

Adjacent points in the worldProject to adjacent points in cortex

Visual consequences of lesions at different locations in the visual pathway.

Visual cortex is a layered structure (6 layers). LGN inputs arrive in Layer 4. Layers 2,3 output to higher visual area., Layers 5,6 output to sub-cortical areas (eg superior colliculus)Massive feedback projection from layer 6 to LGN – 800 lb gorilla.

Cells in V1 respond to moving or flashing oriented bars. Little response to diffuse flashes or steady lights.

LGN cells have circular receptive fields, like retina.Not clear what the role of the LGN is.

Oriented cells emerge in V1, probably composedOf appropriately aligned LGN cells as shown.

Cells in V1 are organized intocolumns. Orientation preferencegradually changes as one progressesacross cortex. Cells at different depthsHave same orientation preference.

Binocular convergence:Cells respond more or less to R and Leye inputs. Ocular dominance varies smoothly across cortical surfaceorthogonal to orientation variation

Orderly anatomical organization in V1

Regular large scale organization of orientation preference across cortical surface. Does thissimplify signal processing?

What is V1 doing?

Early idea: edge detectors – basis for more complex patterns

Later (1970-80’s) – spatial frequency channels any spatial pattern can be composed of a sum of sinusoids

Late 90’s to now:Main idea about V1 is that is represents an efficient recoding of the information inthe visual image.

Images are not random. Random images would require point-by-point representationlike a camera.

Images have clusters of similar pixels and cells designed to pick this up. Cells extract information about spatial variation at different scales (clusters of different sizes).

Can think of receptive fields as “basis functions” (an alphabet of elemental image components that capture clustering in local image regions)

Approximating an image patch w basis functions

V1striate cortex

LGNThalamic nucleus

The outputs of 64cells in the LGN …

… can be coded with only twelve V1 cells …

… where each cell has 64 synapses

The neural coding library of learned RFs

Because there are more than we need - Overcomplete (192 vs 64) - the number of cells that need to send spikes at any moment is Sparse (12 vs 64).

Defining visual areas:

Retinotopic responsesAnatomical projections

Note old simplistic view:One area, one attributeIs not true.Areas are selective in complex and poorly understood ways

Note the case of Mike May.

More complex analysis of image properties in higher visual areas (extra-striate)

Mike May - world speed record for downhill skiing by a blind person.

Lost vision at age 3 - scarred corneas.

Optically 20/20 - functionally 20/500 (cf amblyopia)

Answer to Molyneux’s question: Mike May couldn’t tell difference between sphere and cube. Improved, but does it logically rather than perceptually. (cf other cases)

Color: an orange thing on a basket ball court must be a ball.

Motion: can detect moving objects, distinguish different speeds (structure from motion).

Note: fMRI shows no activity in Infero-temporal cortex (corresponding to pattern recognition) but there is activity in MT, MST (motion areas) and V4 (color). Other parts of brain take over when a cortical area is inactive.

Cannot recognize faces. (eyes, movement of mouth distracting)

Can’t perceive distance very well.

Can’t recognize perspective.

No size constancy or lightness constancy/ segmentation of scene into objects, shadows difficult.

Vision most useful for catching balls (inconsistent with Held & Hein??) and finding things if he drops them.

Fellerman and Van Essen 85

Hippocampus

Cells in MT are sensitive to motion inparticular directions.

Cells are also tuned for particular speeds

MT lesions lead to deficits in motion perception

Methods for measuringmotion senistivity%motion and direction range

The Aperture ProblemCells in V1 can only detect motion orthogonal to the receptive field. Output is ambiguous.MT is thought to resolve this ambiguity by combining motion from different V1 cells. Integration of features (corners) is also used.

A. When the eyes are held still, the image of a moving object traverses the retina. Information about movement depends upon sequential firing of receptors in the retina.B. When the eyes follow an object, the image of the moving object falls on one place on the retina and the information is conveyed by movement of the eyes or the head.

Two ways of perceiving motion.

MTMSTOutput of cells goesto brainstem regionscontrolling pursuit eye movements.

Motion of the body in the world introduces characteristic motion patterns in the image.

MST is sensitive to these patterns.

Motion of animate agents

Output to pursuit system

Optic flow patterns

dorsal

ventral

Lesions in monkey MST lead to deficits in pursuit eye movements.

Right occipito-parietal lesions in human leads to similardeficits in pursuit eye movements.

MST has input from the vestibular system. Thus the cells have information about self motion from sources other than the flow field.

Many cortical areas have inputs form eye movement signals as well, even asEarly as V1. Presumably this is responsible for the ability of the visual system to process image information independent of image motion on the retina.

Perception of Depth

Monocular cues:familiar sizeocclusiongeometric perspectiveshadingmotion parallax

Global (distance) and local (shape)aspects

Stereopsis

Note developmental sensitivity of stereo visionStrabismus, amblyopia

Stereo sensitivity is one of the hyperacuities

Motion parallax a little less sensitive butprobably important because it is ubiquitous.

Disparity – measure of depthDifference

Angle a – angle b

a

b

Neural computation of disparity is complex and not well understood. Disparity signals in V1 and V2, MT.

Investigation of stereo vision using random dot stereograms

Such stimuli have no monocular information and so are experimentally useful for isolating stereo processes, but have disadvantage that they are harder than usual.

Perception of Surfaces

Subjective Contours

Infero-temporal cortex

V4 (color)

MT/MST (motion)

Cortical specialization