Prof. Greg Francis

1PSY 310: Sensory and Perceptual Processes

Purdue University

Neural codes

PSY 310

Greg Francis

Lecture 12

Is 100 billion neurons enough?

Purdue University

COC illusion The COC illusion looks like real squares because the neural

responses are similar

True squares COC squares

Ganglion cell responses

Prof. Greg Francis

2PSY 310: Sensory and Perceptual Processes

Purdue University

Ventral stream

From visual cortex to the temporal lobe

Involved in recognizing or identifying objects

Purdue University

Critical features Cells in inferotemporal cortex respond to complex features in

stimuli Not easy to measure neurophysiologically Why such bizarre patterns?

Prof. Greg Francis

3PSY 310: Sensory and Perceptual Processes

Purdue University

Real stimuli Following the ideas about the COC stimulus, our

perceptual experience is determined by the neuralresponses to stimuli

Suppose you see this picture of a cat

Purdue University

Cell responses This IT cell might respond strongly And the other cells hardly at all

Prof. Greg Francis

4PSY 310: Sensory and Perceptual Processes

Purdue University

Cell responses One might be tempted to suggest that this cell

“codes” the perceptual experience of the cat But this would not be correct This cell would respond similarly to many different

types of stimuli But our perceptual experience is quite different!

Purdue University

Neurophysiology and perception

Does this mean that our perceptual experience isnot determined by neural responses? After all similar neural responses should give rise to

similar percepts

No. Similarity between ganglion responses and

perceptual experience works for the ganglion cellsbecause of their anatomical location Everything else in the visual system is based on their

responses

Prof. Greg Francis

5PSY 310: Sensory and Perceptual Processes

Purdue University

Neurophysiology and perception The rest of visual perception does not depend on

the responses of this IT cell

Purdue University

Neurophysiology and perception The rest of visual perception does not depend on the

responses of this IT cell In fact, probably thousands of IT cells respond to this

stimulus

Prof. Greg Francis

6PSY 310: Sensory and Perceptual Processes

Purdue University

Neurophysiology and perception The rest of visual perception does not depend on the

responses of this IT cell In fact, probably thousands of IT cells respond to this

stimulus Millions of ganglion cells

Purdue University

Neurophysiology and perception The rest of visual perception does not depend on the

responses of this IT cell In fact, probably thousands of IT cells respond to this

stimulus Millions of ganglion cells Millions of orientation sensitive cells

Prof. Greg Francis

7PSY 310: Sensory and Perceptual Processes

Purdue University

Neurophysiology and perception The rest of visual perception does not depend on the

responses of this IT cell In fact, probably thousands of IT cells respond to this

stimulus Millions of ganglion cells Millions of orientation sensitive cells Dorsal stream

Purdue University

Representation of objects Objects are unlikely to be represented by a single neuron

Consider faces

Prof. Greg Francis

8PSY 310: Sensory and Perceptual Processes

Purdue University

Representation of objects Objects are unlikely to be represented by a single neuron

Consider faces

What do you do here?

Purdue University

Representation of objects Objects are unlikely to be represented by a single neuron

Consider faces

What do you do here?

What do you do withdifferent expressions?

Prof. Greg Francis

9PSY 310: Sensory and Perceptual Processes

Purdue University

Representation of objects Objects are unlikely to be represented by a single neuron

Consider faces

What do you do here?

Since we perceive andrecognize all of thesefaces as beingdifferent. There mustbe a different neuralrepresentations.

Purdue University

Distributed code Each cell codes some feature of the image

Prof. Greg Francis

10PSY 310: Sensory and Perceptual Processes

Purdue University

Distributed code Each cell codes some feature of the image

The representationof a face is apattern across thefeatures.No two faces havethe same pattern,so not the samepercept.

Purdue University

Distributed code Each cell codes some feature of the image

Change ofexpression maychange only someof the features.

Similar faces havesimilar patterns; sosimilar percepts.

Prof. Greg Francis

11PSY 310: Sensory and Perceptual Processes

Purdue University

Features So what are the features?

A difficult question to answer

They need not be things that we would name

Nose, eye, mouth, hair

Could be Fourier components

Or something completely different

Purdue University

Fourier features Selecting certain Fourier components as features might not

correspond to anything that we would name in the image

Prof. Greg Francis

12PSY 310: Sensory and Perceptual Processes

Purdue University

Distributed code How does it all get put together?

We don’t seefeatures, we seefaces.

Purdue University

Related issue We earlier discussed how we can consider the visual system to consist

of many retinotopic layers of activities from cells tuned to differentfeatures

Image

Retina

Ganglion cells

Orientationcells

Prof. Greg Francis

13PSY 310: Sensory and Perceptual Processes

Purdue University

Related issue There are actually many layers we never got a chance to talk

about

Color, size, motion, depth, texture, many more

To an extent, they process things independently - features

Image

Retina

Ganglion cells

Orientationcells

ColorMotion

Purdue University

Related issue If you see a red car go speeding by, you gets lots of responses

from different parts of the brain

Which part is the percept?

How do they get coordinated?

Image

Retina

Ganglion cells

Orientationcells

ColorMotion

Prof. Greg Francis

14PSY 310: Sensory and Perceptual Processes

Purdue University

Related issue For that matter, if you are thinking of something else or

listening intently to something

You may not perceive the car

Even though the nervous system responds!

Image

Retina

Ganglion cells

Orientationcells

ColorMotion

Purdue University

Attention It’s not entirely clear what attention is or does

But it seems to be involved in “pulling together” neuralresponses from different parts of the brain

Perhaps by synchronizing action potentials

Perhaps it selects the features that help solve a particular task

E.g., recognize a face

Attention seems to be necessary to actually perceivesomething

Prof. Greg Francis

15PSY 310: Sensory and Perceptual Processes

Purdue University

Attentional blink

Suppose you have to identify rapidly presented (100 ms)letters

e.g., detect J and/or K in a stream of letters

MP

KR

WS

Purdue University

Attentional blink

Turns out that detection of first letter tends to make detection

of the second letter very difficult

if it immediately follows the first

Attentional blink

MP

KR

JS

Prof. Greg Francis

16PSY 310: Sensory and Perceptual Processes

Purdue University

Attentional blink Measure frequency of detection

Implies thatdetecting firstletter causes you to misssecond letter!

Purdue University

Attentional blink Maybe when the J appears, some cell in IT

detects the curve

This suggests it might be a J

But it could be an O, U or S

SLNBJRKH...

J

Image

Retina

Ganglion cells

Orientationcells

ColorMotion

J

Prof. Greg Francis

17PSY 310: Sensory and Perceptual Processes

Purdue University

Attentional blink This cell sends signals back down to other

areas to augment some features

SLNBJRKH...

J

Image

Retina

Ganglion cells

Orientationcells

ColorMotion

J

Purdue University

Attentional blink This leads to strong activation of other cells

in IT

Which overall producse the pattern thatcorresponds to a J

SLNBJRKH...

J

Image

Retina

Ganglion cells

Orientationcells

ColorMotion

J

Prof. Greg Francis

18PSY 310: Sensory and Perceptual Processes

Purdue University

Attentional blink The process takes time and may not be finished

before the J disappears

May continue even while other letters go by

Including the K

Attentional blink

SLNBJRKH...

J

Image

Retina

Ganglion cells

Orientationcells

ColorMotion

K

Purdue University

Conclusions Distributed coding

Features

Neural patterns

Attention to bring it all together

Attentional blink

Prof. Greg Francis

19PSY 310: Sensory and Perceptual Processes

Purdue University

Next time

Review for the exam

You ask questions

I answer questions

Exam 1 on Friday