Visual Neuron Responses

• This conceptualization of the visual system was “static” - it did not take into account the possibility that visual cells might change their response selectivity over time and it was firmly based in the classical notion of a receptive field

– Logic went like this: if the cell is firing, its preferred line/edge must be present and…

– if the preferred line/edge is present, the cell must be firing

• We will encounter examples in which these don’t apply!

• Representing boundaries and surfaces must be more complicated than simple edge detection! WHY??

What can a visual neuron “know” about the image?

• If a neuron is only an edge detector and/or only has a small receptive field, it can’t represent information about the relationship between the contents of its receptive field and other receptive fields elsewhere in the display.

What can a visual neuron “know” about the image?

• If a neuron is only an edge detector and/or only has a small receptive field, it can’t represent information about the relationship between the contents of its receptive field and other receptive fields elsewhere in the display.

• For example the famous 1961 Rosebowl hoax…no single person could know what the big picture showed

What can a visual neuron “know” about the image?

• If a neuron is only an edge detector and/or only has a small receptive field, it can’t represent information about the relationship between the contents of its receptive field and other receptive fields elsewhere in the display.

• Also the 2004 Harvard – Yale Game:

Visual Neuron Responses

• Edges are important because they are the boundaries between objects and the background or objects and other objects

Visual Neuron Responses

• Boundaries between objects can be defined by color rather than brightness

Visual Neuron Responses

• Boundaries between objects can be defined by texture

Visual Neuron Responses

• Boundaries between objects can be defined by motion and depth cues

Visual Neuron Responses

• Boundaries between objects can be defined by motion and depth cues

Feed-Forward and Feed-Back Processing in the Visual System

The Feed-Forward Sweep

• What is the feed-forward sweep?

The Feed-Forward Sweep

• The feed-forward sweep is the initial response of each visual area “in turn” as information is passed to it from a “lower” area

• Characteristics:– a single spike per synapse

– no time for lateral connections

– no time for feedback connections

The Feed-Forward Sweep

• The feed-forward sweep is the initial response of each visual area “in turn” as information is passed to it from a “lower” area

• What does it mean for an area to be “lower” or “higher”

The Feed-Forward Sweep

• Hierarchy of visual cortical areas defined anatomically

Dorsal “where”/”how”

Ventral “what”

Notice the direct connection from SC to MT/V5

The Feed-Forward Sweep

• Hierarchy can be defined more functionaly

• The feed-forward sweep is the initial response of each visual area “in turn” as information is passed to it from a “lower” area

• Consider the latencies of the first responses in various areas

The Feed-Forward Sweep

• Thus the “hierarchy” of visual areas differs depending on temporal or anatomical features

• aspects of the visual system account for this fact:

– multiple feed-forward sweeps progressing at different rates (I.e. magno and parvo pathways) in parallel

• M pathway is myelinated

• P pathway is not

– signals arrive at cortex via routes other than the Geniculo-striate pathway (LGN to V1)

• Will be important in understanding blindsight

The Feed-Forward Sweep

• The feed-forward sweep gives rise to the “classical” receptive field properties– tuning properties exhibited in very first spikes

• Orientation tuning in V1

• Optic flow tuning in MST

– think of cortical neurons as “detectors” only during feed-forward sweep

After the Forward Sweep

• By 150 ms, virtually every visual brain area has responded to the onset of a visual stimulus

• But visual cortex neurons continue to fire for hundreds of milliseconds!

After the Forward Sweep

• By 150 ms, virtually every visual brain area has responded to the onset of a visual stimulus

• But visual cortex neurons continue to fire for hundreds of milliseconds!

• What are they doing?

After the Forward Sweep

• By 150 ms, virtually every visual brain area has responded to the onset of a visual stimulus

• But visual cortex neurons continue to fire for hundreds of milliseconds!

• What are they doing?

• with sufficient time (a few tens of ms) neurons begin to reflect aspects of cognition other than “detection”

Extra-RF Influences

• One thing they seem to be doing is helping each other figure out what aspects of the entire scene are contained within a given receptive field

– That is, the responses of visual neurons begin to change to reflect global rather than local features of the scene

– recurrent signals sent via feedback projections are thought to mediate these later properties

Extra-RF Influences

Note that these are responses to the same stimulus!

Extra-RF Influences

• consider texture-defined boundaries

– classical RF tuning properties do not allow neuron to know if RF contains figure or background

– At progressively later latencies, the neuron responds differently depending on whether it is encoding boundaries, surfaces, the background, etc.

Extra-RF Influences

• Consider this analogy:

– Imagine when each fan puts up a card he or she is told to shake it – so that the entire scene is full of shaking cards

– After some delay, the fans holding up the red cards are told to keep shaking but the fans holding white cards are told to stop…the words will be enhanced

– But the fans can’t each figure that out on their own because they don’t actually know the color of the card they are holding

Extra-RF Influences

• How do these data contradict the notion of a “classical” receptive field?

Extra-RF Influences

• How do these data contradict the notion of a “classical” receptive field?

• Remember that for a classical receptive field (i.e. feature detector):

– If the neuron’s preferred stimulus is present in the receptive field, the neuron should fire a stereotypical burst of APs

– If the neuron is firing a burst of APs, its preferred stimulus must be present in the receptive field

Extra-RF Influences

• How do these data contradict the notion of a “classical” receptive field?

• Remember that for a classical receptive field (i.e. feature detector):

– If the neuron’s preferred stimulus is present in the receptive field, the neuron should fire a stereotypical burst of APs

– If the neuron is firing a burst of APs, its preferred stimulus must be present in the receptive field

Recurrent Signals in Object Perception

• Can a neuron represent whether or not its receptive field is on part of an attended object?

• What if attention is initially directed to a different part of the object?

Recurrent Signals in Object Perception

• Can a neuron represent whether or not its receptive field is on part of an attended object?

• What if attention is initially directed to a different part of the object?

Yes, but not during the feed-forward sweep

Recurrent Signals in Object Perception

• curve tracing

– monkey indicates whether a particular segment is on a particular curve

– requires attention to scan the curve and “select” all segments that belong together

– that is: make a representation of the entire curve

– takes time

Recurrent Signals in Object Perception

• curve tracing

– neuron begins to respond differently at about 200 ms

– enhanced firing rate if neuron is on the attended curve

Feedback Signals and the binding problem

• What is the binding problem?

Feedback Signals and the binding problem

• What is the binding problem?

• curve tracing and the binding problem:

– if all neurons with RFs over the attended curve spike faster/at a specific frequency/in synchrony, this might be the binding signal

Feedback Signals and the binding problem

• So what’s the connection between Attention and Recurrent Signals?

Feedback Signals and Attention

• One theory is that attention (attentive processing) entails the establishing of recurrent “loops”

• This explains why attentive processing takes time - feed-forward sweep is insufficient

Feedback Signals and Attention

• Instruction cues (for example in the Posner Cue-Target paradigm) may cause feedback signal prior to stimulus onset (thus prior to feed-forward sweep)

• think of this as pre-setting the system for the upcoming stimulus

• What does this accomplish?

Feedback Signals and Attention

• What does this accomplish?

• Preface to attention: Two ways to think about attention– Attention improves perception, acts as a gateway to memory

and consciousness

– Attention is a mechanism that routes information through the brain

• It is the brain actively reconfiguring itself by changing the way signals propagate through networks

• It is a form of very fast, very transient plasticity

Feedback Signals and Attention• Put another way:

– It may strike you as remarkable that a single visual stimulus should “activate” so many brain areas so rapidly

– In fact it should be puzzling that a visual input doesn’t create a runaway “chain reaction”

• The brain is massively interconnected

• Why shouldn’t every neuron respond to a visual stimulus

Feedback Signals and Attention

• We’ll consider the role of feedback signals in attention in more detail as we discuss the neuroscience of attention