modeling the visual pathway: some history and a...
TRANSCRIPT
Modeling the Visual Pathway: Some History and a Neuronal
Network Model of V1 SJTU Computational Neuroscience Winter Short Course
12/21/2012
Lecturer: Louis Tao, PKU
taolt @ mail.cbi.pku.edu.cn or letaotao @ pku.edu.cn
© Copyright 2012 Center for Bioinformatics, Peking University
© Copyright 2012 Center for Bioinformatics, Peking University
Line Motion Illusion
Neuronal Networks Are Complex
~1011 neurons & 1015 connections
104 cells & 1 km wiring in 1 mm3 of cortex
Computational Neuroscience
• What “computations” are performed by neurons & neuronal
networks?
• How are these computations done?
• What?
– Feature detection (visual system, olfactory system, …)
– Coincidence / timing (auditory system)
– Memory (hippocampus)
– Sensory-motor (eye saccades, …)
– Neural Code: firing rate, spike timing
• How?
– Cell level: molecular and biophysical
– Network & systems level
• What & How to Study?
– Cellular: membrane potential, ion channels, synaptic mechanisms
– Extracellular: firing rates, spike times, statistics of spike trains, …
– Systems: fMRI, optical imaging, …
Primary Visual Cortex (V1)
Lateral Geniculate
Nucleus (LGN)
V1 & the Visual Pathway
V1 laminaeV1 laminae
4C primary input layer
• A neuron receives inputs via dendrites (thick) and sends
outputs via axons (thin)
• Neurons are ‘connected’ via synapses
Callaway Ann Rev Neurosci 1998
Pre-synaptic neuron
axon
Post-synaptic neuron
dendrite
Excitatory 4C
neurons
Yobuta & Callaway
1998
Excitatory
Pre-synaptic
Post-synaptic
V
Excitatory neuronal action
potentials induce
positive changes in
postsynaptic membrane
potential
V1 Inhibitory Neurons
Wiser & Callaway 1996 Inhibitory
Pre-synaptic
neuron
Post-synaptic
V
Marino et al, Nature Neurosci 2005
Local, inter-laminar
connectivity tends to
be isotropic
Exc/Inh lengthscales
are roughly the same
Fitzpatrick et al ’85, Lund
‘87, Callaway & Wiser ‘96,
Marino et al ‘05
Linear Systems Analysis: Early Visual System
• Concept of the Receptive Field as a Model
• Input / Output Analysis
• Experimental Results
Recording of retinal ganglion cells by Kuffler (1950s)
Hubel & Wiesel Movie!!!
Reverse-Correlation Methods
1
1, , , ,
n
i
i
C x y s x y tn
Spike-triggered average stimulus:
Increment counter
corresponding to this
stimulus by +1
Reverse-Correlation Methods
1
1, , , ,
n
i
i
C x y s x y tn
0
1, , , ,
T
rsQ x y r t s x y t dtT
, ,
, , rsQ x yC x y
r
/r n T
Spike-triggered average stimulus:
Input-output correlation (between stimulus and firing rate):
Where the firing rate is
The two can be related by
Linear Systems Analysis : Early Visual System
, , ,s tD x y D x y D
For most RGC and LGN cells,
we can model D as separable in space and in time:
Linear Systems Analysis : Early Visual System
2 2 2 2
2 2 2 2, , exp exp
2 2 2 2
cen sur
t t
cen cen sur sur
D Dx y x yD x y
, , ,s tD x y D x y D
For most RGC and LGN cells, we can model D as separable in space and in time:
DeAngelis and Freeman, ‘97
2
0( ) ( ) ( , )( )
t
kk B tR t R ds d x A xD x It s x s
Benardete and Kaplan, Visual Neurosci 1999
Linear Systems Analysis : Early Visual System
, 2 2
, , , ,exp expcen sur
t cen sur cen sur cen sur cen surD
Linear Systems Analysis : Early Visual System
Summary Linear Systems Analysis Applied to RGC and LGN cells
• Approximate linearity
• First kernel measured using reverse correlation
• Receptive fields
• Model of spatial RF (difference of Gaussians)
• Model of temporal RF (difference of exponentials)
• Nonlinearity (e.g., Y/M cells in retina, …)
• Success! -> further along the visual pathway…
Many neurons in
V1 are selective
for stimulus
orientation,
stimulus direction,
stimulus phase,
On/Off regions
j
q
Anderson et al, Science 2000
j
q
V1 neurons are orientation-selective
high contrast
medium
low
0( , ) 1 sin( )x t I t k xI j
Drifting Grating
cos , sin , k k k k kq q
grating contrast
Orientation selectivity is “independent” of stimulus contrast
(so-called “contrast invariance”)
Cortical Map of Orientation Preference
right eye
left eye
----
0.5mm
----
Bla
sdel (1
992)
Typical experimental setup for
optical imaging
(above figure taken from Tsodyks et al 1999)
Show Larry’s Movie!!!
1 mm
orientation
hypercolumn
Simple & Complex Classification
Hubel & Wiesel (1962):
- Simple: “linear” spatio-temporal filters
Contrast Reversal:
(1) temporal response at driving frequency
(2) sensitive dependence on spatial phase
(grating location);
- Complex: everything else
Contrast Reversal :
(1) frequency doubled
(2) phase insensitive
V1: 40% Simple
q
j
Simple Complex
j
Contrast Reversal at 8 Different Spatial Phases
De Valois et al. (Vis. Res. 1982)
Simple & Complex Classification
Hubel & Wiesel (1962):
- Simple: “linear” spatio-temporal filters
Drifting Grating:
(1) follows grating location (spatial phase);
(2) temporal response at driving frequency
- Complex: everything else
Drifting Grating:
(1) phase insensitive;
(2) time-independent response
V1: 40% Simple
q
j
Simple Complex
0estr t r F L t
0
, , , ,L t d dxdyD x y s x y t
2 2
, ,, ,, , rs
s s
r C x yQ x yD x y
, , ,s tD x y D x y D
Estimate the firing rate:
Where the linear part is given by
D is the first Wiener kernel / spatio-temporal receptive field, and, using “white noise” stimulus, can be estimated from spike-triggered avg. stimulus via
For some neurons, we can model D as separable in space and in time:
2
sVariance of the white noise
Modeling a V1 Simple Cell
2 21
, exp cos2 2 2
s
x y x y
x yD x y kx
Approximate the “receptive field” with a Gabor function
2 21
, exp cos2 2 2
s
x y x y
x yD x y kx
cos sin
sin cos
x x
y y
q q
q q
A more general form of the Gabor function:
Coordinate transformation (here, a rotation)
spatial length-scales (preferred) spatial frequency (preferred) orientation (preferred) spatial phase
x y
k
q
cos , sink k kq q
q
At this point, we have the “receptive field” (i.e., a “model”) of a Simple cell.
Now what?!?!
5 7
exp, 05! 7!
0, 0
tD
2 21
, exp cos2 2 2
s
x y x y
x yD x y kx
0
, , , ,L t d dxdyD x y s x y t
, , ,s tL x y L x y L
0
, cos cos sin
cos
S S
t t
L dxdyD x y A Kx Ky
L t d D t
Response of a Model Simple Cell to a Counterphase Grating
When D is separable, L is separable
0, , 2x y kq
,K k ,q ,K k q
2 2
2exp cos cos cos sin
2S
x yL A dxdy kx Kx Ky
Response of a Model Simple Cell to a Counterphase Grating
0
,K k ,q
2 2 2
2exp cosh cos2
S
k KL A kK
2 2 2
2 2
exp2 2
cos exp cos cos exp cos
S
k KAL
kK kK
2 2
2exp cos cos cos sin
2S
x yL A dxdy kx Kx Ky
Response of a Model Simple Cell to a Counterphase Grating
2 2 2
2 2
exp2 2
cos exp cos cos exp cos
S
k KAL
kK kK
0q
2 2 2
exp cos2 2
S
k KAL
assuming
2exp 0kK
,K k q
Response of a Model Simple Cell to a Counterphase Grating
,K k ,q ,K k q
0q
2 2 2
exp cos2 2
S
k KAL
0
2 2 2
2exp cosh cos2
S
k KL A kK
0
cost tL t d D t
Response of a Model Simple Cell to a Counterphase Grating
5 7
exp, 05! 7!
0, 0
tD
6 2 2
42 2
4costL t t
28arctan arctan
Response of a Model Simple Cell to a Counterphase Grating
6 2 2
42 2
4costL t t
28arctan arctan
Summary: Early Visual System modeling
From RGC to LGN to Simple Cells
• Reverse-correlation methods
• Receptive fields
– Model of spatial receptive fields (Gabor)
– Model of temporal receptive fields
• Nonlinearities (How to model?)
• Again, these are descriptive and not mechanistic models!!!
• Think about extensions to other visual pathway neurons, other sensory neurons, …
Reverse-Time Correlation
B
Yeh et al, PNAS 2009
Simple & Complex Classification
Hubel & Wiesel (1962):
- Simple: “linear” spatio-temporal filters
Contrast Reversal:
(1) temporal response at driving frequency
(2) sensitive dependence on spatial phase
(grating location);
- Complex: everything else
Contrast Reversal :
(1) frequency doubled
(2) phase insensitive
V1: 40% Simple
q
j
Simple Complex
j
Contrast Reversal at 8 Different Spatial Phases
De Valois et al. (Vis. Res. 1982)
Simple & Complex Classification
Hubel & Wiesel (1962):
- Simple: “linear” spatio-temporal filters
Drifting Grating:
(1) follows grating location (spatial phase);
(2) temporal response at driving frequency
- Complex: everything else
Drifting Grating:
(1) phase insensitive;
(2) time-independent response
V1: 40% Simple
q
j
Simple Complex
Martinez & Alonso (2003)
Phenomenology of V1 Complex Cell
Hubel & Wiesel Movie!!!
Phenomenology of V1 Complex Cell
Nonlinear response to counter-phase grating:
1) Invariance to spatial phase of grating
2) (temporal) frequency-doubled response
DeValois et al (1982)
Phenomenology of V1 Complex Cell
Constant response to drifting grating
DeValois et al (1982)
Complex Cell: Modulation Ratio
DeValois et al (1982)
1
. . 1 exp 2A C F m t i t dtT
0
1. . 0
T
D C F m t dtT
1
0 spontaneous rate
F
F Modulation Ratio =
Complex Cell: Modulation Ratio
Skottun et al (1991)
1
0 spontaneous rate
F
F Modulation Ratio =
Complex Cell: Modulation Ratio
Ibbotson et al (2005)
1
0 spontaneous rate
F
F Modulation Ratio =
Physiology of Complex Cells
• Receptive fields do not have on-, off- subunits
• Complex cells show selectivity to orientation, spatial frequency, but not spatial phase
• Constant response to drifting gratings
• Frequency-doubled response to counter-phase gratings
• Model?
Modeling Complex Cells
• Let us suppose there are 2 Simple cells
Also frequency-doubled response!
0q
1 , cos cosL AB K t
2 , sin cosL AB K t
preferred phase
preferred phase
/ 2
2 2 2 2 2 2 2
1 2
1, cos , 1 cos 2
2L L A B K t A B K t
2 2
1 2F L G L L
0estr t r F L t
Independent of grating phase
Static Nonlinearities: Complex Cells
2 2 2 2 2 2 2
1 2
1, cos , 1 cos 2
2L L A B K t A B K t
Static Nonlinearities: Complex Cells
From Ringach ‘04
Hubel-Wiesel Model of V1 Receptive Fields (aka Hierarchical Model)
Hubel-Wiesel Model of V1 Receptive Fields (aka Hierarchical Model)
Summary Reverse-Correlation Methods and Receptive Fields
• Complex cells
• Hubel-Wiesel Model of V1 receptive fields
• Problems with the HW model …
• Miscellaneous:
– Higher order Wiener kernels???
– Beyond LN models, dynamics and function
Convergent LGN input confers orientation & spatial phase preference
V1 Simple
Cell
Reid & Alonso ‘95
(q,j) preference
Hubel & Wiesel ‘62
Hubel-Wiesel Model of V1 Receptive Fields Experimental support for Simple cells
The Hierarchical Model (Hubel & Wiesel, 1962)
Two Distinct Populations?!?!
The Classical View: Experiment
S S
LGN
S
C
}
LGN drives Simple cells, whose summed
outputs drive Complex cells
Ringach, Shapley & Hawken, 2002
Comparison to experiment (data from
Ferster and colleagues - replotted by F. Mechler)
But unimodal F1/F0 distribution measured in
experiments (cat)
Some Problems of Orientation Tuning
Let’s consider drifting gratings
• Individual RG/LGN cells are not orientation selective
• Therefore, average response of a single LGN cell does not change with orientation
• Therefore, the sum of the average responses of a collection of LGN cells does not change with orientation
• Therefore, the average synaptic input into a Simple cell does not change with orientation
• Where does its orientation selectivity come from?
Some Problems of Orientation Tuning
• Contrast invariance
• Noise?
Anderson et al Science 2000
j
q
Contrast Invariance Orientation Selectivity
Contrast Invariant Orientation Selectivity
high contrast medium low
0( , ) 1 sin( )x t I t k xI j
Drifting Grating
cos , sin , k k k k kq q
grating contrast
Features of a V1 Neuronal Network Model
~ 4,000 I & F neurons, 1 mm2, local patch of V1 4Ca
KijEXC Gaussian with EXC = 200 mm (spatial coupling isotropic)
tjk = kth spike-time of jth neuron
Fi(t) models LGN forcing and activity in other layers
[ gi
INH(t) similar, without LGN drive, INH = 100-200 mm EXC ]
( ) ( ) ( )j j
E I
jj j j
L R E Idv
g v V g v V g v Vdt
( ) ( ) ( )j
E
j E kEE j k l
k l
g t F t S K G t t
( ) ( ) ( )j j j
LGN noiseF t f t f t ( ) ( )j kELGN E l
k l
f t c G t s
Features of a V1 Neuronal Network Model
• Convergent LGN input confers orientation & spatial phase preference
(Reid & Alonso, ‘95)
V1
Reid & Alonso ‘95
(q,j) preference
of cortical neuron
Hubel & Wiesel ‘62
Features of a V1 Neuronal Network Model
• Convergent LGN input confers orientation & spatial phase preference
(Reid & Alonso, ‘95)
Regular Map of Orientation in Pinwheels
(Optical Imaging: Bonhoeffer & Grinvald
1991; Blasdel 1992; Maldonado et al. 1997)
Random Map of Spatial Phase
(DeAngelis et al. 1999: preferred phase
of 2 nearby neurons uncorrelated)
Features of a V1 Neuronal Network Model
• Convergent LGN input confers orientation & spatial phase preference
(Reid & Alonso, ‘95)
• Variability in strength of LGN excitation (Tanaka, ’86); total excitation
(LGN + Cortex) roughly constant (Miller ’96, Royer & Pare, ’02)
I
E
I
E
LGN
Simple Complex
Inhibitory
Excitatory
V1
Features of a V1 Neuronal Network Model
• Convergent LGN input confers orientation & spatial phase preference
(Reid & Alonso, ‘95)
• Variability in strength of LGN excitation (Tanaka, ’86); total excitation
(LGN + Cortex) roughly constant (Miller ’96, Royer & Pare, ’02)
• Local (<500 mm) connections isotropic & non-specific (Fitzpatrick et al.,
’85; Lund, ’87; Callaway & Wiser, ’96; Marino et al, ’05)
Features of a V1 Neuronal Network Model
• Convergent LGN input confers orientation & spatial phase preference
(Reid & Alonso, ‘95)
• Variability in strength of LGN excitation (Tanaka, ’86); total excitation
(LGN + Cortex) roughly constant (Miller ’96, Royer & Pare, ’02)
• Local (<500 mm) connections isotropic & non-specific (Fitzpatrick et al.,
’85; Lund, ’87; Callaway & Wiser, ’96; Marino et al, ’05)
• Cortical inhibition dominant (Borg-Graham et al, ’98; Hirsch et al, ’98,
Anderson et al, ’00)
Features of a V1 Neuronal Network Model
• Convergent LGN input confers orientation & spatial phase preference
(Reid & Alonso, ‘95)
• Variability in strength of LGN excitation (Tanaka, ’86); total excitation
(LGN + Cortex) roughly constant (Miller ’96, Royer & Pare, ’02)
• Local (<500 mm) connections isotropic & non-specific (Fitzpatrick et al.,
’85; Lund, ’87; Callaway & Wiser, ’96; Marino et al, ’05)
• Cortical inhibition dominant (Borg-Graham et al, ’98; Hirsch et al, ’98,
Anderson et al, ’00)
• Dynamics of individual neurons dominated by fluctuations
Data from Anderson et al, Science 2000
Membrane potential: Average over trials (left, 3 contrasts)
vs. Individual trials (right, at medium contrast)
Response to CR: Comparison to Experiments
Simple Complex Contrast Reversal at 8 different spatial phases
DeValois et al. 1982
orthogonal
preferred
Response to DG: Comparison to Experiments
(2 / )
1 00 0
/ ( ) ( )i tF F dt m t e dt m t
F1/F0 Distributions: Comparison to Experiments
Intracellular f1/f0
Extracellular F1/F0
Model of Tao et al (2006)
Model of V1 Simple Cell
• On- / off- segregated input from LGN gives frequency doubled input at “orthogonal” phase
• At “orthogonal” phase, the LGN input looks like a complex cell output!
Wielaard et al (2001)
Model of V1 Simple Cell
Wielaard et al (2001)
• Cortical “inputs” also appear to be frequency-doubled in a network of Simple cells!
• Why?
Model of V1 Simple Cell
Wielaard et al (2001)
• Cortical “inputs” also appear to be frequency-doubled in a network of Simple cells!
• Why?
Model of V1 Complex Cell
Tao and Cai, Acta Physiol. Sinica (review of network model of V1 Simple and Complex cells); nonlinear network model of “HW mechanism”, explained in Tao et al PNAS (2004) ; interesting issues with network dynamics and stability, see Tao et al PNAS (2006)
Model of V1 Simple Cell
Nonlinear LGN Input gLGN at Various Spatial Phases
Simple Cell at Preferred & Orthogonal Phase
(LGN & cortex)
"Simple" Response: Basically, an
interaction between LGN excitation, cortical inhibition, and thresholding
Frequency-doubled & phase-insensitive cortical inhibition removes frequency-doubled LGN excitation at orthogonal phase
Wielaard, Shelley, McLaughlin, Shapley JNS 2001
Two Distinct Populations???
Mechler & Ringach (2002)
Mechanism of Orientation Selectivity
• Recurrent I & F network
• Role of coupling length-scales
• Contrast invariance
• Mean vs. fluctuation driven
• Relation to bistable states
Anderson et al
Science 2000
Tao et al, PNAS 2006
i
i i i i i
L L exc E inh I
dVg V V g V V g V V
dt
Whenever , the neuron fires and i
ThresV t V i
resetV t V
T S
dVg t V V t
dt
S ThresV V
S ThresV V
“Mean-driven”
“Fluctuation-driven”
i i ij k
exc LGN exc exc EXC j
j k
g t F t S K G t t
2 1 2 1/ / / /
2 1 2 1
1 AMPA AMPA NMDA NMDAt t t tNMDA NMDAEXC AMPA AMPA NMDA NMDA
f fG t e e e e
1 2 1 2 1 211 , 5 , 2 , 80 , , 10A AAMPA AMPA NMDA NM GABA GABADAms ms ms mms sms
Individual neurons are orientation selective and contrast invariant
Firing Rate at 2 contrasts
Time Avg Membrane Potential
LGN Input
Cortical Excitation
Cortical Inhibition
Neuronal populations are orientation selective and contrast invariant
CV = circular variance
2
0 01 ( ) ( )iCV dt m t e dt m t
q CV ~ 0 Selective
CV ~ 1 Not selective
Sparsity vs. Synaptic Failure
• Synaptic failure: Self-averaging system
• Sparsity: consequences in spatially extended systems (spatial homogenization)
Network model results consistent with experiments of Sur’s lab at MIT
showing similar orientation selectivity across cortex (Marino et al. 2005)
(independent of location to pinwheel singularity)
Effective coupling orientation is a function of location
Co-variation of tuning
of E/I conductances
Fluctuation-Controlled Critical States
Intrinsic Fluctuations in Recurrent Excitation controlled by
• AMPA vs. NMDA (fast vs. slow synapses)
• finite-size networks, sparse coupling, and synapatic failure
Tao et al, PNAS 2006
Some Problems with Linear Systems Analysis Applied to V1
• Different stimulus gives different ‘receptive fields’ – Spots/dots, bars, gratings,… give different answers!
– Stimulus not ‘white’ noise
– Not nearly the case with retina, thalamus, …
• Fundamentally nonlinear! – Linear model inadequate to
explain many data
– Network Model – Kanizsa Triangle
– lm1 movie
Some References
Dayan, P and Abbott, L.F., Theoretical Neuroscience
Enroth-Cugell, C. and Robson, J.G., “The contrast sensitivity of retinal ganglion cells of the cat,” J. Physiol. 187, 517 (1966)
Skottun, B.C., et al, “Classifying Simple and Complex cells on the basis of response modulation,” Vision Res. 31, 1079 (1991)
Martinez, L.M. and Alonso, J.-M., “Complex receptive fields in primary visual cortex,” The Neuroscientist 9, 317 (2003)
Ringach, D.L., “Mapping receptive fields in primary visual cortex,” J. Physiol. 558, 717 (2004)
Wielaard D.J. et al, “How Simple cells are made in a nonlinear network model of the visual cortex,” J. Neurosci. 21, 5203 (2001)
Mechler, F. and Ringach, D.L., “On the classification of Simple and Complex cells,” Vision Res. 42, 1017, 2002
Rangan et al. PNAS, 2005
Tao, et al. PNAS 2004, 2006
Some of My Other Work
• Optical imaging of neural circuits
– Zebrafish
– C. Elegans: simultaneous tracking behavior and imaging activity of individual neurons
• Empirical, data-driven dimension reduction
• More visual modeling …