characterization of nonlinear neuron responsesrvbalan/teaching/amsc663fall2013...characterization of...
TRANSCRIPT
![Page 1: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/1.jpg)
CHARACTERIZATION OF
NONLINEAR NEURON
RESPONSES
AMSC 663 Mid Year Presentation
Matt Whiteway
Dr. Daniel A. Butts
Neuroscience and Cognitive Science (NACS)
Applied Mathematics and Scientific Computation (AMSC)
Biological Sciences Graduate Program (BISI)
![Page 2: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/2.jpg)
Background
Introduction STA GLM Conclusion
The human brain contains 100 billion neurons
These neurons process information nonlinearly, thus
making them difficult to study
INPUTS OUTPUTS
Given the inputs and the outputs, how can we model
the neuron’s computation?
[1]
[1] http://msjensen.cehd.umn.edu/webanatomy_archive/Images/Histology/
![Page 3: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/3.jpg)
The Models
Many models of increasing complexity have been developed
The models I will be implementing are based on statistics
Linear Models – Linear Nonlinear Poisson (LNP) Model
1. LNP using Spike Triggered Average (STA)
2. LNP using Maximum Likelihood Estimates – Generalized Linear Model (GLM)
3. Spike Triggered Covariance (STC)
Nonlinear Models
4. Generalized Quadratic Model (GQM)
5. Nonlinear Input Model (NIM)
Introduction STA GLM Conclusion
![Page 4: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/4.jpg)
The Models - LNP
INPUT OUTPUT
[2]
Unknowns
is a linear filter, defines the
neuron’s stimulus selectivity
is a nonlinear function
is the instantaneous rate
parameter of an non-
homogenous Poisson process
Knowns
is the stimulus vector
Spike times
[2] http://www.sciencedirect.com/science/article/pii/S0079612306650310
Introduction STA GLM Conclusion
![Page 5: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/5.jpg)
STA1
The STA is the average stimulus preceding a spike in
the output, where is the number of spikes and
is the set of stimuli preceding a spike
[3]
1. Chichilnisky, E.J. (2001) A simple white noise analysis of neuronal light responses.
[3] http://en.wikipedia.org/wiki/Spike-triggered_average
Introduction STA GLM Conclusion
![Page 6: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/6.jpg)
-150.1 -125.1 -100.1 -75.1 -50.0 -25.0 0.0
0
time (ms)
STA Implementation
Introduction STA GLM Conclusion
Filter length
of 20 time
steps
Upsampling
factor of 1
![Page 7: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/7.jpg)
STA Implementation
Resolution of the filter is determined by time interval
between measurements
We can artificially increase resolution by upsampling
the stimulus vector:
Upsample by 1:
Upsample by 2:
Introduction STA GLM Conclusion
DTstim
DTstim/2
![Page 8: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/8.jpg)
STA Implementation
Introduction STA GLM Conclusion
Filter length
of 15 time
steps
-125.1 -100.1 -75.1 -50.0 -25.0 0.0
0
time (ms)
upsample by 1
upsample by 2
upsample by 4
![Page 9: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/9.jpg)
STA Implementation
For the STA, a common approach to finding the
nonlinear response function is to use the histogram
method
Method creates bins for the generator signal, ,
and plots average number of spikes for each bin
Introduction STA GLM Conclusion
![Page 10: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/10.jpg)
STA Implementation
Introduction STA GLM Conclusion
Filter length
of 15 time
steps
Upsampling
factor of 1
Avera
ge S
pik
e C
ount
-5 -4 -3 -2 -1 0 1 2 3 4 5
Generator Signal
-5 -4 -3 -2 -1 0 1 2 3 4 50
0.5
1
Generator Signal Distribution
Response Function
![Page 11: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/11.jpg)
STA Validation
For filter validation, I
created a stimulus with Gaussian random noise
added an artificial filter at random points
Recorded a spike for each instance of the artificial filter
If the STA code is working properly, and if none of
the artificial filters overlap, then the code should
exactly recover the artificial filter
Introduction STA GLM Conclusion
![Page 12: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/12.jpg)
STA Validation
Introduction STA GLM Conclusion
0 10 20 30 40 50-3
-2
-1
0
1
2
3
Random Stimulus
time (units of DTstim)
No Spikes
0 10 20 30 40 50-3
-2
-1
0
1
2
3
Random Stimulus with Filter
time (units of DTstim)
Spikes After Filter
![Page 13: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/13.jpg)
STA Validation
Introduction STA GLM Conclusion
filter length: 10
stimulus length: 15000
20 spikes: No overlap of filters in stimulus, STA code recovers exact filter
3000 spikes: Substantial overlap of filters in stimulus, STA code recovers exact filter with some error
0 5 10
-1
0
1
Recovered Filter for 20 spikes
0 5 10
-1
0
1
Original Filter for 20 spikes
0 5 10-0.05
0
0.05Error
time (units of DTstim)
0 5 10
-1
0
1
Recovered Filter for 3000 spikes
0 5 10
-1
0
1
Original Filter for 3000 spikes
0 5 10-0.1
0
0.1Error
time (units of DTstim)
![Page 14: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/14.jpg)
The Models - LNP
INPUT OUTPUT
[2]
Unknowns
is a linear filter, defines the
neuron’s stimulus selectivity
is a nonlinear function
is the instantaneous rate
parameter of an non-
homogenous Poisson process
Knowns
is the stimulus vector
Spike times
[2] http://www.sciencedirect.com/science/article/pii/S0079612306650310
Introduction STA GLM Conclusion
![Page 15: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/15.jpg)
GLM2
Now we will approximate the linear filter using the
Maximum Likelihood Estimate (MLE)
A likelihood function is the probability of an outcome
given a probability density function with parameter
The LNP model uses the Poisson distribution
where is the vector of spike counts binned at a resolution
We want to maximize a log-likelihood function
2. Paninski, L. (2004) Maximum Likelihood estimation of cascade point-process neural encoding models.
Introduction STA GLM Conclusion
![Page 16: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/16.jpg)
GLM Implementation
Can employ likelihood optimization methods to obtain
maximum likelihood estimates for linear filter
If we make some assumptions about the form of the
nonlinearity F, the likelihood function has no non-
global local maxima – gradient ascent!
F(u) is convex in u
log(F(u)) is concave in u
I use F(u) = log(1+exp(u-c))
Introduction STA GLM Conclusion
![Page 17: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/17.jpg)
GLM Implementation
Originally coded a gradient descent method – took too many function evaluations
About 1000 iterations for a filter of length 15 at ~1s per function evaluation
Next used a Newton-Raphson method – less iterations, but needed to compute Hessian
About 150 for a filter of length 15 at ~2s per function evaluation
Need a quasi-Newton method
Now use Matlab’s fminunc
About 10 – 150 iterations at ~1s per function evaluation
Introduction STA GLM Conclusion
![Page 18: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/18.jpg)
GLM Implementation
Introduction STA GLM Conclusion
Filter length
of 15 time
steps
0 -125.1 -100.1 -75.1 -50.0 -25.0 0.0
0
time (ms)
upsampling by 1
upsampling by 2
upsampling by 4
![Page 19: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/19.jpg)
GLM Implementation
For the GLM, finding the parameters to the
nonlinearity can be done at the same time as finding
the filter
Assume the parametric form F(u) and include its
parameter(s) in the optimization
Use log(1+exp(x-c)), fit offset c
Introduction STA GLM Conclusion
![Page 20: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/20.jpg)
GLM Implementation
Introduction STA GLM Conclusion
Filter length
of 15 time
steps
Upsampling
factor of 1
-60 -40 -20 0 20 40 60
Nonlinear Response Function
Generator Signal-60 -40 -20 0 20 40 60
20
40
60
Firin
g R
ate
(H
z)
![Page 21: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/21.jpg)
GLM Implementation
We can also use regularization to add additional prior
knowledge about solution attributes
We know the filters should be smoothly varying in time
Penalize large curvatures in filter
Laplacian gives us the second derivative; we want to maximize
likelihood while minimizing the L2 norm of the Laplacian of the filter
λ is a parameter that is not explicitly part of the
model, hence called a hyperparameter
Introduction STA GLM Conclusion
![Page 22: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/22.jpg)
GLM Implementation
How to choose optimal λ?
For a variety of λ values,
fit model parameters using part of the data (80%)
validate model on rest of the data (20%)
Introduction STA GLM Conclusion
![Page 23: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/23.jpg)
GLM Implementation
Introduction STA GLM Conclusion
Filter length
of 15 time
steps
Upsampling
factor of 2
0 500 1000 1500 2000
4,000
4,200
4,400
4,600
4,800
5,000
5,200
Regularization Hyperparameter
Mo
difie
d L
og
Lik
elih
oo
d o
f T
rain
ing
Set
0 500 1000 1500 2000
-2,300
-2,200
-2,100
-2,000
-1,900
-1,800
-1,700
Log
Lik
elih
oo
d o
f V
alid
ation
Set
![Page 24: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/24.jpg)
GLM Implementation
Introduction STA GLM Conclusion
-125.1 -100.1 -75.1 -50.0 -25.0 0.0
0
time (ms)
= 0
= 700
= 10000
![Page 25: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/25.jpg)
GLM Implementation
Introduction STA GLM Conclusion
Filter length
of 15 time
steps
Upsampling
factor of 4
50 150 250 350 4503600
3620
3640
3660
3680
3700
Regularization Hyperparameter
Modifie
d L
og L
ikelih
ood o
f T
rain
ing S
et
50 150 250 350 450-3218
-3214
-3210
-3206
-3202
Log L
ikelih
ood o
f V
alid
ation S
et
![Page 26: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/26.jpg)
GLM Implementation
Introduction STA GLM Conclusion
-100.1 -75.1
time (ms)
= 0
= 360
![Page 27: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/27.jpg)
Schedule
Introduction STA GLM Conclusion
PHASE I (October-December)
Implement and validate the LNP model using the STA
(October)
Develop code for gradient descent method and validate
(October)
Done, but not efficient enough. I am currently using MATLAB’s
fminunc command instead
Implement and validate the GLM with regularization
(November-December)
Complete mid-year progress report and presentation
(December)
![Page 28: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/28.jpg)
Schedule
Introduction STA GLM Conclusion
PHASE II (January-May)
Implement quasi-Newton method for gradient descent
(January)
Implement and validate the LNP model using the STC
(January-February)
Implement and validate the GQM with regularization
(February)
Implement and validate the NIM with regularization using
rectified linear upstream functions (March)
Test all models (April)
Complete final report and presentation (April-May)
![Page 29: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/29.jpg)
References
Chichilnisky, E.J. (2001) A simple white noise analysis of neuronal light responses. Network: Comput. Neural Syst., 12, 199-213.
Schwartz, O., Chichilnisky, E. J., & Simoncelli, E. P. (2002). Characterizing neural gain control using spike-triggered covariance. Advances in neural information processing systems, 1, 269-276.
Paninski, L. (2004) Maximum Likelihood estimation of cascade point-process neural encoding models. Network: Comput. Neural Syst. ,15, 243-262.
Schwartz, O. et al. (2006) Spike-triggered neural characterization. Journal of Vision, 6, 484-507.
Paninski, L., Pillow, J., and Lewi, J. (2006) Statistical models for neural encoding, decoding, and optimal stimulus design.
Park, I., and Pillow, J. (2011) Bayesian Spike-Triggered Covariance Analysis. Adv. Neural Information Processing Systems ,24, 1692-1700.
Butts, D. A., Weng, C., Jin, J., Alonso, J. M., & Paninski, L. (2011). Temporal precision in the visual pathway through the interplay of excitation and stimulus-driven suppression. The Journal of Neuroscience, 31(31), 11313-11327.
McFarland, J.M., Cui, Y., and Butts, D.A. (2013) Inferring nonlinear neuronal computation based on physiologically plausible inputs. PLoS Computational Biology.
Introduction STA GLM Conclusion
![Page 30: CHARACTERIZATION OF NONLINEAR NEURON RESPONSESrvbalan/TEACHING/AMSC663Fall2013...CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 663 Mid Year Presentation Matt Whiteway whit8022@umd.edu](https://reader034.vdocument.in/reader034/viewer/2022042315/5f0442297e708231d40d16f4/html5/thumbnails/30.jpg)
Figures
1. http://msjensen.cehd.umn.edu/webanatomy_archive
/Images/Histology/
2. http://www.sciencedirect.com/science/article/pii/S
0079612306650310
3. http://en.wikipedia.org/wiki/Spike-
triggered_average
Introduction STA GLM Conclusion