understanding neural content and coding using population … · 2020-04-23 · the neural decoding...
TRANSCRIPT
![Page 1: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/1.jpg)
Using population decoding to understand
neural content and coding
1
![Page 2: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/2.jpg)
Motivation
We have some great theory about how the brain works
We run an experiment and make neural recordings
We get a bunch of data…
How can we convert data into answers?
2
![Page 3: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/3.jpg)
What do I want from a data analysis method?
Clear answers to:
Neural content: What information is in a brain region?
Neural coding: What features of the activity contain information?
© Wired. All rights reserved. This contentis excluded from our Creative Commonslicense. For more information, seehttps://ocw.mit.edu/help/faq-fair-use/.
© Source Unknown. All rights reserved.This content is excluded from our CreativeCommons license. For more information,see https://ocw.mit.edu/help/faq-fair-use/.
© Orion Pictures. All rights reserved. Thiscontent is excluded from our CreativeCommons license. For more information,see https://ocw.mit.edu/help/faq-fair-use/.
3
![Page 4: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/4.jpg)
Talk Outline
What is population decoding?
Using decoding to understand neural content
Using decoding to understand neural coding
How to analyze your own data
4
![Page 5: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/5.jpg)
Neural population decoding
Decoding: Predict stimulus/behavior from neural activity
f(neural activity)
Decoding approaches have been used for 30 years• Motor system: e.g., Georgopoulos et al, 1986
• Hippocampus: e.g., Wilson and McNaughton, 1993
• Computational work: e.g., Salinas and Abbott, 1994
Alternative names for decoding: • Multivariate Pattern Analysis (MVPA)
• Readout
stimulus
5
![Page 6: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/6.jpg)
Training the classifier
neuron 1
neuron 2
neuron 3
neuron n
Pattern Classifier
Learning association betweenneural activity an image
6
![Page 7: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/7.jpg)
Training the classifier
neuron 1
neuron 2
neuron 3
neuron n
Pattern Classifier
Learning association betweenneural activity an image
7
![Page 8: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/8.jpg)
Using the classifier
neuron 1
neuron 2
neuron 3
neuron n
Pattern Classifier
Prediction“Correct”
8
![Page 9: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/9.jpg)
Using the classifier
Pattern Classifier
“Incorrect”
neuron 1
neuron 2
neuron 3
neuron n
Prediction
9
![Page 10: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/10.jpg)
Using the classifier
Pattern Classifier
neuron 1
neuron 2
neuron 3
neuron n
10
![Page 11: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/11.jpg)
Courtesy of MIT Press. Used with permission.Source: Meyers, E. M., and Gabriel Kreiman. "Tutorial on pattern classificationin cell recording." Visual Population Codes (2012): 517-538.
11
![Page 12: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/12.jpg)
Pseudo-populations
12
![Page 13: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/13.jpg)
Maximum Correlation Coefficient Classifier
…
Neuron 1
Neuron 2
Neuron 3
Neuron 4
Neuron 5
Neuron 6
13
![Page 14: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/14.jpg)
…
Neuron 1
Neuron 2
Neuron 3
Neuron 4
Neuron 5
Neuron 6
Test pointLearned prototypes
Maximum Correlation Coefficient Classifier
14
![Page 15: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/15.jpg)
Decoding can be viewed as assessing the information available to downstream neurons
© Source Unknown. All rights reserved. This content is excluded from our CreativeCommons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/.
15
© Source Unknown. All rights reserved. This
content is excluded from our Creative
Commons license. For more information,see https://ocw.mit.edu/help/faq-fair-use/.
![Page 16: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/16.jpg)
Neural content
16
![Page 17: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/17.jpg)
A simple experiment
Zhang, Meyers, Bichot, Serre, Poggio, and Desimone, PNAS, 2011
Seven objects:
132 neurons recorded from IT
17
![Page 18: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/18.jpg)
Applying decoding
Train
18
![Page 19: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/19.jpg)
Test
Applying decoding
100 ms bins, sample every 10 ms
19
![Page 20: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/20.jpg)
Basic decoding results
20
![Page 21: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/21.jpg)
Basic results are similar to other methods
21
![Page 22: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/22.jpg)
Confusion matrices
True classes
Pre
dic
ted
cla
sses
22
![Page 23: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/23.jpg)
Generally robust to the choice of classifier
Time (ms)
23
![Page 24: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/24.jpg)
Abstract/invariant representations
The ability to form abstract representations is essential for complex behavior
© Source Unknown. All rights reserved. This content isexcluded from our Creative Commons license. For moreinformation, see https://ocw.mit.edu/help/faq-fair-use/.
© Source Unknown. All rights reserved. This content is excluded from our CreativeCommons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/.
24
![Page 25: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/25.jpg)
Example: position invariance
Hung, Kreiman, Poggio, and DiCarlo, Science, 2005 25
![Page 26: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/26.jpg)
Stimulus set: 25 individuals, 8 head poses per individual
Meyers, Borzello, Freiwald, Tsao, J Neurosci, 2015
Face identification invariant to head pose
26
![Page 27: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/27.jpg)
Face identification invariant to head poseTrain
Left Profile
...
Test Pose Invariance
...
Test Same Pose
...
Courtesy of Society for Neuroscience. License CC BY.Source: Meyers, Ethan M., Mia Borzello, Winrich A. Freiwald, and Doris Tsao. "Intelligentinformation loss: the coding of facial identity, head pose, and non-face information in themacaque face patch system." Journal of Neuroscience 35, no. 18 (2015): 7069-7081. 27
![Page 28: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/28.jpg)
Face identification invariant to head pose
AnteriorPosterior
Courtesy of Society for Neuroscience. License CC BY.Source: Meyers, Ethan M., Mia Borzello, Winrich A. Freiwald, and Doris Tsao. "Intelligentinformation loss: the coding of facial identity, head pose, and non-face information in themacaque face patch system." Journal of Neuroscience 35, no. 18 (2015): 7069-7081.
28
![Page 29: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/29.jpg)
Learning abstract category information
Meyers, Freedman, Kreiman, Poggio, Miller, J Neurphys, 2008
© American Physiological Society. All rights reserved. This content is excluded from our CreativeCommons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/.Source: Meyers, Ethan M., David J. Freedman, Gabriel Kreiman, Earl K. Miller, and Tomaso Poggio."Dynamic population coding of category information in inferior temporal and prefrontal cortex."Journal of Neurophysiology 100, no. 3 (2008): 1407-1419.
29
![Page 30: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/30.jpg)
Summary of neural content
Decoding offers a way to clearly see information flow over time
For assessing basic information, decoding often yields similar results as other methods
Decoding allows one to assess whether information is contained in an abstract/invariant format, which is not possible with other methods
30
![Page 31: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/31.jpg)
Neural coding
31
![Page 32: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/32.jpg)
Motivating study
Meyers, Qi, Constantinidis, PNAS, 2012
© Source Unknown. All rights reserved. This content is excluded from our CreativeCommons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/.
32
![Page 33: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/33.jpg)
Monkeys were first trained to passively fixate
Time (ms)
Fixation 1st stimulus 2nd stimulus1st delay 2nd delay Reward
33
![Page 34: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/34.jpg)
Time (ms)
Fixation 1st stimulus 2nd stimulus1st delay 2nd delay Reward
Monkeys were first trained to passively fixate
34
![Page 35: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/35.jpg)
Time (ms)
Fixation 1st stimulus 2nd stimulus1st delay 2nd delayChoice targets/
saccade
Monkeys then engaged in a delayed-match-to-sample task (DMS task)
35
![Page 36: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/36.jpg)
Time (ms)
Fixation 1st stimulus 2nd stimulus1st delay 2nd delayChoice targets/
saccade
Monkeys then engaged in a delayed-match-to-sample task (DMS task)
36
![Page 37: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/37.jpg)
Time (ms)
Fixation 1st stimulus 2nd stimulus1st delay 2nd delay Choice targets/saccade
Decoding applied
Train
37
![Page 38: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/38.jpg)
Time (ms)
Fixation 1st stimulus 2nd stimulus1st delay 2nd delay Choice targets/saccade
Test
Decoding applied
500 ms bins, sample every 50 msDecoding is based on 750 neurons
38
![Page 39: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/39.jpg)
Decoding match/nonmatch information
Courtesy of Proceedings of the National Academy of Sciences. Used with permission.Source: Meyers, Ethan M., Xue-Lian Qi, and Christos Constantinidis. "Incorporation ofnew information into prefrontal cortical activity after learning working memory tasks."Proceedings of the National Academy of Sciences 109, no. 12 (2012): 4651-4656.
39
![Page 40: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/40.jpg)
Is the new information widely distributed?
Passive fixation DMS task
Courtesy of Proceedings of the National Academy of Sciences. Used with permission.Source: Meyers, Ethan M., Xue-Lian Qi, and Christos Constantinidis. "Incorporation ofnew information into prefrontal cortical activity after learning working memory tasks."Proceedings of the National Academy of Sciences 109, no. 12 (2012): 4651-4656.
40
![Page 41: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/41.jpg)
Compact/sparse coding of information
…
Neuron 1
Neuron 2
Neuron 3
Neuron 4
Neuron 5
Neuron 6
Training set
Labels M NN M
41
![Page 42: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/42.jpg)
Compact/sparse coding of information
Test set
……
Neuron 1
Neuron 2
Neuron 3
Neuron 4
Neuron 5
Neuron 6
Training set
Labels M NN M
42
![Page 43: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/43.jpg)
Compact/sparse coding of information
Test set
……
Neuron 1
Neuron 2
Neuron 3
Neuron 4
Neuron 5
Neuron 6
Training set
Labels M NN M
43
![Page 44: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/44.jpg)
Compact/sparse coding of information
Test set
……Neuron 1
Neuron 4
Neuron 6
Training set
Labels M NN M
44
![Page 45: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/45.jpg)
Using only the 8 most selective neurons Excluding the 128 most selective neurons
Is the new information widely distributed?
Courtesy of Proceedings of the National Academy of Sciences. Used with permission.Source: Meyers, Ethan M., Xue-Lian Qi, and Christos Constantinidis. "Incorporation ofnew information into prefrontal cortical activity after learning working memory tasks."Proceedings of the National Academy of Sciences 109, no. 12 (2012): 4651-4656.
45
![Page 46: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/46.jpg)
Selectivity value
Freq
uenc
y
Sel
ectiv
ity v
alue
Implications for analyzing data
46
![Page 47: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/47.jpg)
Is information contained in a dynamic population code?
Mazor and Laurent 2005; Meyers et al, 2008; King and Dehaene 2014
Time Courtesy of Elsevier, Inc., http://www.sciencedirect.com. Used with permission.Source: Mazor, Ofer, and Gilles Laurent. "Transient dynamics versus fixed pointsin odor representations by locust antennal lobe projection neurons." Neuron 48,no. 4 (2005): 661-673.
47
![Page 48: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/48.jpg)
Time (ms)
Fixation 1st stimulus 2nd stimulus1st delay 2nd delay Choice targets/saccade
Decoding applied
TestTrain
48
![Page 49: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/49.jpg)
Time (ms)
Fixation 1st stimulus 2nd stimulus1st delay 2nd delay Choice targets/saccade
Test
Decoding applied
Test
49
![Page 50: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/50.jpg)
Dynamic population coding
50
![Page 51: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/51.jpg)
Dynamic population coding
Passive fixation DMS task
Courtesy of Proceedings of the National Academy of Sciences. Used with permission.Source: Meyers, Ethan M., Xue-Lian Qi, and Christos Constantinidis. "Incorporation ofnew information into prefrontal cortical activity after learning working memory tasks."Proceedings of the National Academy of Sciences 109, no. 12 (2012): 4651-4656.
51
![Page 52: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/52.jpg)
The dynamics can be seen in individual neurons
Neuron 1 Neuron 2 Neuron 3
Courtesy of Proceedings of the National Academy of Sciences. Used with permission.Source: Meyers, Ethan M., Xue-Lian Qi, and Christos Constantinidis. "Incorporation ofnew information into prefrontal cortical activity after learning working memory tasks."Proceedings of the National Academy of Sciences 109, no. 12 (2012): 4651-4656.
52
![Page 53: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/53.jpg)
Poisson Naïve Bayes ClassifierTotal activity and pattern
Minimum Angle ClassifierPattern only
Total Population Activity ClassifierTotal activity only
wc are the classification weights for class c
x is the test point to be classifier
Decision Rule
Is information coded in high firing rates or patterns?
53
![Page 54: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/54.jpg)
Poisson Naïve Bayes – Total activity and pattern
Minimum Angle – Pattern only
Total Population Activity - Total activity only
Pose specific face identification
Is information coded in high firing rates or patterns?
Meyers, Borzello, Freiwald, Tsao, 201554
![Page 55: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/55.jpg)
Independent neuron code?
Data courtesy of the DiCarlo lab; see Ami Patel’s MEng thesis55
![Page 56: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/56.jpg)
Precision of the neural code (temporal coding)
(basic results – clearly room to explore this further) See Meyers et al, COSYNE, 200956
![Page 57: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/57.jpg)
Summary of neural coding
Decoding allows you examine many questions in neural coding including:
• Compact/sparse coding
• Dynamic population coding
• Independent neural code
• Temporal precision/temporal coding
57
![Page 58: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/58.jpg)
Decoding can be applied to other types of data
MEG/EEG (LFPs, ECog)• New tutorial on readout.info
fMRI• Princeton-mvpa-toolbox
• PyMVPA
• The decoding toolbox
Continuous decoding• nSTAT
Isik, Meyers, Liebo, Poggio, J. Neurophys, 2014
Figure removed due to copyright restrictions. Please see the video.Source: Figure 2, Isik, Leyla, Ethan M. Meyers, Joel Z. Leibo, and TomasoPoggio. "The dynamics of invariant object recognition in the human visualsystem." Journal of neurophysiology 111, no. 1 (2014): 91-102.
58
![Page 59: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/59.jpg)
Limitations of decoding
Hypothesis based – could be overlooking information that is not explicitly tested for
Just because information is present, doesn’t mean it’s used
Decoding focuses on the computational and algorithmic/representational levels, does not give a mechanistic explanation of the phenomena
Decoding methods can be computationally intensive, analyses can be slow to run
59
![Page 60: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/60.jpg)
The Neural Decoding Toolbox (NDT)
Makes it easy to do decoding in MATLAB:
1 binned_file = ‘Binned_data.mat’;
2 ds = basic_DS(binned_file, ‘stimulus_ID’, 20);
3 cl = max_correlation_coefficient_CL;
4 fps{1} = zscore_normalize_FP;
5 cv = standard_resample_CV(ds, cl, fps)
6 DECODING_RESULTS = cv.run_cv_decoding;
Open Science philosophy: open source for reproducible results• The code open source for reproducible results• Hope to encourage open science culture, so please share your data
www.readout.infoMeyers, Front Neuroinfo, 2013
60
![Page 61: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/61.jpg)
The Neural Decoding Toolbox Design
Toolbox design: 4 abstract classes
1. Datasource: creates training and test splits• E.g., can examine the effects from different binning schemes
2. Preprocessors: learn parameters from training data apply them to the training and test data
• E.g., can examine sparse/compact coding
3. Classifiers: learn from training data and make predictions on test data• E.g., can examine whether information is in high firing rates or patterns
4. Cross-validators: run the training/test cross-validation cycle
61
![Page 62: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/62.jpg)
Getting started with your own data
You can use the NDT on your own data by putting your data into ‘raster format’
Figure removed due to copyright restrictions. Please see the video or Figure 3 from Meyers,Ethan M. "The neural decoding toolbox." Frontiers in neuroinformatics 7 (2013).
62
![Page 63: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/63.jpg)
Funding: The Center for Brains, Minds and Machines, NSF STC award CCF-1231216
Acknowledgements:
Narcisse Bichot, Mia Borzello, Christos Constantinidis, Jennie Deutsch, Jim DiCarlo, Robert Desimone, David Freedman, Winrich Freiwald, Leyla Isik, Gabriel Kreiman, Andy Leung, Joel Liebo, Earl Miller, Ami Patel, Tomaso Poggio, Xue-Lian Qi, Doris Tsao, Ying Zhang
www.readout.info
Questions?
63
![Page 64: Understanding Neural Content and Coding Using Population … · 2020-04-23 · The Neural Decoding Toolbox Design Toolbox design: 4 abstract classes 1. Datasource: creates training](https://reader033.vdocument.in/reader033/viewer/2022050323/5f7c995a580ff060216c9bff/html5/thumbnails/64.jpg)
MIT OpenCourseWarehttps://ocw.mit.edu
Resource: Brains, Minds and Machines Summer CourseTomaso Poggio and Gabriel Kreiman
The following may not correspond to a particular course on MIT OpenCourseWare, but has beenprovided by the author as an individual learning resource.
For information about citing these materials or our Terms of Use, visit: https://ocw.mit.edu/terms.