jan 16: the visual systemmatilde/lecture3visualcortex.pdf · santiago ramon y cajal (1 may 1852...
TRANSCRIPT
![Page 1: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/1.jpg)
Geometry of Neuroscience
Matilde Marcolli & Doris Tsao
Jan 16: The Visual System
![Page 2: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/2.jpg)
References for this lecture
• 1977 Hubel, D. H., Wiesel, T. N., Ferrier lecture
• 2010 Freiwald, W., Tsao, DY. „Functional compartmentalization and viewpoint generalization in the macaque face patch system“
• 2013 Nature Mante, Susillo, Shenoy, Newsome, „Context-dependent computation by recurrent dynamics in prefrontal cortex.”
![Page 3: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/3.jpg)
Santiago Ramon y Cajal
(1 May 1852 – 18 October 1934)
Neuron doctrine: The brain is made up of discrete individual cells.
Vision is explicable in terms of the firing patterns that emerge from the interactions
among many individual neurons.
• Characterizing functional transformations along anatomically-connected pathways
• Characterizing population dynamics among large groups of neurons
![Page 4: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/4.jpg)
Retina
![Page 5: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/5.jpg)
![Page 6: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/6.jpg)
David Hubel, “Eye, Brain, Vision”
Stephen Kuffler
Visual processing in the retina
![Page 7: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/7.jpg)
LGN
![Page 8: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/8.jpg)
![Page 9: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/9.jpg)
![Page 10: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/10.jpg)
![Page 11: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/11.jpg)
1
2
3
4
5
6
1-2: “Magno” fast and transient, large RF
2-6: “Parvo” slow and sustained, small RF
K: innervate extrastriate cortex
K
K
K
K
K
K
![Page 12: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/12.jpg)
V1
“Primary Visual cortex”
“Striate cortex”
“Area 17”
![Page 13: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/13.jpg)
Hubel & Wiesel, 1959 (HMS)
![Page 14: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/14.jpg)
![Page 15: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/15.jpg)
![Page 16: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/16.jpg)
![Page 17: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/17.jpg)
![Page 18: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/18.jpg)
![Page 19: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/19.jpg)
![Page 20: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/20.jpg)
![Page 21: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/21.jpg)
Hubel & Wiesel, 1962
![Page 22: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/22.jpg)
![Page 23: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/23.jpg)
![Page 24: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/24.jpg)
Functional Architecture
![Page 25: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/25.jpg)
Monkey
Human
![Page 26: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/26.jpg)
Critical Period
![Page 27: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/27.jpg)
![Page 28: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/28.jpg)
![Page 29: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/29.jpg)
Livingstone and Conway, 2003
RF of V1 simple cellSparse noise stimulus
![Page 30: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/30.jpg)
Ohki et al. Nature 2006
![Page 31: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/31.jpg)
Features computed by V1
• Orientation
• Color
• Motion
• Binocular Disparity
![Page 32: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/32.jpg)
What happens to visual information after V1?
![Page 33: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/33.jpg)
![Page 34: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/34.jpg)
![Page 35: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/35.jpg)
What happens to visual information after V2?
![Page 36: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/36.jpg)
![Page 37: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/37.jpg)
![Page 38: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/38.jpg)
Criteria for Cortical Area
• Topographic map
• Unique connections
• Unique cytoarchitecture
• Unique function
![Page 39: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/39.jpg)
Lewis and Van Essen 2000
![Page 40: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/40.jpg)
![Page 41: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/41.jpg)
Felleman and Van Essen 1991
![Page 42: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/42.jpg)
![Page 43: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/43.jpg)
Border ownership cell
V1 Simple cell(Hubel & Wiesel)
V2 Border-ownershipcell
(von der Heydt)
![Page 44: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/44.jpg)
![Page 45: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/45.jpg)
What & Where Pathways
“Vision is knowing what is where by
looking” –David Marr
![Page 46: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/46.jpg)
The Ungerleider & Mishkin (1982) Experiment
Task 1:
Object discrimination
study an object
select the familiar object
(reward)
parietal lesions
impair LANDMARK TASK
temporal lesions
impair OBJECT TASK
Task 2:
Landmark discrimination• select foodwell closest to
the TOWER
![Page 47: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/47.jpg)
The macaque face patch system
![Page 48: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/48.jpg)
V1
V2V3V4
Inferotemporal
Cortex (IT)
Object identification
Pathway for object representation
Image segmentation
“Discouraged by my inability to understand the frontal lobe…I decided to turn my attention to the cortex on the inferior convexity of the temporal lobe” –Charles Gross, History of Neuroscience autobiography
Charles Gross
![Page 49: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/49.jpg)
Kanwisher, McDermott, & Chun, J. Neurosci, 1997
Fusiform face area
?
![Page 50: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/50.jpg)
Face-selective regions exist in
monkeys
![Page 51: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/51.jpg)
![Page 52: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/52.jpg)
Localization of face patches
Faces vs. objects
occipital
temporal
frontal
Tsao et al., Nature Neuroscience 2002
posterior
PL
middle
ML
MF
anterior
AL
AF
AM
![Page 53: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/53.jpg)
FMRI response time course from
middle face patch ML
![Page 54: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/54.jpg)
What is the selectivity of single
neurons in each patch?
![Page 55: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/55.jpg)
What is the selectivity of single
neurons in each patch?
![Page 56: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/56.jpg)
What is the selectivity of single
neurons in each patch?
![Page 58: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/58.jpg)
Tsao et al., Science, 2006
![Page 59: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/59.jpg)
Face cells are clustered in face patches
PL
ML
MF
AL
AF
AM
Tsao et al., Science 2006
Freiwald and Tsao, Science 2010
Cell
num
ber
![Page 60: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/60.jpg)
Face patches are strongly and
specifically connected to each other
PL
ML
MF
AL
AF
AM
Moeller et al. Science 2008
Grimaldi et al., Neuron, in press
![Page 61: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/61.jpg)
Probing connectivity of face
patches: Microstimulation combined
with fMRI
Sebastian MöllerMoeller et al., Science, 2008
![Page 62: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/62.jpg)
x
Stimulation site
posteriorventra
l
Stimulation site on the flatmap
![Page 63: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/63.jpg)
Microstimulation >> Blank
x
Stimulation site
posteriorventra
l
![Page 64: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/64.jpg)
x
Stimulation site
posteriorventra
l
ML projection map overlaps with other
face patches
![Page 65: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/65.jpg)
Connections of ML
![Page 66: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/66.jpg)
Connections within IT cortex largely
confined to other face patchesposterior
ventra
l
Injection site in AL,
retrograde tracer (Fast Blue)
xAL
ML
MF
AF
AM
PL
P v
alu
e
10-10
10-3
66Grimaldi et al., Neuron 2016
![Page 67: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/67.jpg)
Connections within IT cortex largely
confined to other face patches
Ce
lls / m
m3
1
103
posteriorventra
l
Injection site in AL,
retrograde tracer (Fast Blue)
xAL
ML
MF
AF
AM
PL
67Grimaldi et al., Neuron 2016
![Page 68: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/68.jpg)
Connections within IT cortex largely
confined to other face patches
Ce
lls / m
m3
1
103
P v
alu
e
10-10
10-3
Injection site in AL,
retrograde tracer (Fast Blue)
posteriorventra
l
xAL
ML
MF
AF
AM
PL
68Grimaldi et al., Neuron 2016
![Page 69: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/69.jpg)
Face patch system
![Page 70: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/70.jpg)
Experimental obstacles
Huge cortical territory
Huge parameter space
![Page 71: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/71.jpg)
Face patches
Faces
Experimental obstacles
![Page 72: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/72.jpg)
What are functional properties of
cells in each patch?
![Page 73: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/73.jpg)
NO HAIR HAIR
Cells are selective for the presence
and shape of subsets of face
features
1 128
Cell 1
Cell 2
No
rm. r
esp
.
Freiwald et al., NN 2009
![Page 74: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/74.jpg)
Cell 1
Cell 2
NO IRISES IRISES NO IRISES IRISES
No
rm. r
esp
.
1 128
Freiwald et al., NN 2009
Cells are selective for the presence
and appearance of subsets of face
features
![Page 75: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/75.jpg)
Increasing view invariance from ML/MF to AL to AM
ALAM
ML/MF
![Page 76: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/76.jpg)
Anterior medial face patch (AM) neurons
represents face-view invariant identity
Freiwald and Tsao, 20100.1s
![Page 77: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/77.jpg)
Middle face patch (MF/ML) neurons were tuned to head orientation
0.1sFreiwald and Tsao, 2010
![Page 78: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/78.jpg)
0.1s
Middle face patch (MF/ML) neurons were tuned to head orientation
Freiwald and Tsao, 2010
![Page 79: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/79.jpg)
Santiago Ramon y Cajal
(1 May 1852 – 18 October 1934)
Neuron doctrine: The brain is made up of discrete individual cells.
Vision is explicable in terms of the firing patterns that emerge from the interactions
among many individual neurons.
• Characterizing functional transformations along anatomically-connected pathways
• Characterizing population dynamics among large groups of neurons
![Page 80: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/80.jpg)
Mante and Newsome 2013
Understanding the mechanism for flexible behavior
![Page 81: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/81.jpg)
Step 1: Compute top 12 PCs of population data
D = matrix of eigenvectors
Step 2: Demixing (goal: plot neural state in meaningful coordinates)
Step 3: Denoise the regression vectors (project onto 12 PCs)
![Page 82: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/82.jpg)
Regression coefficients for individual neurons
![Page 83: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/83.jpg)
Step 4: Orthogonalize regression vectors (Gramm Schmidt)
b1
b2Q2
Q1
![Page 84: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/84.jpg)
![Page 85: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/85.jpg)
…So far, not THAT surprising (a fancy way of showing that (1) the monkey can
do the task and the choice signal is also represented in PFC, and (2) both color and
motion information are represented regardless of context,)
What is the underlying mechanism?
![Page 86: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/86.jpg)
Recurrent Neural Network
• x represents activation of neuron, r represents firing rate, J represents recurrent connections, u represents input, c represents offsets, rho represents noise
• J, b, c are modified through training
![Page 87: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/87.jpg)
Search for fixed points (where x = 0).
![Page 88: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/88.jpg)
Red crosses: fixed points (same in both contexts)
![Page 89: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/89.jpg)
![Page 90: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/90.jpg)
• Context-depending integration is explained by different neural dynamics in the two situations!
![Page 91: Jan 16: The Visual Systemmatilde/Lecture3VisualCortex.pdf · Santiago Ramon y Cajal (1 May 1852 –18 October 1934) Neuron doctrine: The brain is made up of discrete individual cells](https://reader034.vdocument.in/reader034/viewer/2022042316/5f051be87e708231d4114ef1/html5/thumbnails/91.jpg)
“We conclude that the observed complexity and functional roles of single neurons are readily understood in the framework of a dynamical process unfolding at the level of a population .”