Principles of neuronal coding
Grand Challenges WorkshopInstitute of Medicine Neuroscience Forum
25 June 2008
William BialekJoseph Henry Laboratories of Physics
Lewis-Sigler Institute for Integrative GenomicsPrinceton Center for Theoretical Science
Princeton University
http://www.princeton.edu/~wbialek/wbialek.html
The enormous variety of our sensory experiences,
abstract thoughts,
and actions
all are represented in our brain by sequences of identical electrical pulses,
called action potentials or “spikes.”
This is the problem of neural coding.
What are the elementary symbols in the code, and what do they represent?
Classical ideas about “rate,” evolution of ideas about “timing.”Classical ideas of feature selectivity, modern version as “dimensionality reduction.”
How are features extracted (invariantly, robustly)? Coding meets computation.*Are populations of cells more than the sum of their parts?
Why does the code have the structure that it does, as opposed to any other possible structure?
*Has the brain chosen codes that are, in some precise mathematical sense, optimal?Could this be a principle from which aspects of neural function can be predicted?
Are there principles that could unify our understanding of information flow in biological systems?
There is no way to give an exhaustive account of these efforts here, so I’ll choose some examples*(shamelessly drawn from work done by my colleagues and myself), and then point to lessons for the future.
Note: I am going to hide almost all of the math, but I give references.
Are populations of cells more than the sum of their parts?
cell #1
cell #2
cell #3
cell #4
cell #5
cell #6
cell #7
cell #8
cell #9
cell #10
states or patterns
a moment ago
no spike
spike no spike
no spike
no spike
spike no spike
spike no spike
spike 100010101
right now spikeno
spikeno
spike spikeno
spikeno
spikeno
spikeno
spike spikeno
spike 1001000010
the next moment
no spike
no spike spike
no spike spike
no spike spike
no spike
no spike
no spike 10101000
states of the network = binary patterns of spiking and silence
A common observation in many brain areas (including cortex) is that when we look at pairs of
cells, the 1s and 0s are only weakly correlated.
(e.g., pairs of ganglion cells in the vertebrate retina as it responds to natural movies)
Does this mean we can ignore correlations?Or could something dramatic be hiding in these
rather mundane observations?Analogous problems in statistical physics lead us
to be suspicious of widespread correlations ...
Although pairwise correlations are weak, there are collective effects in larger populations ...
What can pair correlations tell us about the distribution of states in the whole network?
In general, nothing. But there is a unique “simplest” or “least structured” model
consistent with the measured correlations(technically, the maximum entropy model).
Weak pairwise correlations imply strongly correlated network states in a neural population. E Schneidman, MJ Berry II, R Segev & W Bialek,
Nature 440, 1007-1012 (2006).
Ising models for networks of real neurons. G Tkacik, E Schneidman, MJ Berry II & W Bialek, arXiv.org/q-bio.NC/0611072 (2006).
(an example of N=40 cells)
independent neurons
simplestpairwisemodel
(an example of N=10 cells)
These minimal models are exactly the Ising model in statistical mechanics, and
also the Hopfield model of neural networks. Here these “physics-style” models emerge directly from the data!
Multiple attractors, phase transitions: more than the sum of its parts!
Optimal information transmission?In an efficient neural code, the input/output relation of neurons is matched to the distribution of inputs.Intuitively, we want the cell to be driven through its full dynamic range. This can be formalized by asking for a code that maximizes information transmission.
A simple coding procedure enhances a neuron's information capacity. SB Laughlin, Z Naturforsch 36c, 910-912 (1981).
10
1
0.1
0.01
0.001
1000-100stimulus
angular velocity (deg/s)
relativespike rate
distribution of input stimuli has large dynamic range
distribution of input stimuli has small dynamic range
Adaptive rescaling optimizes information transmission. N Brenner, W Bialek & R de Ruyter van Steveninck, Neuron 26, 695-702 (2000).
Efficiency and ambiguity in an adaptive neural code. AL Fairhall, GD Lewen, W Bialek & RR de Ruyter van Steveninck, Nature 412, 787-792 (2001).
Unifying principles?
Early development of the Drosophila embryomother puts mRNA
for Bicoid at (future)
head of the embryo
Bicoid activates expression
of Hunchback
intensity of bcd stain~ bcd concentration
inte
nsit
y of
hb
stai
n
Here we measure the input/output relations and noise ... Can we predict the distributions? Same principle as for neurons!
Information flow and optimization in transcriptional regulation.G Tkacik, CG Callan Jr & W Bialek, arXiv:0705.0313 [q-bio.MN] (2007).
Probing the limits to positional information. T Gregor, DW Tank, EF Wieschaus & W
Bialek, Cell 130, 153-164 (2007)
(no free parameters!)
Hunchback expression level
Prob
abili
ty d
ensi
ty
Theory (not just models) as a full partner with experiment
Maximum entropy for pairwise + recording many cells simultaneously leads to reconcile weak correlations with strong collective behavior;connect experiments directly to large body of neural network models;strength of correlations seems “just enough” for critical behavior.
Information maximization + measuring distributions (not means) leads to new forms of adaptation in the neural code;new questions about mechanisms;common principles across neural and genetic signaling.
Crucially, theory doesn’t just come after exp’t, it can and should come before: pointing to the next interesting experiment, and
motivating larger scale, more quantitative measurements.
A grand challenge: principled (not parameterized!) theories for real networks.