systems/circuits...

Systems/Circuits

Benefits of Pathway Splitting in Sensory Coding

Julijana Gjorgjieva,1 Haim Sompolinsky,1,2 and Markus Meister3

1Center for Brain Science, Harvard University, Cambridge, Massachusetts 02138, 2Edmond and Lily Safra Center for Brain Sciences, Hebrew University,Jerusalem, 91904, Israel, and 3Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, California 91125

In many sensory systems, the neural signal splits into multiple parallel pathways. For example, in the mammalian retina, �20 types ofretinal ganglion cells transmit information about the visual scene to the brain. The purpose of this profuse and early pathway splittingremains unknown. We examine a common instance of splitting into ON and OFF neurons excited by increments and decrements of lightintensity in the visual scene, respectively. We test the hypothesis that pathway splitting enables more efficient encoding of sensorystimuli. Specifically, we compare a model system with an ON and an OFF neuron to one with two ON neurons. Surprisingly, the optimalON–OFF system transmits the same information as the optimal ON–ON system, if one constrains the maximal firing rate of the neurons.However, the ON–OFF system uses fewer spikes on average to transmit this information. This superiority of the ON–OFF system is alsoobserved when the two systems are optimized while constraining their mean firing rate. The efficiency gain for the ON–OFF split iscomparable with that derived from decorrelation, a well known processing strategy of early sensory systems. The gain can be orders ofmagnitude larger when the ecologically important stimuli are rare but large events of either polarity. The ON–OFF system also providesa better code for extracting information by a linear downstream decoder. The results suggest that the evolution of ON–OFF diversificationin sensory systems may be driven by the benefits of lowering average metabolic cost, especially in a world in which the relevant stimuli aresparse.

Key words: efficient coding; ON–OFF; optimality; parallel pathways; retina; sensory processing

IntroductionRetinal ganglion cells are the output neurons of the retina andconvey all visual information from the eye to the brain. Just twosynapses separate these neurons from the photoreceptors, and yetthe population of ganglion cells already splits into �20 differenttypes. Each type covers the entire visual field and sends a separateneural image to higher brain areas for additional processing(Wassle, 2004; Masland, 2012). What factors drove the evolutionof such an early and elaborate pathway split remains mysterious.Here we test some explanations based on the principles of effi-cient coding.

We focus on a specific prominent instance of sensory splitting:the emergence of ON and OFF pathways. Among retinal ganglioncells, several of the well known cell types come in matched pairsthat share similar spatiotemporal response properties, except thatone type is excited by light increments (ON) and the other type bylight decrements (OFF) (Wassle, 2004; Wassle et al., 1981a,b). Asimilar ON–OFF dichotomy is observed in other sensory modal-

ities, including insect vision (Joesch et al., 2010), thermosensa-tion (Gallio et al., 2011), chemosensation (Chalasani et al., 2007),audition (Scholl et al., 2010), and electrolocation in electric fish(Bennett, 1971). The broad prevalence of this organizing princi-ple suggests that it provides an evolutionary fitness benefit of avery general nature. In the present work, we treat the problem inthe context of vision but seek an explanation that generalizesbeyond this specific sensory system while ignoring details that areoften specific to one neural system or another.

Previous investigations of this question suggested that theON–OFF split evolved for the rapid and metabolically efficientsignaling of opposite changes in light intensity, because both in-crements and decrements are prominent in natural scenes (Schil-ler et al., 1986; Schiller, 1992; Westheimer, 2007). Here weformalize such an argument by testing different notions of effi-ciency, by examining different models of the neural response anddifferent assumptions about the relevant stimuli to be encoded.Throughout, we compare side by side the performance of a modelsystem consisting of an ON and an OFF cell with that of a systemwith two ON cells. The results support the notion that ON–OFFsplitting enables more efficient neural coding; the benefits forON–OFF coding already appear in the simplest possible codingmodel and become more pronounced as we make the modelmore realistic. If the dominant constraint on neural signalingcomes from a limit on the maximal firing rate of cells, then path-way splitting into ON and OFF channels does not deliver greaterinformation, but it uses fewer spikes than the alternate codingscheme with two ON cells. As a result, if the time-average firingrate is constrained, ON–OFF coding offers an improvement ininformation of �15%; this holds under a broad range of stimuli

Received March 14, 2014; revised July 3, 2014; accepted July 22, 2014.Author contributions: J.G., H.S., and M.M. designed research; J.G., H.S., and M.M. performed research; J.G.

analyzed data; J.G., H.S., and M.M. wrote the paper.This work was supported by a grant from the National Institutes of Health (J.G., H.S. and M.M.) and grants from

the Gatsby Charitable Foundation and the Swartz Foundation (H.S.). J.G. thanks Xaq Pitkow for providing help withthe sigmoidal nonlinearities from retinal data.

The authors declare no competing financial interests.Correspondence should be addressed to either of the following: Julijana Gjorgjieva, Center for Brain

Science, Harvard University, Cambridge, MA 02138, E-mail: [email protected]; or Markus Meister,Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA 91125,E-mail: [email protected].

DOI:10.1523/JNEUROSCI.1032-14.2014Copyright © 2014 the authors 0270-6474/14/3412127-18$15.00/0

The Journal of Neuroscience, September 3, 2014 • 34(36):12127–12144 • 12127

and response models, including linear decoding by downstreamneurons, when the benefits become more pronounced. The larg-est benefit for ON–OFF splitting emerges if ecologically impor-tant stimuli consist of rare events with large ON or OFF signalsembedded in a sea of events with low variance. Our results showthat ON–OFF coding improves information transmission at lowaverage metabolic costs under a broad set of conditions, with themost pronounced benefits appearing for a sparse stimulusensemble.

Materials and MethodsMutual information with binary nonlinearities and Poisson outputnoise. We first studied how static stimuli are encoded by ON–OFF andON–ON systems. Each ganglion cell fires action potentials in re-sponse to a common scalar stimulus s drawn from a stimulus distri-bution Ps(s). We model each ganglion cell with a firing rate � � g(s),a binary function with threshold �, and a maximal value of �max (Fig.1). For instance, � � �max �(s � �) for ON type cells and � � �max

�(� � s) for OFF type cells, where �(x) is the Heaviside function thatis 0 if x � 0 and 1 if x � 0. In a system with two cells, each cell can havea different threshold, �1 and �2.

The observable variable in our model is the number of spikes k in atime period T. We assume Poisson output noise; thus, the probability ofobserving k spikes given a firing rate � can be written as

P�k�� Tk

k!e��T. (1)

The mutual information between the stimulus s and the observed spikecount k for a single cell can be obtained from the equation

I�s;k� � �k�0

� �s

Ps�s� P�k�s�log2

P�k�s�Pk�k�

ds, (2)

where Pk(k) is the output distribution of the observed spike count, whichcan be calculated from Pk�k� � �sPs�s�P�k�s� ds. In the absence of inputnoise, knowing the stimulus s unambiguously determines the responsefiring rate �; for instance, for an ON cell, if s � �, � � 0, and if s � �, � ��max. Therefore, in Equation 2, we can replace P(k�s) with P(k��) fromEquation 1.

For a binary response function with two firing rate levels, 0 and �max,we can lump together all states with nonzero spike counts into a singlestate, which we denote as 1. Correspondingly, the state with zero spikes is0. The reason for this simplification is that, if the ON (OFF) neuron emitsat least one spike, then the stimulus must necessarily be above (below) itsthreshold, and the number of spikes does not yield any additional infor-

mation about the stimulus. Hence, we can evaluate the mutual informa-tion between stimulus and spiking response using the followingexpressions for the spike count probabilities:

P�0�� 0� � 1, P�1�� 0� � 0,P�0�� max� � 1 � r, P�1�� max� � r, (3)

where r � 1 � e �Nmax and Nmax � �max T is the expected spike count ina time period T given the maximal firing rate �max of the cell.

The mutual information between the stimulus and the spiking re-sponse of the two cells can be written as

I�s;k1,k2� � �k1�0

� �k2�0

� �s

Ps�s� P�k1,k2�s�log2

P�k1,k2�s�

�s�

P�s��P�k1,k2�s�� ds�

ds.

(4)

When we compute the mutual information for two cells with onlyoutput noise, we assume that stimulus encoding by the two neurons isstatistically independent conditional on the stimulus s; Pitkow andMeister (2012) found that noise correlations arising from overlappingreceptive fields in salamander ganglion cells were very small. There-fore, P(k1,k2�s) � P(k1�s)P(k2�s), and as above, in the absence of inputnoise, we can replace P(kj�s) with P(kj��j) for j � 1, 2 from Equation 3.

With a binary step-shaped rate function, the mutual information de-pends on the stimulus distribution only through the probability of hav-

ing a firing rate of �max, which for an ON cell is PON��

�ds� Ps�s��

and for an OFF cell is POFF��

�

ds� Ps�s��. We can thus replace � by

the corresponding cumulative probability, which essentially maps thestimulus distribution into a uniform distribution from 0 to 1. The result-ing states for the ON–OFF and ON–ON systems are shown schematicallyin Figure 2. In this work, we determine the thresholds by maximizing themutual information as a function of these variables, and because thestimulus dependence enters only through these values, the maximal mu-tual information is independent of the stimulus distribution, providedthat the stimulus cumulative distribution is continuous.

We first consider the case where the value of Nmax is fixed. Maximizingthe mutual information with respect to the thresholds, we obtain a sym-metric solution for the ON–OFF system, with �1 �2 and �2 � 1 � �1,where �1 is the threshold for the ON cell and �2 for the OFF cell. In fact,for any Nmax, the optimal solution is for the ON and OFF cells to havecells with non-overlapping response ranges, i.e., �1 �2 (Fig. 3). Thus,for the ON–OFF system, the mutual information only depends on one ofthe thresholds:

ON

OFF

stimulus

observedspike count

Figure 1. Modeling framework. A stimulus s is encoded by a system of two cell types: one ON and OFF (as shown) or two ON cells. Each cell has a binary response nonlinearity with thresholds �1

or �2. During each coding window of duration T, the stimulus is constant and the spike count is drawn from a Poisson distribution. A measure of coding efficiency here is the mutual informationbetween the stimulus s and the spike count responses of the two cells k1 and k2, I(s; k1, k2).

12128 • J. Neurosci., September 3, 2014 • 34(36):12127–12144 Gjorgjieva et al. • Benefits of Pathway Splitting in Sensory Coding

ION�OFF � �1 � 2r�1 � �1�log21 � 2r�1 � �1� �

2�1 � �1��1 � r�log21 � r � 2�1 � �1�r log21 � �1. (5)

This quantity is maximized for thresholds �1 for the ON cell and �2 � 1 ��1 for the OFF cell:

�1 � 1 �1

2r � �1 � r��r�1�/r and �2 �1

2r � �1 � r��r�1�/r.

(6)

Similarly, we obtain the same maximized mutual information transmit-ted by the ON–ON system but achieved with thresholds:

�1 � 1 �1

2r � �1 � r��r�1�/r and �2 � 1 �1 � r

2r � �1 � r��r�1�/r.

(7)

For the system with two identical ON cells (2ON) with threshold �, themutual information is equal to

I2ON;ident � �� 1 � ��1 � r�2 log2� � �1 � ��1 � r�2.

(8)

The information is maximized for an optimal threshold equal to

� ident �1 � �1 � r�

2

r�2�r�

1 � �1 � r�2�1�r�2

r�2�r� r�2 � r�. (9)

The maximal mutual information for the system with a non-monotonicnonlinearity for the so-called U-cell, � � �max �(s � �1)�(�2 � s), wascomputed using Equation 4 by numerically iterating through a grid ofthresholds �1 and �2 for the U-cell and �3 for the ON cell.

Instead of the maximal firing rate of the cells, �max, we can alternativelyconstrain the overall mean firing rate of each system, �mean, or the totalmean spike count of each system, Nmean � �mean T. In addition to thethresholds, here we also need to optimize the allocation of Nmean to eachof the cells in the system. As in the case of constraining Nmax, we find thatthe optimal solution for the ON–OFF system is symmetric, such that�2 � 1 � �1, and the two cells have response ranges that do not overlapfor the entire range of Nmean, i.e., �1 �2. This implies that the twothresholds divide stimulus space into three regions: one region codedonly by the ON cell, another coded only by the OFF cell, and a region thatis not coded by either cell. The mean spike count for each cell is one-halfof the total mean spike count. For the ON–ON system, the fraction, f, ofthe total mean spike count to be assigned to the cell with the lowerthreshold was generally greater than 1/2 (Fig. 3C, inset).

To compute the mutual information from Equation 2, we expressedNmax as a function of the total mean spike count Nmean. For an ON cellwith a larger threshold �1, we can write Nmax � (1 � f )Nmean/(1 � �1),

and for an ON cell with a smaller threshold �2,we can write Nmax � fNmean/(1 � �2), where0 � f � 1.

Using sigmoidal nonlinearities and sub-Poisson noise from retinal data. We examinedsigmoidal nonlinearities fitted from recordedretinal ganglion cells (Pitkow and Meister,2012) of the form (see Fig. 6)

gON�s� � �max

1

1 � e��s��ON� (10)

to describe the firing rate of an ON cell withthreshold �ON as a function of the stimulus s,and

gOFF�s� � �max

1

1 � e�s��OFF� (11)

to describe the firing rate of an OFF cell withthreshold �OFF. The sigmoids have gain and a maximal firing rate of�max. We studied nonlinearities with the same gain for both ON andOFF cells.

Here we assumed an output noise distribution that matches the ob-served sub-Poisson noise in spike counts of salamander retinal ganglioncells. For a given mean spike count c, the measured spike count distribu-tion P(k�c) had a width that stayed constant with c after an initial Poisson-like growth (Pitkow and Meister, 2012, their Supplemental Fig. 3). Thedistribution is well described by the heuristic formula

P�k�c� � exp� ��k � k0�c��

2

22 �, k � 0,1,2, . . . (12)

where

k0�c� � a log��1 � ec/a�, (13)

with a � 0.5 and � 0.75 (Pitkow and Meister, 2012). We computed themutual information by numerically evaluating the integrals arising inEquation 4, where the spike counts k1 and k2 of the two cells weresummed up to some cutoff.

Figure 6 shows the results for two nonlinearities that were fit usingexperimental data from salamander (Pitkow and Meister, 2012) and ma-caque (Uzzell and Chichilnisky, 2004) ganglion cells. The average gainfor salamander ganglion cells is � 5.8 (expressed in inverse SD of thestimulus) and for macaque ganglion cells � 2.3. Pitkow and Meister(2012) cite a median peak firing rate of 48 Hz for salamander ganglioncells, which using a coding time window of 50 ms translates into a max-imum expected spike count Nmax � 2.4. The median peak firing rate formacaque ganglion cells is 220 Hz, but these have a shorter coding timewindow of 10 ms, producing Nmax � 2.2. The measured mean firing ratesfor each species are 1.1 Hz for salamander and 30 Hz for macaque (Uzzelland Chichilnisky, 2004; Pitkow and Meister, 2012).

Input noise. In addition to output Poisson noise, in a subset of ourresults, we also studied the effect of common input Gaussian noise withvariance 2 (see Fig. 7). Then the firing rate for an ON neuron can bewritten as

�1�s� � �max��s � z � �1� (14)

and for an OFF neuron

�2�s� � �max��2 � s � z� (15)

where z � N(0, 2). The information was computed using Equation 4with the conditional probability distribution of the two cells given by

Figure 2. Output states in the binary neuron model. An arbitrary stimulus distribution P(s) can be mapped into a uniformdistribution so that the optimal thresholds can be depicted in terms of the fraction of stimuli below threshold, ranging from 0 to 1.A, Possible output states for the two neurons in the ON–OFF system, depending on the stimulus value. 0 denotes silence, 1 denotesone or more spikes in the coding window. B, Same as A but for two ON cells.

Gjorgjieva et al. • Benefits of Pathway Splitting in Sensory Coding J. Neurosci., September 3, 2014 • 34(36):12127–12144 • 12129

P�k1,k2�s� � ��1��0,�max�

��2��0,�max�

P�k1��1�P�k2��2�P��1,�2�s� (16)

because now the two cells are correlated conditioned on the stimuluss. In this expression, P(kj��j) can be computed from Equation 3 forj � 1, 2. Furthermore, P(�1,�2�s) can be computed by integrating thefiring rates of the two cells over the Gaussian distribution with variance 2.

In the text, we studied the effects of stimulus sparseness by using twodistributions: a Gaussian and a Laplace stimulus distribution with amean of 0 and SD of 1. The Laplace distribution is consistent with fitsproduced by filtering natural images with difference-of-Gaussian linearfilters, corresponding to center-surround receptive fields (Field, 1994;Bell and Sejnowski, 1997).

To implement the sigmoidal nonlinearities with Poisson output noisein Figure 5, we used the same input noise model but assumed that theinput noise for each cell was independent, i.e., we added a noise term z1 tothe stimulus for one cell and z2 to the stimulus for the other cell, with z1,z2 � N(0, 2). Now, for the mutual information in Equation 4, theconditional probability distribution of the two cells is (because the firingrates are independent conditioned on s)

P�k1,k2�s� � P�k1�s� P�k2�s� �

� ��1��0,�max�

P�k1��1�P��1�s�� 2��0,�max�

P�k2��2�P��2�s� (17)

where P(kj��j) can be computed from Equation 3 for j � 1, 2, andP(�j�s) was computed by integrating the firing rate of the cell over theGaussian distribution with variance 2; for instance, for an ON cellwith threshold �1,

P��1 � �max�s� � H��1 � s

, (18)

and P(�1 � 0�s) � 1 � P(�1 � �max�s), where H is the complementaryerror function

H�� 1

2erfc� �

2 ��

�

� 1

2�e�z2/ 2dz. (19)

Now the effective sigmoidal nonlinearity of the cells is given by Equation18. The gain of these sigmoids can be mapped to the gain of the sigmoidalnonlinearities fitted from the retinal data (Eq. 10) by � 1/�2��.

Linear estimator: static. As an alternative measure of coding efficiencyto the mutual information, we minimized the mean square error (MSE)between the static stimulus s and a reconstructed version of the stimulussest (see Fig. 8):

E � ��sest � s�2�, (20)

where the angle brackets denote average over stimulus values. The stim-ulus estimate was obtained from the observed ganglion cell responses,x1 � �x1� and x2 � �x2� (normalized to have zero mean), using appropriatedecoding weights, w1 and w2:

sest � �i�1,2

wi�xi � �xi��. (21)

The MSE can be written as:

E � �s2� � wTCw � 2wTU, (22)

where C denotes the matrix of pairwise correlations between the re-sponses of the cells, and U denotes the correlation between response andstimulus, i.e.,

Cij � �xixj� � �xi��xj�, and Ui � �s xi� � �s��xi�. (23)

Differentiating the MSE in Equation 22 with respect to the weights w, weobtained the expressions for the optimal weights:

w � C�1U, (24)

so that the error can also be written as

E � �s2� � UTC�1U. (25)

For the ON–OFF system, we derived the following expression for theoptimal weights:

w1 � �w2 ��x1 s�

�x1 x1��

��

�

s Ps�s� ds

�1 � Nmax��

�

Ps�s� ds

(26)

and for this value, the optimal MSE is

EON–OFF � �s2� � 2Nmax

1 � Nmax

��

�

s Ps�s� ds2

��

�

Ps�s� ds

. (27)

For the ON–ON system, we derived the optimal weights w1 (ON cell withthreshold �1) and w2 (ON cell with threshold �2) as

w1

�J1H2�1 � N2 � N2H2� � J2N2�H3 � H1H2�

H1�1 � N1 � N1H1� H2�1 � N2 � N2H2� � N1N2�H3 � H1H2�2,

(28)

w2

�J2H1�1 � N1 � N1H1� � J1N1�H3 � H1H2�

H1�1 � N1 � N1H1� H2�1 � N2 � N2H2� � N1N2�H3 � H1H2�2

(29)

where

H1 ��1

�

Ps�s� ds, H2 ��2

�

Ps�s� ds and H3 ��max��1,�2�

�

Ps�s� ds

and

J1 � ��1

�

s Ps�s� ds and J2 ��2

�

s Ps�s� ds

and for this value, the optimal MSE is


EON–ON � �s2� �J2

2H1N2�1 � N1 � N1H1� � J12H2N1�1 � N2 � N2H2� � 2N1N2J1J2�H3 � H1H2�

H1�1 � N1 � N1H1�H2�1 � N2 � N2H2� � �H3 � H1H2�2N1N2

. (30)

Note that here N1 and N2 are the maximal firing rates of cell 1and cell 2, respectively. In the case of constraining the maximalspike count to Nmax, then N1 � N2 � Nmax. When the total meanspike count is constrained, then N1 and N2 are optimized. Given theMSE, the signal-to-noise ratio (SNR) of the reconstruction wasdefined as

SNR ��s2�

�E��

�s2�

��s � sest�2�

. (31)

Again, we studied the effect of stimulus sparseness by using two staticstimulus distributions, Gaussian and Laplace, with mean 0 and SD 1 (seeFig. 8).

Linear estimator: dynamic. We also studied how dynamic stimuli areencoded by ON–OFF and ON–ON systems (see Figs. 9, 10). Dong andAtick (1995) have shown that, in the regime of low spatial frequencies,the temporal part of the power spectrum of natural movies falls off as apower law with exponent �2. Therefore, we modeled temporal stimuli aswhite noise low-pass filtered with a time constant of � � 100 ms, longerthan all other time constants in the system (i.e., the temporal biphasicfilter below). The input into the ganglion cell nonlinearity is a linearlyfiltered version, g(t), of the temporally correlated stimulus, s(t), with atemporally biphasic filter:

f�t� �1

�14 t3e�t/�1 �

1

�24 t3e�t/�2 for t � 0, (32)

and 0 otherwise, where �1 � 5 ms, �2 � 15 ms, and � 0.8 were takenfrom fits to macaque ganglion cells (Chichilnisky and Kalmar, 2002;Pitkow et al., 2007). This filter represents the temporal processing bybipolar and amacrine cell circuits preceding ganglion cells in the retina(see Fig. 9).

The MSE and the optimal decoding filters (which in this case havetemporal structure) were derived using Equations 22 and 24 (Fig. 10).For simplicity, we provide expressions for these filters and the MSE in theFourier domain, but results in the main text are presented in the timedomain (see Figs. 9, 10). Note that we define the Fourier transform of a

temporal signal x(t) as X�w� � ��

�x�t�e�i�tdt and the inverse Fourier

transform as x�t� � 1/�2��

�X�w�ei�td�. We found that the optimal

ON and OFF filters were symmetric:

FON�� FOFF��

Q��e��2/ 2

2�2

H�� Nmax�G1�� G3��(33)

and the optimal MSE

EON�OFF � �s2�t�� 1 �2

��

� Nmax�Q��2e��2

2�2/�1

H�� Nmax�G1�� G3��d��,

(34)

where H is defined as in Equation 19, the function Q(�) is the Fouriertransform of the cross-correlation between the stimulus, s(t), and thefiltered stimulus, g(t) � �f(t)s(t)�,

Q�� 1

1 � �2�2� 1

�1 � i��1�4 �

1

�1 � i��2�4 , (35)

and G1(�) and G3(�) are the Fourier transforms of

g1�t�

� ��

� 1

2�e�z2/ 2 H�� t�� z

1 � ��t�� H�� t�� sgn��t�� z

1 � ��t�� dz,

(36)

g3�t�

� ��

� 1

2�e�z2/ 2 H�� t�� z

1 � ��t�� H�� t�� sgn��t�� z

1 � ��t�� dz,

(37)

where �(t) � c(�)/ 2 is the normalized autocorrelation function of thefiltered stimulus g(t), with 2 � c(0). The power spectrum of the filteredstimulus g(t) is given by (note that this is a real number)

C�� 1

1 � �2�2 � 1

�1 � i�1��4 � 1

�1 � i�2��4 � 2

. (38)

For each Nmax, we determined the thresholds for the ON and OFF cells atwhich the minimum MSE was achieved (see Fig. 10).

Similarly, we can write the expressions for the two filters and theminimum MSE for the system of two ON cells:

FON1��

Q��H��2� � N2G2��e�� 1

2/ 2

2�2� N2G3��

e��22/ 2

2�2H��1� � N1G1��H��2� � N2G2�� N1N2�G3��2

(39)

with a similar expression for FON2 by swapping the subscripts 1 and 2,and the optimal MSE is

EON�ON � �s2�t�� 1 �1

��

� �Q��2�N1H��2� � N2G2��e��1

2

2�2/�1� N2H��1� � N1G1��

e��22

2�2/�1� 2N1N2G3��

e��12/ 2��2

2/ 2

2�2/�1

H��1� � N1G1��H��2� � N2G2�� N1N2�G3��2 d��,

(40)


where G1(�) is the Fourier transform of Equation 36 with �1 instead of �,G2(�) is the Fourier transform of Equation 36 with �2 instead of �, andG3(�) is the Fourier transform of:

g3�t�

� ��

� 1

2�e�z2/ 2 H��1 � ��t�� z

1 � ��t�� H��2 � ��t�� sgn��t�� z

1 � ��t�� dz.

(41)

ResultsOur aim is to inspect the phenomenon of sensory pathway split-ting by the criteria of efficient coding. The efficient coding hy-pothesis holds that sensory systems are designed to optimallytransmit information about the natural world given limitationsof their biophysical components and constraints on energy use(Attneave, 1954; Barlow, 1961). This theory has been remarkablysuccessful in the retina, where it predicts correctly a generic shapefor ganglion cell receptive fields in both space and time (Atick andRedlich, 1990, 1992; van Hateren, 1992a,b). However, the retinalganglion cells are not a homogeneous population, but split into�20 different types (Wassle, 2004; Masland, 2012) with diverseresponse properties. In many cases, these responses are highlynonlinear and thus cannot be characterized entirely by a receptivefield (Gollisch and Meister, 2010; Pitkow and Meister, 2014).Thus, an understanding of what drives the early pathway split willlikely require a treatment that takes the nonlinearities into ac-count.

The simplest and most prominent pathway split—in the ret-ina and other sensory systems—involves ON and OFF responsepolarities: some neurons are excited by an increase in the stimu-lus and others by a decrease (Kuffler, 1953; Bennett, 1971; Wassle,2004; Chalasani et al., 2007; Joesch et al., 2010; Gallio et al., 2011).Therefore, we begin by considering a simple sensory system oftwo neurons, with ON-type and OFF-type polarity encoding acommon stimulus variable. We will compare the efficiency of thiscode to a system of two neurons with the same response polarity,say both ON-type. This way, one can measure the specific benefitsof nonlinear pathway splitting. Of course, the specifics of thiscomparison will depend on detailed assumptions of the model.We will evaluate different versions of the stimulus–responsefunction, starting with the simplest nonlinear form and progress-ing to more realistic versions. Efficient coding theory predictsthat the optimal neuronal response properties depend on thenatural stimulus statistics and the constraints on the system.Therefore, we consider different stimulus distributions and dif-ferent constraints on the neural system. Finally, we will measurecoding efficiency by three different criteria: (1) the mutual infor-mation between the response and stimulus; (2) the MSE achievedby a linear downstream decoder; and (3) the mean spike countneeded to achieve a given performance (information or decodingerror). We focus on results that emerge as robust under manydifferent conditions.

Information transmission in a binary response model withPoisson output noiseThroughout this study, we assume that a neuron transmits infor-mation about a stimulus via the spike count observed over shorttime windows. The duration, T, of this coding window is chosenbased on the observed dynamics of responses of retinal ganglioncells, typically 10 –50 ms (Uzzell and Chichilnisky, 2004; Pitkowand Meister, 2012).

To build intuition for the benefits of ON–OFF splitting, wefirst studied a binary model of the response, which assumes thatthe firing rate of a neuron can only take two values: 0 and maxi-mal, �max (Fig. 1). An ON cell is silent when the stimulus liesbelow a threshold �1 and fires at the maximal rate above thresh-old. An OFF cell fires when the stimulus is below its threshold �2

and remains silent above. For a given firing rate and time win-dow, the spike counts follow Poisson statistics. In this modelsystem, one can derive analytic solutions for the coding efficiency(see Materials and Methods). Although this set of assumptionsmay seem grossly simplified, the binary response function isprovably optimal under certain constraints (Stein, 1967; Shamai,1990; Bethge et al., 2003; Nikitin et al., 2009) and even offers areasonable approximation of retinal ganglion cell behavior (Pit-kow and Meister, 2012). In later sections, we will relax theseassumptions.

We begin by considering the mutual information transmittedby the responses about the stimulus. This information dependson the maximum expected spike count Nmax � �max T. Biophys-ically, such a constraint on the maximal firing rate arises naturallyfrom refractoriness of the spike-generating membrane. Given alimit on the maximal firing rate, one can derive the optimal place-ment of the two thresholds �1 and �2 and compute the resultinginformation. This efficiency measure is independent of the stim-ulus distribution as long as its cumulative distribution is contin-uous (see Materials and Methods and Fig. 2). As expected,information increases monotonically as one raises Nmax (Fig. 3A).Interestingly, the ON–OFF and ON–ON systems encode exactlythe same amount of information (Fig. 3A) over the entire range ofNmax.

The optimal thresholds can be measured in terms of the frac-tion of stimuli below threshold, ranging from 0 to 1 (Fig. 2). Forthe ON–OFF system, the optimal thresholds depend only weaklyon Nmax (Fig. 3B). They are mirror symmetric about the stimulusmedian (�2 � 1 � �1), such that each of the two neurons reportsthe same fraction of stimulus values. Moreover, the threshold ofthe OFF cell is always below the threshold of the ON cell, resultingin an optimal threshold placement where the central stimulusregion is coded by the silence of both cells. In contrast, althoughthe thresholds of the optimal ON–ON system of the two ON cellsbegin identical for low Nmax, as Nmax increases, the separationbetween the two thresholds also increases.

To understand these behaviors, it helps to look more closely athow the stimulus is encoded. The two thresholds divide the stim-ulus range into three regions (Fig. 3B). When the ON cell fires,the stimulus lies above the ON threshold with certainty; likewise,when the OFF cell fires, the stimulus lies below the OFF thresh-old. When neither cell fires, the stimulus may lie in the centerregion or not, because zero spike counts can happen occasionallyeven at the maximal firing rate. This ambiguity gets smaller thelarger the maximal firing rate, and in the limit of very large firingrates, the stimulus can be assigned uniquely to each of the threeregions. Thus, in the low noise limit (large Nmax), the systemmaximizes information by maximizing the output entropy, i.e.,by making those three regions equal in size, which explains thelimiting values of the thresholds at 1⁄3 and 2⁄3 for large Nmax (Fig.3B), resulting in log2(3) bits of information (Fig. 3A). A similarargument holds for the ON–ON system, leading to the samethreshold choices at high Nmax. However, at low Nmax, the twosystems act differently: whereas the ON–OFF system responds toboth high and low stimulus values, the ON–ON system respondsonly to high stimulus values but uses both neurons to do so, thusincreasing the SNR. Thus, the representational benefit of the


0 2 4 6 8 100

0.4

0.8

1.2

1.6

Info

rmat

ion

[bits

]

Total mean spike count

Maximum expected spike count

MAX

0

0.4

0.8

1.2

1.6

Info

rmat

ion

[bits

]

0 2 4 6 8 10

MEAN

low MAX spike count high MAX spike count

0.4

0.8

0.6

1.0

0.2

0

Opt

imal

thre

shol

dsMEAN

0 2 4 86 10Total mean spike count

0 2 4 6 8 100

0.4

0.8

1.2

1.6

Info

rmat

ion

[bits

]

0.4

0.8

0.6

1.0

0.2

00 2 4 6 8 10

MEAN MEAN


Opt

imal

thre

shol

ds


A B

H

E F

ON-ON (opt)

2ON (ident)

ON-OFF

ON-ON

ON-ON (opt)

2ON (ident)

G

C D

Rat

io o

f inf

orm

atio

n

ON-OFF/ON-ON (opt)

Total mean spike count0 2 4 6 8 10

0 2 4 86 10


Opt

imal

thre

shol

ds

MAX

0.4

0.8

0.6

1.0

0.2

MEAN

low MEAN spike count high MEAN spike count

low MEAN spike count high MEAN spike count

I

0.7

0.8

0.9

1.0N

orm

aliz

ed to

tal m

ean

0.6

0 0.5 1.0 1.51

1.1

1.2

1.3

1.4

1.5

Information [bits]

Rat

io o

f tot

al m

ean

spik

e co

unt

ON-ON (opt)/ON-OFF

MEAN

ON-ON (opt)/2ON (ident)


Rat

io o

f inf

orm

atio

n

0.7

1.0

0.8

0.9

0.6

Frac

tion

of m

ean

of O

N2

1

1.05

1.1

1.15

1.2

1

1.1

1.2

1.3

1.4

1.5

1.6

ON-OFF

ON-ON

ON-OFF

ON-ON

ON-OFF

ON-ON

MEAN

Figure 3. Mutual information with binary neurons and Poisson output noise. A, The mutual information for an ON–OFF and an ON–ON system is identical when constraining the maximumexpected spike count, Nmax. Inset, The ON–ON system uses more spikes to convey the same information. The total mean is normalized by Nmax. B, The thresholds of the response functions for theoptimal ON–OFF and ON–ON systems when constraining Nmax. Right, Optimal response functions in the limits of low and high Nmax. C, The ON–OFF system transmits more information than theON–ON system when the total mean spike count is constrained. Inset, the fraction of total mean spike count assigned to the ON cell with the smaller threshold. D, The thresholds of the responsefunctions for the optimal ON–OFF and ON–ON systems when constraining the total mean spike count. Right, Optimal response functions in the limits of low and high mean spike count (compare withB). E, The optimal ON–ON system transmits more information than a system of identical ON cells. F, The thresholds of the response functions for the optimal ON–ON and identical 2ON systems asa function of the total mean spike count. G, The ratio of mutual information transmitted by the optimal ON–OFF versus ON–ON systems in C. H, The ratio of mean spike counts required to convey thesame information by the ON–ON versus ON–OFF systems. I, The ratio of mutual information from the optimal ON–ON versus identical 2ON systems in E.


ON–OFF system is exactly compensated in the ON–ON systemby its increased reliability, leading to the same information. Re-markably, the information achieved by the two systems remainsidentical throughout the range of Nmax. We have found no intu-itive explanation for this analytical result.

Despite the equality in the information conveyed by the twosystems, their efficiency differs. The ON–ON system requires upto 50% more spikes on average than the ON–OFF system (Fig.3A, inset). Therefore, the ON–OFF system conveys higher infor-mation per spike than the ON–ON system. Next, we asked howthe two systems behave if one constrains the total firing rateaveraged over all stimuli rather than the maximal firing rate.Biophysically, such a constraint may arise from limits on theoverall energy use for spiking neurons. Now each neuron canchoose to use a high maximal firing rate for a small stimulusregion or a low maximal firing rate over a larger region (Fig. 3C,inset). Also, the firing may be distributed unequally between thetwo neurons. Under this constraint on the mean spike count andallowing the two ON cells to adjust their maximal firing rates, wefound that the ON–OFF system transmits more information thanthe ON–ON system (Fig. 3C). The largest performance ratio(ON–OFF/ON–ON) was 1.15, achieved at a total mean spikecount of 0.4 (Fig. 3G). Interestingly, in both the low and highmean firing rate limits, the information ratio approaches 1. Theability of the ON–ON system to achieve similar information perspike as the ON–OFF system in these limits is attributable to thefreedom to optimize not only the thresholds but also the maximalfiring rates of the two ON cells.

Alternatively, one may ask how many more spikes theON–ON system needs to match the information transmission ofthe ON–OFF system: this excess varies from �0 to 50% (Fig. 3H).Again, we can understand the optimal threshold choices: at verylow Nmean, the constraint on the mean spike count forces bothsystems to signal only an infinitesimally small stimulus region, tomaximize the SNR. The ON–OFF system chooses small regionson either end of the range, whereas the ON–ON system uses bothneurons redundantly for the same region. In the limit of largeNmean, the optimal thresholds again divide the stimulus rangeinto equal thirds and maximize output entropy (Fig. 3D). For theentire range of Nmean, the ON–ON system assigned �2⁄3 of thespikes to the ON cell with a lower threshold (Fig. 3C, inset).

Under both constraints analyzed so far, Nmax or Nmean, theoptimal system of two ON cells consists of two neurons that differin their thresholds or firing rates. Because this is another instanceof nonlinear pathway splitting, we inspected the benefits of theoptimal ON–ON system by comparing it to a suboptimal

ON–ON system constrained to use two identical response func-tions. The optimal system of ON cells transmits higher informa-tion when the total mean spike count Nmean is constrained (Fig.3E, I; a similar result was obtained while constraining Nmax; datanot shown). The largest difference appears in the limit of highspike counts, where the suboptimal ON–ON scheme with iden-tical response functions can distinguish only two equal stimulusregions [log2(2) � 1 bit] compared with three regions for theoptimal ON–ON system [log2(3) bits; Fig. 3F].

Although ON–OFF splitting yields a coding benefit amongneurons encoding the same stimulus variable, the results of thissection demonstrate that the details of the benefits of ON–OFFsplitting depend on the assumed constraints. Two cells of oppo-site polarity have the same information capacity as two cells of thesame polarity that use different response functions, when no ad-ditional cost than a constraint on their maximal firing rates isapplied. What makes the ON–OFF system superior over ON–ONis the cost in terms of the mean firing rate; the ON–OFF systemtransmits higher information per spike than the ON–ON systemwhen the maximal firing rate is constrained. However, when thetotal mean spike cost is constrained while allowing the two ONcells to adjust their maximal firing rates, the two systems achievethe same information per spike in the regime of high and lowspike counts. The largest information benefit for the ON–OFFsystem of �15% emerges in the intermediate firing rate regime,which is most biologically relevant.

Information transmission with non-monotonicnonlinearitiesWe also considered non-monotonic response functions, for ex-ample, a U-shaped nonlinearity, where a cell fires for both highand low stimulus values but not in between. We will call this aU-cell. An additional ON cell with monotonic nonlinearity canresolve the ambiguity between high and low stimuli.

A system with a U-cell and an ON cell encodes even higherinformation than the ON–OFF and ON–ON systems when themaximal spike count is constrained (Fig. 4A; the largest ratio is1.26). This is because the U–ON system allows the stimulus rangeto be split into a total of four regions (as opposed to three in Fig.3). Therefore, the maximum information transmitted by this sys-tem is log2(4) � 2 bits. From the optimal thresholds derived forthis system (Fig. 4B), we compute the resulting total mean firingrate at which this higher information is achieved (Fig. 4A, inset).The total mean of the U-ON system is larger than the mean usedby the ON–OFF system; therefore, the information per spike stillremains the highest for the ON–OFF system. Similarly, a pair of

0 2 4 6 8 100

0.5

1

1.5

2

0 2 4 6 8 100

0.2

0.4

0.6

0.8

1

Info

rmat

ion

[bits

]

Opt

imal

thre

shol

ds

Nor

mal

ized

tota

l mea

n

Maximum expected spike count Maximum expected spike count

A B

UOF

FO

N

U

ON

1=O

N2

ON

U and ON

ON and ON

ON and OFF

U

OF

FO

N UO

N

low MAX spike count high MAX spike count

MAXMAX

ON

1O

N2

0.7

0.9

1

0.8

0.6

Figure 4. Mutual information with a non-monotonic response function. A, The mutual information for three different systems: (1) one ON and one OFF cell; (2) two ON cells; and (3) one U-cellwith a U-shaped nonlinearity and an ON cell. Although the first two systems divide the stimulus range into three regions, the last divides it into a total of four regions and transmits a maximum oflog2(4) � 2 bits of information. Inset, The total mean spike count (normalized by Nmax). B, The optimal thresholds of the response functions for the three systems in A when constraining Nmax. Right,Optimal response functions in the limits of low and high Nmax.


U-cells produces identical results to using one U-cell and one ONcell (data not shown). Therefore, it appears that two pathways ofopposite polarity, one ON and one OFF, provide a more cost-effective way of coding a single stimulus variable, and for theremainder of the report, we proceed using ON or OFF cells ratherthan U-cells.

Information transmission with sigmoidalresponse nonlinearitiesThe stimulus–response functions of many neurons are not assharp as a binary step (Chichilnisky and Kalmar, 2002; Uzzell andChichilnisky, 2004). Therefore, we next considered sigmoidal re-sponse nonlinearities with limited gain (see Materials and Meth-ods; Fig. 5A).

Information transmission by the ON–OFF and ON–ON sys-tems behaved similarly to the case of step nonlinearities. Forinstance, when constraining the maximum expected spike count,the information was very similar (although not identical; the larg-est observed difference was 3%) for the two systems, whereas thetotal mean spike count was higher for the ON–ON system (datanot shown). Similarly, when constraining the total mean spikecount and optimizing the maximal firing rates of the cells, theON–OFF system achieved higher mutual information about thestimulus than the ON–ON system (Fig. 5B).

A qualitative difference from the case of step nonlinearities(infinite gain) occurs when one examines the optimal thresholdsfor the two systems. In the ON–ON system, the two cells adoptdifferent thresholds at high gain values (as in the infinite gainbinary model; Fig. 3D), but at low gain, their thresholds collapse

to the same value (Fig. 5C; Kastner et al.,2014). The critical gain value at which thisoccurs depends on the mean spike count(Fig. 5C). For the ON–OFF system, thetwo thresholds were different for all gainvalues but approached each other in thelimit of low gain and high mean spikecount (Fig. 5D). Just as observed with stepnonlinearities, the optimal thresholds forthe ON and OFF cells were mirror sym-metric about the stimulus median. This isexpected based on the symmetries of theproblem.

We also considered the effect of anasymmetric distribution of stimuli. Forexample, the distribution of spatial con-trasts in natural images has a heavier tailbelow the median than above (Tadmorand Tolhurst, 2000; Ratliff et al., 2010).Under such a distribution, the ON andOFF cells adopt asymmetric responsefunctions: the threshold of the OFF cellmoves away from the stimulus medianand that of the ON cell moves closer (datanot shown). Such an asymmetry has in-deed been observed for the nonlinearitiesof ON and OFF macaque ganglion cells(Chichilnisky and Kalmar, 2002). Highermaximal firing rates of the ON cell (Uzzelland Chichilnisky, 2004) or a steeper gainof the OFF cell (Chichilnisky and Kalmar,2002) can also produce asymmetries inthe response functions (data not shown).

Realistic encoding and noise models from retinal dataGiven the intuition acquired from studying the effects of variablegain on the response functions, we next explored two-neuronsystems with realistic choices for the response gain, firing rates,noise, and stimulus distributions. For the stimulus–responsefunction, we used a sigmoidal nonlinearity derived frommeasured responses of primate (Chichilnisky and Kalmar, 2002;Uzzell and Chichilnisky, 2004) and salamander (Pitkow andMeister, 2012) retinal ganglion cells. Because the spiking re-sponses of ganglion cells are more regular than expected fromPoisson statistics (Berry et al., 1997; Uzzell and Chichilnisky,2004), we used a noise distribution that was measured empirically(Pitkow and Meister, 2012). With this encoding model, the shapeof the stimulus distribution matters for the efficiency of informa-tion transmission. In choosing a stimulus distribution, we fol-lowed previous work on image processing of natural scenes(Field, 1994; Bell and Sejnowski, 1997), which suggests that thesignal encoded by a typical retinal ganglion cell follows a Laplacedistribution. Similar distributions with positive kurtosis alsoarise from temporal filtering of optic flow movies during humanlocomotion and simulated fixational eye movements on staticimages (Ratliff et al., 2010). However, the main conclusions wepresent here hold also for a Gaussian distribution.

Three parameters characterize the sigmoidal nonlinearity(Eqs. 10 and 11): (1) the maximum expected spike count Nmax;(2) the gain ; and (3) the threshold � (Fig. 6A). These parameterswere set to measured values from salamander (Pitkow and Meis-ter, 2012) and macaque (Chichilnisky and Kalmar, 2002) gan-glion cells (see Materials and Methods). Under these conditions,

0 0.1 0.2 0.3 0.4 0.5 0.60

0.2

0.4

0.6

0.8

1

1/gain

Opt

mal

thre

shol

ds O

N−

OF

F

0 0.1 0.2 0.3 0.4 0.5 0.61

1.05

1.1

1.15

1.2

1.25

1.3

1/gain

Rat

io o

f Inf

orm

atio

n (

ON

−O

FF

/ON

-ON

)

Stimulus

decrease gain

Firi

ng r

ate

Maximum expectedspike count

Gai

n

Threshold

0.1

510

1

increase

B

C D

A

0 0.1 0.2 0.3 0.4 0.5 0.60

0.2

0.4

0.6

0.8

1

1/gain

Opt

imal

thre

shol

ds O

N-O

N increase

Figure 5. Mutual information with sigmoidal response functions and Poisson output noise. A, The sigmoidal response model(see Materials and Methods) is illustrated for an ON cell showing the maximum expected spike count Nmax, the gain (red dashed),and the threshold (green dashed) of the cell. The schematic shows the effect of varying the gain for a fixed threshold. B, The ratioof mutual information transmitted by the optimal ON–OFF versus ON–ON systems about a stimulus drawn from a Laplace distri-bution. The total mean spike count of each system, Nmean, was constrained. C, The optimal thresholds of the sigmoidal responsefunctions for the ON–ON system when constraining the total mean spike count. Below a critical value of the gain, the twothresholds are identical. D, The optimal thresholds of the sigmoidal response functions for the ON–OFF system when constrainingthe total mean spike count. The two thresholds are different except in the limit of small gain and large mean spike count.


the ON–OFF and ON–ON systems transmitted almost identicalinformation (Fig. 6B), just as observed with the simple binaryresponse model (Fig. 3A). Again, the ON–ON system requiredconsiderably more spikes on average (Fig. 6C; ratio of total meanspike count for ON–ON/ON–OFF, 1.73 for salamander and 1.50for macaque). However, the mean firing rates produced by thismodel greatly exceed those observed in real ganglion cells: forexample, the model salamander ganglion cells fire at average ratesof 12–32 Hz compared with 1.1 Hz for the average real salaman-der ganglion cells (Pitkow and Meister, 2012). This discrepancyarises because the optimal model thresholds are closer to thestimulus median (ranging from 0.3 to 0.5 in units of the stimulusSD) than the experimentally measured value of 2. For the ma-caque parameters, the mean firing rates and thresholds followsimilar patterns: model neurons have mean firing rates in therange of 65–132 Hz compared with 30 Hz for real neurons (Uzzelland Chichilnisky, 2004). Again, this discrepancy results frommismatched thresholds (0.2– 0.5 in the model vs 1.3 in experi-ment, again in units of the stimulus SD).

This mismatch disappears if one constrains the total meanfiring rate instead of the maximum rate. After setting the meanfiring rate to the values observed experimentally, the ON–OFFsystem outperformed the ON–ON system for both salamanderand macaque parameters (Fig. 6D; information ratio ON–OFF/ON–ON, 1.3 for salamander and 1.4 for macaque), just as ob-served under the simpler response model in the intermediatesignal-to-noise regime (Fig. 3G,H). Conversely, the ON–ON sys-tem requires a considerably greater firing rate to match the per-formance of the ON–OFF system: almost twice the rate measuredexperimentally in the case of macaque data (Fig. 6E). In thismodel—with a constraint on the mean firing rate—the optimalthresholds of the response functions are much closer to the ex-

perimentally measured values (2.0 for both analysis and experi-ment for salamander and 1.1–1.9 for analysis vs 1.3 forexperiment for macaque, in units of the stimulus SD).

Along with the thresholds, the optimization procedure also ad-justed the maximal firing rates, and their values are again muchcloser to the experimental values than the mean firing rates obtainedpreviously under a constraint on the maximum rate (salamander,12–38 Hz in the model and 48 Hz in the data; macaque, 200–580 Hzin the model and 220 Hz in the data). Therefore, although we did notconstrain the maximal firing rates in this scenario, the optimal solu-tion converged on reasonable values. Note that these maximal rateswere rarely encountered: because of the high firing thresholds, themodel retinal ganglion cells spent only 3–7% of the time in the upperhalf of the sigmoid nonlinearity.

In summary, this section shows that the main results observedwith a binary response model and Poisson firing hold also undermore realistic assumptions about the encoding function and out-put noise. For the remainder of the analysis, we will continueusing binary nonlinearities and Poisson output noise, becausethese choices are conducive to analytical solutions and thus allowus to explore in greater detail other factors that might influenceinformation transmission. We also continue constraining themean firing rate, because our results show that it produces morerealistic predictions for retinal nonlinearities than a constraint onthe maximal firing rate.

The effect of common input noiseUntil now, we made the assumption that variability in the spikingoutput of each individual ganglion cell is the dominant source ofnoise, consistent with the small shared variability among these neu-rons observed in some studies (Pitkow and Meister, 2012). However,under different experimental conditions, nearby ganglion cells ex-

ON-OFF ON-ON

Info

rmat

ion

[bits

]In

form

atio

n [b

its]

Tot

al m

ean

spik

e co

unt

Tot

al m

ean

spik

e co

unt

salamander

macaque

salamandermacaque

salamander

macaque macaque

B C

D E

Maximum expectedspike count

Gai

n

Threshold Stimulus

Firi

ng r

ate

A

MAX MAX

MEANMEAN

0

0.2

0.4

0.6

0.8

1

1.2

1.4

ON-OFF ON-ON

ON-OFF ON-ON ON-OFF ON-ON ON-OFF ON-ON ON-OFF ON-ON

ON-OFF ON-ON ON-OFF ON-ON0

0.5

1

1.5

2

2.5

0

0.2

0.4

0.6

0.8

1

1.2

salamander

0

0.2

0.4

0.6

0.8

1

Figure 6. Mutual information with response functions and noise from retinal data. A, Two sigmoidal nonlinearities for an ON cell (green) and an OFF cell (orange), describing the firing rate as afunction of stimulus with the maximum expected spike count Nmax, the gain , and the threshold �. The shaded curve denotes the Laplace stimulus probability distribution. B, The informationtransmitted by the ON–OFF and ON–ON systems, using nonlinearities measured from salamander (left) and macaque (right) ganglion cells (Pitkow and Meister, 2012). The maximum spike countwas constrained. C, The total mean spike count used by the ON–OFF relative to the ON–ON system to transmit the maximal information for the two types of nonlinearities in B. D, The informationtransmitted by the two systems using a constraint on the total mean spike count. E, The total mean spike count that would be needed by the ON–OFF and the ON–ON systems to transmit the sameinformation as the ON–OFF system shown in D.


hibit strong noise correlations, presumably arising from noise inshared presynaptic neurons, as early as the photoreceptors (Ala-Laurila et al., 2011). To capture these cases of neural encoding, weextended the existing model by adding a source of common additivenoise at the input to the two neurons; the combined input (stimulusplus noise) is passed through the ON and OFF nonlinearities andconverted into spike counts by a Poisson process (Fig. 7A). The inputnoise was drawn from a Gaussian distribution. Again, we computedthe information about the stimulus transmitted by ON–OFF andON–ON systems with binary response functions and optimally cho-sen thresholds, while holding a constraint on the total mean firingrate but allowing the cells to adjust their maximum firing rates. Spe-cifically, we examined the effect on information transmission byvarying the level of input noise and the sparseness of the stimulusdistribution.

We first studied the effect of input noise using a Laplace stimulusdistribution. As expected, addition of input noise degrades the infor-mation transmission in both systems and more so the stronger thenoise (data not shown). In terms of information capacity, the ON–OFF system outperforms the ON–ON system most significantly foran intermediate constraint on the total mean spike count, and theaddition of input noise further enhances those benefits (Fig. 7B). Therelative advantage of the ON–OFF system is maximal when the noisepower is approximately equal to the stimulus power. Under optimalconditions, the ON–OFF information rate is up to 34% higher thanfor the ON–ON system. Alternatively, if the goal is to transmit a fixedamount of information about the stimulus, the ON–OFF schemeachieves it with a much lower mean firing rate, by a factor of up to 3.4

(Fig. 7C). This far exceeds the maximal benefit without input noise(Fig. 3H). If the noise power exceeds stimulus power, these benefitsdecrease again (Fig. 7B,C).

One can trace the effects of input noise to the optimal thresholdsettings in the encoding model. As the input noise increases, theON–OFF system places its thresholds closer to the two ends of thestimulus range, where the stimulus values are less confounded withnoise (Fig. 7D). The ON–ON system does the same in the regime ofhigh spike counts (Fig. 7E). This implies that one of the ON cells firesalmost all the time, which produces the great difference in codingefficiency between the two models. At low mean spike count, theON–ON system again opts for redundant coding of high stimulusvalues, as in the model without input noise (Fig. 3D).

In the presence of input noise, the coding efficiency and theresulting optimal thresholds do depend on the shape of the stim-ulus distribution, unlike the situation without input noise (Fig.3). Besides the empirically founded Laplace distribution, we alsoexamined a Gaussian distribution that has been popular in manyneural coding models. The two distributions had matched meanand variance, but the Laplace distribution has higher kurtosis andthus greater stimulus sparseness. A higher stimulus sparsenessincreased the efficiency of the ON–OFF system relative to theON–ON system, as measured by the information rates (Fig. 7B,dashed) and the ratio of mean spike counts required to achieve agiven information (Fig. 7C, dashed). Similar to the effects of add-ing input noise, increasing stimulus sparseness also pushed thethresholds closer to the two ends of the stimulus range (Fig.7D,E).

input noise SD 0.25

stimulus SD 1.00

A

C

B

0 0.5 1 1.5.1

1.5

2

2.5

3

3.5

input noise SD 1.00

input noise SD 2.00

0.4

0.8

0.6

1.0

0.2

0

Opt

imal

thre

shol

ds

0 2 4 86 10Total mean spike count

0.4

0.8

0.6

1.0

0.2

0

Opt

imal

thre

shol

ds

0 2 4 86 10Total mean spike countInformation [bits]

Rat

io o

f mea

n sp

ike

coun

tR

atio

of i

nfor

mat

ion


D

E

ON

ON

ON

OFF

input noise SD 0.25 (Laplace)

no input noise


input noise SD 1.00 (Gaussian)

0.250.20

0.30

0

1.00

2.00

ON OFF

add inputnoise

stimulus

0

1

1.1

1.2

1.3

1.4

+


Figure 7. Input noise increases the benefits of ON–OFF splitting. A, Top, Input noise is added to the stimulus before the nonlinearities. Bottom, Distribution of stimulus (Laplace) and input noise(Gaussian with different SDs). B, Ratio of information transmitted by the ON–OFF versus ON–ON systems when constraining the total mean spike count. Input noise increases the benefit of theON–OFF system but non-monotonically. In this and subsequent panels, the dashed green line demonstrates the results for a Gaussian stimulus distribution with input noise SD 1 (compare to the fullgreen line for the Laplace stimulus distribution with SD 1). C, The ratio of total mean spike count needed to transmit a given information by the ON–ON versus the ON–OFF system, computed fordifferent values of the noise SD. For instance, to transmit 0.9 bits (dashed black line), the ON–ON scheme needs twice more spikes at input noise SD 0.2 and 2.75 times more spikes at input noise SD0.3. D, The optimal thresholds in the ON–OFF system get pushed toward the tails of the distribution as the input noise increases. E, The optimal thresholds in the ON–ON system are also distributedat the two tails of the distribution for large spike counts but cluster at one end of the stimulus distribution for low spike counts.


These results demonstrate that, in thepresence of input noise, the ON–OFF sys-tem again outperforms the ON–ON sys-tem. In fact, the benefits of ON–OFFcoding can be substantially greater thanwithout input noise. Bigger improve-ments in ON–OFF performance are ob-served for sparser stimulus distributionswhere one of the ON cells of the ON–ONsystem fires for a larger fraction of stimulinear the stimulus median.

Linear decoding as a measure ofcoding efficiencyThus far, we have measured the total in-formation transmitted by model neurons,without regard to how it can be decoded.An alternative criterion for coding effi-ciency is the ease with which these mes-sages can be decoded by downstreamneurons. In particular, it has been sug-gested that the important variables repre-sented by a population of neurons shouldbe obtainable by linear decoding of theirspike trains (Seung and Sompolinsky,1993; Marder and Abbott, 1995; Eliasmithand Anderson, 2002; Gollisch and Meis-ter, 2010). Hence, we now ask how well anoptimal linear decoder can extract thestimulus value from the firing of the twomodel neurons. In each coding window,the decoder sums the two spike countswith appropriate decoding weights to es-timate the stimulus value. We measure thequality of the decoding by the SNR, wheresignal is the power of the stimulus andnoise is the power of the difference be-tween the true stimulus and the estimateof the decoder (see Materials and Meth-ods). For any model cell pair, the decoderis optimized by choosing the two decod-ing weights that maximize the SNR. Asbefore, we compare an ON–OFF pair andan ON–ON pair, each with optimizedthresholds.

Because we wanted to compare thecoding benefit realized by two measures,mutual information and the SNR of theoptimal linear decoder, we returned tothe original noise model in which the solesource of noise was at the level of the spikeoutput with Poisson statistics. Unlike themutual information (Fig. 3), the results oflinear decoding depend on the shape ofthe stimulus distribution (see Materialsand Methods); hence, we examine the role of stimulus sparsenesson the accuracy of stimulus decoding by studying Laplace andGaussian stimulus distributions. As before (Fig. 3), we also exploredthe effects of two different constraints on encoding: first on the max-imal firing rate of each neuron and then on the total mean firing ratewhile allowing the maximal firing rates of the cells to vary.

As expected, a higher maximal or mean firing rate constraintproduced higher SNR after decoding (Fig. 8A,B). However, the

ON–OFF system achieved higher SNR than the ON–ON system,regardless of how the firing rates are constrained (Fig. 8C,D).With a constraint on the mean firing rate, the most pronouncedbenefit of ON–OFF coding was 40% (Fig. 8D). Alternatively, onecan ask how much firing would be required to accomplish acertain quality of decoding, and again the ON–OFF system wasmuch more efficient (Fig. 8G). For example, achieving the highestpossible SNR requires 4-fold more spikes with the ON–ON sys-


Opt

imal

thre

shol

ds

MAX

Opt

imal

thre

shol

ds

MEAN


E F

ON-OFF (Gaussian)

ON-ON (Gaussian)

MAX

SN

R O

N-O

FF

/ON

-ON


MEAN

ON-OFF (Laplace)

ON-ON (Laplace)

C D

A B


SN

R O

N-O

FF

/ON

-ON

MAX MEAN

Total mean spike countMaximum expected spike count

SN

R

SN

R

0 2 4 6 8 101

2

3

4

5

0 2 4 6 8 101

2

3

4

5

0 2 4 6 8 100

0.2

0.4

0.6

0.8

1

0 2 4 6 8 100

0.2

0.4

0.6

0.8

1

G

SNR

Tot

al m

ean

ON

-ON

/ON

-OF

F GaussianLaplace

1 2 3 4 51

3

5

7

9

GaussianLaplace

0 2 4 6 8 101

1.1

1.2

1.3

1.4

0 2 4 6 8 101

1.1

1.2

1.3

1.4

Figure 8. Linear decoding of stimuli from cell responses increases the benefits of ON–OFF splitting. A, B, The SNR of the stimulusreconstruction for two stimulus distributions (Gaussian and Laplace) and subject to two different constraints (maximum expectedspike count and total mean spike count). C, D, The ratio of SNR curves shown in A and B under the two different constraints. E, F, Theoptimal thresholds (shown as the fraction of stimuli below threshold) for the response functions under the two different con-straints (compare with Fig. 3 B, D). G, The ratio of total mean spike count needed by the ON–ON versus the ON–OFF system toachieve the same SNR. Larger benefits for ON–OFF are observed for the Laplace distribution.


tem than with the ON–OFF system if the stimulus distribution isGaussian (Fig. 8G). With the more kurtotic Laplacian stimulusdistribution, this increases to a 10-fold difference (Fig. 8G).

The optimal thresholds of the encoding neurons behave sim-ilarly to what was seen in previous instances of the model: at highspike count, the two neurons partition the stimulus range (Fig.8E,F), although not quite into thirds as seen under the mutualinformation criterion (Fig. 3B,D). The improved performance ofthe ON–OFF system relative to ON–ON in the case of the Laplacestimulus distribution occurs with placing the optimal thresholdscloser to the two ends of the stimulus range as in the case of themutual information with input noise (Fig. 7D,E). At low spikecounts, the ON–ON system again opts for redundant coding atthe positive tail of the stimulus distribution.

In summary, this section shows that, if one adopts the alter-native criterion of linear decodability, the ON–OFF system out-performs the ON–ON system by an even larger margin than seenbefore, both in terms of the SNR at a fixed maximum spike count,and the total mean spike count required to achieve a certain SNR.

Decoding a time-dependent stimulusWe next examined a more realistic encoding model in which thetask is to reconstruct a stimulus with a temporal structure. Dongand Atick (1995) measured the spatiotemporal correlation struc-ture of a large ensemble of natural movies and found that, in theregime of low spatial frequencies, the temporal power spectrum iswell fit by the universal power law with exponent �2. As a limit-ing case of this result, we used a Gaussian low-pass filtered stim-ulus with a correlation time longer than all other time constantsin the system (see Materials and Methods). To mimic how thecircuits of the retina encode such a stimulus, we first passed itthrough a linear filter with biphasic impulse response (Fig. 9), asseen in measurements from primate ganglion cells (Chichilniskyand Kalmar, 2002). This filtered stimulus was passed to the twoneurons in the encoding model (with binary response functions)and spikes were derived as an inhomogeneous Poisson process.The task of the linear decoder is to reconstruct the time-dependent stimulus by summing the output of two temporalfilters applied to the spike trains (Fig. 9). For a given set of thresh-olds, the decoding filters were derived analytically by maximizingthe SNR of the decoder (see Materials and Methods).

This generalized encoding model led to qualitatively identicalconclusions: the ON–OFF system outperformed the ON–ON sys-tem under all conditions (Fig. 10). Quantitatively, the benefits of theON–OFF system were further enhanced over previous instantiationsof the model. For example, the ON–OFF system produced a maxi-mal SNR improvement of 24% compared with ON–ON at the samemaximal firing rate of each cell (Fig. 10C) and 51% at the same totalmean firing rate (Fig. 10D). At the same SNR, the ON–OFF systemused 4.5 times fewer spikes (Fig. 10G). Interestingly, as in the case ofthe mutual information with input noise and the linear decoder ofstatic stimuli, the more favorable performance of the ON–OFF sys-tem compared with ON–ON goes hand-in-hand with a shifting ofthe optimal thresholds away from the stimulus median toward thetails of the distribution (compare Figs. 8E,F, 10E,F).

The coding benefit of ON–OFF splitting is comparable withthat from decorrelationUnder the assumptions considered so far, the ON–OFF systemgenerally outperformed the ON–ON system by an average of25% in information rate (15% in Fig. 3 and 34% in Fig. 7). Howsignificant is a 25% gain for the purpose of efficient coding insensory systems? We compare the present effects to another pop-

ular application of efficient coding theory: decorrelation. Ac-cording to the theory, early sensory circuits are designed toremove redundancies that arise in natural stimuli attributable tothe strong correlations they induce within the population of sen-sory receptors, for example, among nearby pixels of a visual im-age (Ruderman, 1994). The resulting neural signals are moreindependent, thus allowing greater information transmissiongiven the biophysical constraints on neural spike trains (Atickand Redlich, 1990, 1992; van Hateren, 1992b). How muchgreater?

Consider the simplest such system of just two neurons, eachencoding a scalar stimulus variable using a step-shaped responsefunction (Fig. 11; Pitkow and Meister, 2012). Suppose the twostimulus variables have positive correlation. To achieve optimalencoding, they first need to be decorrelated by lateral inhibitionso that the inputs to the two neurons become statistically inde-pendent before passing through the nonlinear response function(Pitkow and Meister, 2012). We ask how much information isgained by the decorrelation step under the most favorable condi-tions. First, decorrelation is most effective in the regime of lownoise, because the procedure tends to amplify the unshared noise

reconstructed stimulus

decoding filters

ON

time

biphasic filter

stimulus

OFF ON ON1 2

filtered stimulus

−0.5 0.50time [s]

−0.5 0.50time [s]

Figure 9. Schematic of linear decoding of time-varying stimuli from cell responses. A stim-ulus with temporal correlations was first passed through a biphasic filter and then encoded byan ON–OFF or ON–ON system. The resulting time series of spike counts were convolved withdecoding filters to produce the reconstructed stimulus estimate. Examples of the decodingfilters for each system (ON–OFF and ON–ON) and the resulting reconstructed stimuli are shownfor Nmax � 10. Stimulus (black), stimulus estimate from ON–OFF (blue), and stimulus estimatefrom ON–ON (red).


over the shared signal (Atick and Redlich,1992). Thus, we will assume zero noise.Furthermore, the greatest gain accrues ifthe two signals are strongly correlated, sayalmost identical. Without lateral inhibi-tion, the two neurons are forced to repre-sent essentially the same stimulus variable(Fig. 11A). They can adjust their thresh-olds so as to encode three different regionsof the stimulus (as in Fig. 3) and thustransmit up to log2(3) bits of information(Fig. 11B). In contrast, if the two variablesare first decorrelated and then each en-coded with a binary neuron, the informa-tion is 2 bits (Fig. 11C). Thus, the benefitfor a two-neuron system from imple-menting stimulus decorrelation is [2 �log2(3)]/log2(3) � 26% in the best case.We conclude that, for the goal of codingefficiency, ON–OFF pathway splitting isjust as significant as decorrelation, a cir-cuit function whose interpretation standsas a celebrated success of efficient codingtheory (Atick and Redlich, 1990; Atick,1992).

A sparse stimulus distributionemphasizes the benefits ofON–OFF codingWhereas previous sections assumed thatall stimulus values were in a sense equallyimportant for transmission, we now sup-pose that the downstream brain areas areparticularly concerned about large devia-tions in the stimulus of either sign. This isecologically plausible in several situations.For example, consider a visual observer ofa scene in which a potential prey animalmoves against a static background (Dongand Atick, 1995). Almost every pixel in theimage stays constant across time, except afew pixels at the leading and trailing edgesof the moving object. These show a sud-den increase or decrease in intensity de-pending on the relative brightness oftarget and background. Here all the eco-logical relevance lies in the rare stimulusvalues with large positive or negativechanges. They are embedded in a sea ofvalues with zero or very small change,which do not need to be discriminateddownstream. Similar scenarios can bedrawn for other senses, such as olfactionand electrosensation.

To implement this subjective criterion of ecological relevance,we can compress the stimulus distribution into just three discretevalues: 0, �1, and �1. Most of the time, the stimulus is 0 and onlyrarely �1 or �1. We suppose that �1 and �1 occur with equalfrequency p�1 � p1. Thus, the stimulus distribution has only oneparameter, and we will use the “sparseness,” 1/p1. Again, thestimulus drives two model neurons with binary rate functionsand Poisson firing statistics, and we compute the mutual infor-mation between stimuli and responses, with a constraint on the

total mean firing rate while we allow for the cells to adjust theirmaximal firing rates (Fig. 12A).

As expected, the information rate increases with the meanfiring rate (Fig. 12B). There is an extended region of firing rates inwhich the ON–OFF system transmits twice as much informationas ON–ON. This region increases with the sparseness of the stim-ulus. For example, for a sparseness of 100, the region extends overa �100-fold range of firing rates. As before, one can also phrasethe comparison in terms of how many spikes are needed to


Opt

imal

thre

shol

ds

MAX

Opt

imal

thre

shol

ds

MEAN


E F

SN

R O

N-O

FF

/ON

-ON


MEAN

G

C D

A B


SN

R O

N-O

FF

/ON

-ON

MEAN

Total mean spike countMaximum expected spike count

SN

R

SN

R

0 2 4 6 8 10

0 2 4 6 8 100 2 4 6 8 10

0 2 4 6 8 100 2 4 6 8 10

ON-OFF

ON-ON

MAX

0 2 4 6 8 101

2

3

4

5

6

7

1

2

3

4

5

6

7

0

0.2

0.4

0.6

0.8

1

0

0.2

0.4

0.6

0.8

1

1

1.1

1.2

1.3

1.4

1

1.2

1.4

1.6

1.8MAX

SNR

Tot

al m

ean

ON

-ON

/ON

-OF

F

1 2 3 4 5 61

2

3

4

Figure 10. Decoding of temporally correlated stimuli benefits from ON–OFF splitting. A, B, The SNR from linear reconstructionof a dynamic stimulus while constraining the maximum expected spike count (A) or the total mean spike count (B). C, D, The SNRachieved by the ON–OFF versus the ON–ON system under the conditions of A and B. E, F, The optimal thresholds (shown as thefraction of stimuli below threshold) for the response functions of the ON–OFF and ON–ON systems of A and B. G, The ratio of totalspike counts used by the ON–ON versus the ON–OFF system to achieve the same SNR.


achieve a certain information rate. If the target rate is above half-maximum, then the ON–OFF system requires far lower firingrate than the ON–ON system (Fig. 12C). The factors here areenormous, on the order of the sparseness of the stimulusdistribution.

These large effects can be understood by considering the di-lemma of the ON–ON system (Fig. 12A). It can signal the s � �1events only by cessation of firing in one of the neurons, but thatrequires maintaining a constant firing rate during the much morefrequent s � 0 events. If a low mean firing rate is specified, thenthat is simply not an option. Thus, the optimal ON–ON modelforgoes signaling the s � �1 events entirely and therefore has todiscard half the available stimulus information (Fig. 12B). Onlywhen the available mean firing rate increases enough does theencoding through silence become an option. In contrast, theON–OFF system can allocate spikes exclusively to the rare eventss � �1 and thus can encode both of them throughout the rangeof mean firing rates.

This section illustrates that the coding advantages of an ON–OFF coding scheme become truly dramatic when the importantstimuli consist of sparse events on either side of the stimulusmedian.

DiscussionThe goal of this study was to illuminate why sensory systemscreate multiple parallel pathways to encode the same signal. Wefocused on one aspect of pathway splitting: the use of ON-typeand OFF-type neurons to signal the same stimulus variable,which is a prominent feature in several sensory modalities acrossthe animal kingdom. Using a range of models for the encoding ofa stimulus, we asked what the functional benefits are of using twoneurons of opposite response polarity (ON–OFF) compared withtwo neurons of the same polarity (ON–ON). The general ap-proach was to set a plausible constraint on the nature of theresponse models, allow each of the two systems to optimize theparameters of the response, and then evaluate their relative per-formance by criteria related to the efficiency of neural coding.

Our analysis shows that the benefit of the ON–OFF architec-ture is not as universal and straightforward as it would initiallyseem. First, we find that the two systems have identical capacityfor information transmission when only the maximal firing rateis constrained (Fig. 3), despite the fact that they tile stimulus

space differently and possess different overall noise statistics (Fig.2). However, the ON–OFF system is more efficient, because itconveys higher information per spike (Fig. 3). In the relevantregime for the retina, with an average of one or a few spikes percoding window (Uzzell and Chichilnisky, 2004; Pillow et al.,2008; Pitkow and Meister, 2012), the ON–OFF system offers anincrease in information about the stimulus of up to 15% or areduction in spikes required for the same information rate of upto 50% under the most conservative conditions of binary re-sponse models and Poisson output noise (Fig. 3). This benefitincreases for more realistic versions informed by neural coding inthe retina to 20 – 40% gain in information and 80% in averagefiring rate in Figures 5 and 6.

If the two neurons share substantial input noise on the com-mon sensory variable, then the benefits of ON–OFF coding in-crease, to 34% enhancement of the maximal information rate, ora 3.4-fold reduction in spikes required for a given informationrate (Fig. 7). In the presence of input noise, the thresholds arepushed to the tails of the stimulus distribution, resulting in theON–ON system being active most of the time. The input noiseforces the system to discriminate between noise and signal; thiseffect is accentuated when the stimulus distribution is sparse,making the tails most informative (Fig. 7). For certain stimulusenvironments, the benefits of ON–OFF coding become dramatic,offering virtually unlimited savings of spikes for a given perfor-mance. In particular, this occurs when the behaviorally relevantstimuli consist of rare but large fluctuations of both polaritiesembedded in a sea of frequent small fluctuations (Fig. 12).

The benefit of ON–OFF splitting is apparent also when opti-mality criteria other than information are used. When we de-mand that the stimulus value be read out by a simple lineardecoder, the ON–OFF system again offers substantial improve-ments, with a maximum gain of 40% in the SNR and a 4-fold to10-fold reduction in the number of spikes required for a givenperformance (Fig. 8). Here again, the advantage of the ON–OFFsystem further increases with the sparseness of the stimulus dis-tribution. The ON–OFF system also achieves better linear decod-ing of dynamic stimuli that include temporal correlations, of thetype encountered by the retina in natural vision (Figs. 9, 10).Finally, we demonstrated that the efficiency gains achieved byON–OFF splitting compare favorably with those that arise fromlateral inhibition, a circuit function widely held to implementdecorrelation and efficient coding of signals in the retina.

Representing the distribution of natural signalsAlthough ON–OFF pathway splitting reaps large benefits for cod-ing large stimulus deviations of either sign, sensory systemsclearly encode small fluctuations in the stimuli as well. For exam-ple, humans can detect visual stimuli with a contrast of �1% (DeValois et al., 1974). One approach to cover the entire range of thenatural stimulus distribution is to use different circuits for en-coding large and small contrasts. For instance, the primate retinaincludes ganglion cell types with very different contrast sensitiv-ity: highly sensitive “phasic” cells, and less sensitive “tonic” cells(Lee et al., 1989). Each of these cell classes includes ON and OFFvarieties, consistent with the present modeling framework. In-deed, one can extend this formalism to cover multiple cell typeswith very different thresholds that span a broad stimulus distri-bution (Gjorgjieva et al., 2014). A second solution encountered inbiological systems is adaptation to contrast, whereby the systemadjusts its thresholds dynamically within the prevailing stimulusdistribution (Smirnakis et al., 1997; Chander and Chichilnisky,

ONB

A

-

-

decorrelationC

ON

ON

OFF

+

+

2

1

25%

26%

Figure 11. Comparing the effects of ON–OFF splitting and decorrelation. A, B, Two neuronsencode almost identical stimulus variables. As shown in the previous figures, ON–OFF codingenhances the transmitted information by �25%. C, Here the two stimulus variables are firstdecorrelated by lateral inhibition to statistical independence and then encoded by two neurons.Under best conditions, this transmits 2 bits, a 26% increase over the log3(2) bits in B.


2001). This contrast adaptation allows a system to maintain theproper choice of thresholds within the stimulus distribution.

In all our modeling, we assumed that large positive and nega-tive stimulus values are equally represented and equally relevant.Is this justified? If one analyzes raw stimulus values—such as thelight intensity in a natural scene or the intensity of naturalsounds—the resulting distributions can be very asymmetric (Ru-derman and Bialek, 1994; Dong and Atick, 1995; van Hateren,1997; Thomson, 1999; Singh and Theunissen, 2003; Geisler,2008; Ratliff et al., 2010). However, there are several aspects ofearly neural processing, universal across nervous systems, thateffectively symmetrize the distribution of the relevant sensorysignals.

Fundamentally, living systems are interested in the time de-rivative of natural signals, because a change in the environmentrequires action, whereas static features do not. A diverse set ofadaptation mechanisms, both cellular and network based, ac-complishes this sensitivity to change. For example, all retinal gan-glion cells studied to date have a biphasic impulse response,characteristic of a differentiator, and the same applies to otheranimal senses (Reyes et al., 1994; Geffen et al., 2009; Kim et al.,2011; Nagel and Wilson, 2011) and even the sensory systems ofbacteria (Block et al., 1982; Segall et al., 1986; Lazova et al., 2011)and fungi (Dennison and Foster, 1977; Foster and Smyth, 1980).After differentiation, the resulting signal is much more balancedin terms of positive and negative changes than the raw stimulusdistribution because of the basic principle of “what goes up mustcome down.” Indeed, if one takes the retinal scenes that humansubjects encounter during natural locomotion or saccadic eyemovements and applies to them the biphasic filter characteristicof retinal processing, one obtains an almost perfectly symmetricaldistribution of signals (Ratliff et al., 2010). Therefore, on a basiclevel, the cross-species universality of time differentiation insensory systems leads to a symmetry between increments anddecrements of the relevant stimulus variables, which in turndrives the universal appearance of ON–OFF coding in earlysensory pathways.

At a more refined level, one often finds asymmetries in thedistribution of natural stimuli that depend on the sensory modal-ity in question. For example, the statistics of photon counts implythat bright and dim lights differ in SNR, and this has been in-voked to explain certain asymmetries between ON and OFF path-ways in the retina (Pandarinath et al., 2010; Nichols et al., 2013).These ON–OFF asymmetries propagate to downstream visualareas, including the thalamus and cortex (Jin et al., 2008; Yeh et

al., 2009; Kremkow et al., 2014), although it remains unknownwhether ON and OFF ganglion cell signals are decoded together.Similarly, natural scenes tend to contain more regions of negativethan of positive spatial contrast (Tadmor and Tolhurst, 2000),and indeed one finds that OFF ganglion cells tend to have smallerreceptive fields and outnumber ON cells (Ratliff et al., 2010).Under an asymmetric stimulus distribution, our efficient codingmodels will also favor asymmetries in the response functions ofON and OFF cells, as observed in the experiment (Chichilniskyand Kalmar, 2002). Of course, pressures other than efficient cod-ing play an important role. For instance, amphibians have astrong preference for hiding in dark spots (Himstedt, 1967; Roth,1987), and this has been invoked to explain the numerical dom-inance of the OFF pathway in their retina. Given that the ON andOFF signals in the retina split already at the first synapse, evolu-tion has been free to sculpt the downstream circuitry in eachpathway to accomplish such fine adjustments to the two halves ofthe stimulus distribution.

Other drivers for ON–OFF codingHere we treated ON–OFF coding as a specific instance of splittinginto multiple sensory pathways and found that it improves theefficiency of neural transmission as measured by mutual infor-mation or by mean squared error of the optimal linear estimator.These two measures are entirely agnostic about the content of thetransmission. They do not take into account that certain stimulicarry greater relevance than others or that downstream areas ofthe nervous system may need the information formatted in acertain way. It seems plausible that an agnostic criterion of thissort provides the evolutionary driving force toward ON–OFFcoding, because this organization emerges in such a disparatecollection of sensory systems that process anything from electricfields to odors. Nevertheless, one can envision other benefits thatresult from an ON–OFF pathway split. In particular, the split maygreatly simplify downstream computations.

Consider the visual processing of moving objects. An objectpassing over a background produces an intensity change of onepolarity at the leading edge and the opposite polarity at the trail-ing edge. Both edges are informative for analyzing the motion ofthe object. To process both edges by a standard motion-detectingcircuit, it would be useful if both polarities were represented inthe same way, namely by excitation of an interneuron with arectified response. In fact, the retina itself already reaps thesebenefits: the ON–OFF direction-selective ganglion cells respondto motion of both bright and dark edges (Barlow et al., 1964).

10−3

10−2

10−1

100

101

Info

rmat

ion

[bits

]

A BON

OFF

ON

ON1

2

Total mean spike count Information [bits]

x 95

C

Rat

io o

f mea

n sp

ike

coun

t

0 0.05 0.1 0.150

20

40

60

80

100

ON-OFF

ON-ON

0

0.04

0.08

0.12

0.16

Figure 12. Sparse stimulus distributions enhance the advantage of ON–OFF splitting. A, Left, A stimulus distribution consisting of three events, s � �1, 0, 1, that occur with differentprobabilities p�1, p0, p1 � p�1, indicated by the bars. Right, Schematic responses of an ON–OFF system (top) and an ON–ON system (bottom) to such a stimulus. B, The mutual information for theON–OFF and ON–ON systems as a function of the total mean spike count. Here the sparseness of the distribution is 1/p1 � 100. Given a desired level of information (horizontal dashed line), theON–ON system needs 95 times higher firing rate than the ON–OFF system for the sparse stimulus in A (compare with Fig. 3). C, The ratio of the overall mean spike count used by the ON–ON versusthe ON–OFF system as a function of the mutual information. The vertical dashed line corresponds to the dashed line in B.


This results from computations in both the outer and the innerplexiform layer using essentially identical circuits of ON- or OFF-bipolar cell terminals, starburst amacrine cells, and retinalganglion cell dendrites (Borst and Euler, 2011). Motion compu-tations in later stages, like the superior colliculus or visual cortex,can benefit similarly from the ON–OFF diversification. If, in-stead, the retina used only sensors of one polarity, then motiondetection for bright edges and dark edges would require the con-struction of two very different circuits.

Another benefit from ON–OFF splitting is that it enablesdownstream computations based on the timing of action poten-tials. Rapid saccades of the eyes break vision up into a sequence ofsnap shots. After a saccade, many ganglion cells fire a brief burstof spikes, and the timing of the first action potential conveys agreat deal of information about the image (Gollisch and Meister,2008). It has been suggested that downstream visual centers cancompute on the image content by combining just one spike perganglion cell, with millisecond sensitivity to spike timing (Delo-rme and Thorpe, 2001; Gutig et al., 2013). This can work only ifthe entire range of image contrasts, including both increases anddecreases, is signaled by a burst of spikes. Thus, any such spike-time computation requires retinal coding by ON and OFF cells.

As one seeks to understand further what motivated the splitinto 20 visual pathways emerging from the retina, the concerns ofefficient coding will likely need to be supplemented by an under-standing of the specific image functions computed in these cir-cuits and what sensorimotor behaviors they support.

ReferencesAla-Laurila P, Greschner M, Chichilnisky EJ, Rieke F (2011) Cone photore-

ceptor contributions to noise and correlations in the retinal output. NatNeurosci 14:1309 –1316. CrossRef Medline

Atick JJ (1992) Could information theory provide an ecological theory ofsensory processing? Network 3:213–251. CrossRef

Atick JJ, Redlich AN (1990) Towards a theory of early visual processing.Neural Comput 2:308 –320. CrossRef

Atick JJ, Redlich AN (1992) What does the retina know about naturalscenes? Neural Comput 4:196 –210. CrossRef

Attneave F (1954) Some informational aspects of visual perception. PhycholRev 61:183–193.

Barlow HB (1961) Possible principles underlying the transformations ofsensory messages. In: Sensory communication, Chap 13, pp 217–234.Cambridge, MA: Massachusetts Institute of Technology.

Barlow HB, Hill RM, Levick WR (1964) Retinal ganglion cells responding toselectively to direction and speed of image motion in the rabbit. J Physiol173:377– 407. Medline

Bell AJ, Sejnowski TJ (1997) The “independent components” of naturalscenes are edge filters. Vision Res 37:3327–3338. CrossRef Medline

Bennett MVL (1971) Electroreception. In: Fish physiology, Vol V (HoarWS, Randall DJ, eds), pp 347– 491. New York: Academic.

Berry MJ, Warland DK, Meister M (1997) The structure and precision ofretinal spike trains. Proc Natl Acad Sci U S A 94:5411–5416. CrossRefMedline

Bethge M, Rotermund D, Pawelzik K (2003) Optimal neural rate codingleads to bimodal firing rate distributions. Network 14:303–319. CrossRefMedline

Block SM, Segall JE, Berg HC (1982) Impulse responses in bacterial che-motaxis. Cell 31:215–226. CrossRef Medline

Borst A, Euler T (2011) Seeing things in motion: models, circuits, andmechanisms. Neuron 71:974 –994. CrossRef Medline

Chalasani SH, Chronis N, Tsunozaki M, Gray JM, Ramot D, Goodman MB,Bargmann CI (2007) Dissecting a circuit for olfactory behaviour in Cae-norhabditis elegans. Nature 450:63–70. CrossRef Medline

Chander D, Chichilnisky EJ (2001) Adaptation to temporal contrast in pri-mate and salamander retina. J Neurosci 21:9904 –9916. Medline

Chichilnisky EJ, Kalmar RS (2002) Functional asymmetries in ON and OFFganglion cells of primate retina. J Neurosci 22:2737–2747. Medline

Delorme A, Thorpe SJ (2001) Face identification using one spike per neu-

ron: resistance to image degradations. Neural Netw 14:795– 803. CrossRefMedline

Dennison DS, Foster KW (1977) Intracellular rotation and the phototropicresponse of Phycomyces. Biophys J 18:103–123. CrossRef Medline

De Valois RL, Morgan H, Snodderly DM (1974) Psychophysical studies ofmonkey vision—III. Spatial luminance contrast sensitivity tests of ma-caque and human observers. Vision Res 14:75– 81. CrossRef Medline

Dong DW, Atick JJ (1995) Statistics of natural time-varying images. Net-work 6:345–358. CrossRef

Eliasmith C, Anderson CH (2002) Neural engineering: computation, repre-sentation, and dynamics in neurobiological systems. Cambridge, MA:Massachusetts Institute of Technology.

Field DJ (1994) What is the goal of sensory coding? Neural Comput 6:559 –601. CrossRef

Foster KW, Smyth RD (1980) Light antennas in phototactic algae. Micro-biol Rev 44:572– 630. Medline

Gallio M, Ofstad TA, Macpherson LJ, Wang JW, Zuker CS (2011) The cod-ing of temperature in the Drosophila brain. Cell 144:614 – 624. CrossRefMedline

Geffen MN, Broome BM, Laurent G, Meister M (2009) Neural encoding ofrapidly fluctuating odors. Neuron 61:570 –586. CrossRef Medline

Geisler WS (2008) Visual perception and the statistical properties of naturalscenes. Annu Rev Psychol 59:167–192. CrossRef Medline

Gjorgjieva J, Meister M, Sompolinsky H (2014) Optimal sensory coding bypopulations of ON and OFF neurons. Presented at the Meeting of Com-putational and Systems Neuroscience, Salt Lake City, UT, February 27 toMarch 2.

Gollisch T, Meister M (2008) Rapid neural coding in the retina with relativespike latencies. Science 319:1108 –1111. CrossRef Medline

Gollisch T, Meister M (2010) Eye smarter than scientists believed: neuralcomputations in circuits of the retina. Neuron 65:150 –164. CrossRefMedline

Gutig R, Gollisch T, Sompolinsky H, Meister M (2013) Computing complexvisual features with retinal spike times. PLoS One 8:e53063. CrossRefMedline

Himstedt W (1967) Experimentelle analyse der optischen sinnesleistungenim beutefangverhalten der einheimischen urodelen. Zool Jahrb Physiol73:281–320.

Jin JZ, Weng C, Yeh CI, Gordon JA, Ruthazer ES, Stryker MP, Swadlow HA,Alonso JM (2008) On and off domains of geniculate afferents in catprimary visual cortex. Nat Neurosci 11:88 –94. CrossRef Medline

Joesch M, Schnell B, Raghu SV, Reiff DF, Borst A (2010) ON and OFFpathways in Drosophila motion vision. Nature 468:300 –304. CrossRefMedline

Kastner D, Baccus SA, Sharpee TO (2014) Second order phase transitiondescribes maximally informative encoding in the retina. Presented at theMeeting of Computational and Systems Neuroscience, Salt Lake City, UT,February 27 to March 2.

Kim AJ, Lazar AA, Slutskiy YB (2011) System identification of Drosophilaolfactory sensory neurons. J Comput Neurosci 30:143–161. CrossRefMedline

Kremkow J, Jin J, Komban SJ, Wang Y, Lashgari R, Li X, Jansen M, Zaidi Q,Alonso JM (2014) Neuronal nonlinearity explains greater visual spatialresolution for darks than lights. Proc Natl Acad Sci U S A 111:3170 –3175.CrossRef Medline

Kuffler SW (1953) Discharge patterns and functional organization of mam-malian retina. J Neurophysiol 16:37– 68. Medline

Lazova MD, Ahmed T, Bellomo D, Stocker R, Shimizu TS (2011) Responserescaling in bacterial chemotaxis. Proc Natl Acad Sci U S A 108:13870 –13875. CrossRef Medline

LeeBB,MartinPR,ValbergA (1989) Sensitivityofmacaqueretinalganglioncells tochromatic and luminance flicker. J Physiol 414:223–243. Medline

Marder E, Abbott LF (1995) Theory in motion. Curr Opin Neurobiol5:832– 840. CrossRef Medline

Masland RH (2012) The neuronal organization of the retina. Neuron 76:266 –280. CrossRef Medline

Nagel KI, Wilson RI (2011) Biophysical mechanisms underlying olfactoryreceptor neuron dynamics. Nat Neurosci 14:208 –216. CrossRef Medline

Nichols Z, Nirenberg S, Victor J (2013) Interacting linear and nonlinearcharacteristics produce population coding asymmetries between ON andOFF cells in the retina. J Neurosci 33:14958 –14973. CrossRef Medline

Nikitin AP, Stocks NG, Morse RP, McDonnell MD (2009) Neural popula-


http://dx.doi.org/10.1038/nn.2927

http://www.ncbi.nlm.nih.gov/pubmed/21926983

http://dx.doi.org/10.1088/0954-898X/3/2/009

http://dx.doi.org/10.1162/neco.1990.2.3.308



http://dx.doi.org/10.1016/S0042-6989(97)00121-1


http://dx.doi.org/10.1073/pnas.94.10.5411


http://dx.doi.org/10.1088/0954-898X/14/2/307


http://dx.doi.org/10.1016/0092-8674(82)90421-4


http://dx.doi.org/10.1016/j.neuron.2011.08.031


http://dx.doi.org/10.1038/nature06292




http://dx.doi.org/10.1016/S0893-6080(01)00049-1


http://dx.doi.org/10.1016/S0006-3495(77)85599-9


http://dx.doi.org/10.1016/0042-6989(74)90118-7


http://dx.doi.org/10.1088/0954-898X/6/3/003



http://dx.doi.org/10.1016/j.cell.2011.01.028




http://dx.doi.org/10.1146/annurev.psych.58.110405.085632


http://dx.doi.org/10.1126/science.1149639




http://dx.doi.org/10.1371/journal.pone.0053063


http://dx.doi.org/10.1038/nn2029




http://dx.doi.org/10.1007/s10827-010-0265-0


http://dx.doi.org/10.1073/pnas.1310442111






http://dx.doi.org/10.1016/0959-4388(95)80113-8






http://dx.doi.org/10.1523/JNEUROSCI.1004-13.2013


tion coding is optimized by discrete tuning curves. Phys Rev Lett 103:138101. CrossRef Medline

Pandarinath C, Victor JD, Nirenberg S (2010) Symmetry breakdown in theON and OFF pathways of the retina at night: functional implications.J Neurosci 30:10006 –10014. CrossRef Medline

Pillow JW, Shlens J, Paninski L, Sher A, Litke AM, Chichilnisky EJ, SimoncelliEP (2008) Spatio-temporal correlations and visual signalling in a com-plete neuronal population. Nature 454:995–999. CrossRef Medline

Pitkow X, Meister M (2012) Decorrelation and efficient coding by retinalganglion cells. Nat Neurosci 15:628 – 635. CrossRef Medline

Pitkow X, Meister M (2014) Neural computation in sensory systems. In:The cognitive neurosciences V (Gazzaniga MS, Mangun GR, eds), pp305–318. Cambridge, MA: Massachusetts Institute of Technology.

Pitkow X, Sompolinsky H, Meister M (2007) A neural computation for vi-sual acuity in the presence of eye movements. PLoS Biol 5:2898 –2911.CrossRef Medline

Ratliff CP, Borghuis BG, Kao YH, Sterling P, Balasubramanian V (2010)Retina is structured to process an excess of darkness in natural scenes.Proc Natl Sci USA 107:17368 –17373. CrossRef

Reyes AD, Rubel EW, Spain WJ (1994) Membrane properties underlyingthe firing of neurons in the avian cochlear nucleus. J Neurosci 14:5352–5364. Medline

Roth G (1987) Visual behavior in salamanders. Berlin: Springer.Ruderman DL (1994) The statistics of natural images. Network 5:517–548.

CrossRefRuderman DL, Bialek W (1994) Statistics of natural images: Scaling in the

woods. Phys Rev Lett 73:814 – 817. CrossRef MedlineSchiller PH (1992) The on and off channels of the visual system. Trends

Neurosci 15:86 –92. CrossRef MedlineSchiller PH, Sandell HH, Maunsell JHR (1986) Functions of the on and off

channels of the visual system. Nature 322:824 – 825. CrossRef MedlineScholl B, Gao X, Wehr M (2010) Nonoverlapping sets of synapses drive On

responses and Off responses in auditory cortex. Neuron 65:412– 421.CrossRef Medline

Segall JE, Block SM, Berg HC (1986) Temporal comparisons in bacterialchemotaxis. Proc Natl Acad Sci U S A 83:8987– 8991. CrossRef Medline

Seung HS, Sompolinsky H (1993) Simple models for reading neuronal populationcodes. Proc Natl Acad Sci U S A 90:10749–10753. CrossRef Medline

Shamai S (1990) Capacity of a pulse amplitude modulated direct detectionphoton channel. IEE Proc Commun Speech Vis 137:424 – 430.

Singh NC, Theunissen FE (2003) Modulation spectra of natural sounds andethological theories of auditory processing. J Acoust Soc Am 114:3394 –3411. CrossRef Medline

Smirnakis SM, Berry MJ, Warland DK, Bialek W, Meister M (1997) Adap-tation of retinal processing to image contrast and spatial scale. Nature386:69 –73. CrossRef Medline

Stein RB (1967) The information capacity of nerve cells using a frequencycode. Biophys J 7:797– 826. CrossRef Medline

Tadmor Y, Tolhurst DJ (2000) Calculating the contrasts that retinal gan-glion cells and LGN neurones encounter in natural scenes. Vision Res40:3145–3157. CrossRef Medline

Thomson MG (1999) Visual coding and the phase structure of naturalscenes. Network 10:123–132. CrossRef Medline

Uzzell VJ, Chichilnisky EJ (2004) Precision of spike trains in primate retinalganglion cells. J Neurophysiol 92:780 –789. CrossRef Medline

van Hateren JH (1992a) Theoretical predictions of spatiotemporal re-ceptive fields of fly LMCs, and experimental validation. J CompPhysiol A Neuroethol Sens Neural Behav Physiol 171:157–170.CrossRef

van Hateren JH (1992b) A theory of maximizing sensory information. BiolCybern 68:23–29. CrossRef Medline

van Hateren JH (1997) Processing of natural time series of intensities by thevisual system of the blowfly. Vision Res 37:3407–3416. CrossRef Medline

Wassle H (2004) Parallel processing in the mammalian retina. Nat Rev Neu-rosci 5:747–757. CrossRef Medline

Wassle H, Boycott BB, Illing RB (1981a) Morphology and mosaic of on- andoff-beta cells in the cat retina and some functional considerations. Proc RSoc Lond B 212:177–195. CrossRef Medline

Wassle H, Peichl L, Boycott BB (1981b) Morphology and topography of on-and off-alpha cells in the cat retina. Proc R Soc Lond B 212:157–175.CrossRef Medline

Westheimer G (2007) The ON–OFF dichotomy in visual processing: fromreceptors to perception. Prog Retin Eye Res 26:636 – 648. Medline

Yeh CI, Xing D, Shapley RM (2009) “black” responses dominate ma-caque primary visual cortex V1. J Neurosci 29:11753–11760. CrossRefMedline


http://dx.doi.org/10.1103/PhysRevLett.103.138101








http://dx.doi.org/10.1371/journal.pbio.0050331




http://dx.doi.org/10.1088/0954-898X/5/4/006

http://dx.doi.org/10.1103/PhysRevLett.73.814


http://dx.doi.org/10.1016/0166-2236(92)90017-3


http://dx.doi.org/10.1038/322824a0








http://dx.doi.org/10.1121/1.1624067


http://dx.doi.org/10.1038/386069a0


http://dx.doi.org/10.1016/S0006-3495(67)86623-2


http://dx.doi.org/10.1016/S0042-6989(00)00166-8


http://dx.doi.org/10.1088/0954-898X/10/2/302


http://dx.doi.org/10.1152/jn.01171.2003


http://dx.doi.org/10.1007/BF00188924

http://dx.doi.org/10.1007/BF00203134


http://dx.doi.org/10.1016/S0042-6989(97)00105-3


http://dx.doi.org/10.1038/nrn1497


http://dx.doi.org/10.1098/rspb.1981.0033


http://dx.doi.org/10.1098/rspb.1981.0032





systems/circuits...

Documents