Mismatch reduction through dendritic nonlinearitiesin a 2D silicon dendritic neuron array
Yingxue Wang and Shih-Chii Liu
Institute of NeuroinformaticsUniversity of Zurich and ETH Zurich
Winterthurerstrasse 190, CH-8057 Zurich, SwitzerlandEmail: yingxue,[email protected]
Abstract— This paper describes a novel 2D programmable den-dritic neuron array consisting of a 3x32 dendritic compartmentarray and a 1x32 somatic compartment array. Each dendriticcompartment contains two types of regenerative nonlinearities:an NMDA nonlinearity and a dendritic spike nonlinearity. Thechip supports the programmability of local synaptic weightsand the configuration of dendritic morphology for individualneurons through the address-event representation protocol. Witha novel local cable circuit between neighboring compartments,different dendritic morphologies can be constructed. From resultsmeasured on a chip fabricated in a 4-metal, 2-poly, 0.35µmCMOS technology, we show one instance of how dendritic non-linearities can contribute to neuronal computation: the dendriticspike mechanism dynamically reduces the mismatch-inducedcoefficient of variation of the somatic response amplitude fromapproximately 40% to 3.5%.
I. INTRODUCTION
Custom aVLSI (analog Very-Large-Scale-Integrated) spik-ing neuronal networks are of great interest because they can beuseful for exploring the role of real-time event-driven brain-like architectures in computation [1]–[3]. Hitherto, most ofthese networks implement point neuron models. However, withaccumulating experimental evidence that a wide variety ofvoltage-dependent ion channels exist in dendrites [4], it isclear that a neuron with dendrites can be computationally morepowerful than a simple point processing unit.
In the early aVLSI neuron designs, dendrites are modelledas passive spatial cables using switched-capacitor circuits [6],[7]. More recently, implementations of dendrites with activeion channels are presented [8]–[10]. Our previous imple-mention of an aVLSI neuron with a two-dimensional (2D)dendritic morphology include dendritic circuits that modeltwo widely studied classes of regenerative event mechanisms:one that is supported by synaptically activated N-methyl-D-aspartate (NMDA) channels, and the second that is mediatedby classical voltage-gated K+, Na+, and Ca2+ channels [11].Using this prototype, we demonstrate that the spatio-temporaldistribution of synaptic inputs can activate a range of pro-cessing modes on the dendrite ranging from linear integrationmode to coincidence detection.
This paper presents a new 2D programmable dendriticneuronal array design. The array is composed of a 2D dendriticcompartment array and an 1D somatic compartment array.
Each dendritic compartment has excitatory and inhibitorysynapses with global bias-controlled synaptic weights, exci-tatory synapses with locally programmable weights throughdigital-to-analog converters (DACs), circuits for two types ofregenerative event mechanisms, and a new local cable circuit.The local cable circuits allow the programmability of con-nections between neighboring compartments to form differentdendritic morphologies. Moreover, the connections betweencertain dendritic compartments and somatic compartments canalso be set so that one can trade-off between the complexity ofthe dendritic structure of a neuron and the number of neuronsin the network. The chip also includes circuits that modelthe backpropagation of somatic action potentials which havebeen observed in the nervous system. All measurements inthis paper are obtained from a chip fabricated in a 4-metal,2-poly, 0.35µm CMOS technology. This paper also describesan experiment which demonstrates how the active dendriticmechanisms can help to reduce mismatch in the circuits.
II. CHIP ARCHITECTURE WITH ACTIVE DENDRITES
The chip layout and architecture are illustrated in Fig. 1.Each dendritic compartment on the chip has 4 excitatoryAMPA synapses, 2 excitatory AMPA-NMDA synapses (inwhich an AMPA synapse is associated with a NMDA synapse),2 inhibitory GABA synapses, a voltage-gated dendritic spikegenerating circuit, and a local cable circuit (Fig. 2). Two ofthe four excitatory AMPA synapses in one compartment have asynaptic weight which is set by a global parameter. Each of theremaining two AMPA synapses includes two 5-bit local DACswhich store the local synaptic weight. Each of the two AMPA-NMDA synapses consists of two components: An AMPAsynapse whose synaptic weight can be programmed through 2global DACs and a NMDA synapse. The GABA synapses arecontrolled by global bias settings. The synaptic componentsand the dendritic spike generation circuit are based on thecurrent-mode low-pass filter (LPF) blocks initially describedin [8].
The fully bidrectional programmable connections betweenneighboring dendritic compartments allow us to constructdendritic structures with up to 96 compartments and of dif-ferent morphologies (Fig. 1(b)). The connections between thedendritic compartments in the last row of the dendritic array
978-1-4244-9472-9/11/$26.00 ©2011 IEEE 677
and the somatic compartments can also be programmed. Inthis design, one can tradeoff between the complexity of thedendritic structure for a neuron and the number of neuronswithin the network depending on the purpose of study. Anetwork can be configured so that it consists of a few neuronswith complex dendritic structures or a larger number ofneurons with simple structures. The backpropagation of actionpotentials from each somatic compartment to the dendriticcompartment directly above it can also be enabled througha programmable connection.
To handle the spike communication between neurons, a2D address-event representation (AER) decoder and encodercircuitry is built into the chip. The dendritic morphology andthe AMPA DACs circuits are also programmed through theAER protocol. The current and voltage values at predefinedcircuit nodes on the chip can be read out through a 2D scanner.The scanner and DAC circuits are previously described in [12].
Dendritic neuron array
Point neuron array
Dendritic compartment
X-DecoderX-Arbiter
Y-D
ecod
erY
-Arb
iter
X-Scanner
Y-S
cann
er
AERBlock DACs
(a)
X-ArbiterX-Arbiter
Y-A
rbite
r
X-Decoder
Y-D
ecod
er
AERBlockAER
BlockReqAck
Synapse Address
Syna
pse
Add
ress
Output Neuron Address
Out
put N
euro
n A
ddre
ss
ReqoutAckout
GlobDACs
(b)
Fig. 1. Architecture of the dendritic neuron array. (a) Layout of thechip shows both the 2D dendritic neuron array described in this paper andan additional 2D point neuron array whose results will not be describedhere. The inset shows the layout of a dendritic compartment with a size of87.8 µm by 395.35 µm in a 4-metal, 2-poly, 0.35µm CMOS technology.(b) Architecture of the dendritic neuron array. This array is composed of3x32 dendritic compartments (C) and 1x32 somatic compartments (S). Theconnections between neighboring dendritic compartments are bidirectionallyprogrammable (in black arrows). The connections between the somaticcompartments and the dendritic compartments directly above can also beprogrammed (in gray arrows). The 2D AER decoder and encoder blocks areused for spike communication and a 2D scanner (not shown) is used to monitorthe responses from predefined circuit nodes.
III. CIRCUIT COMPONENTS
Measurements of the responses and their mismatch variationfrom individual circuit blocks are quantified next. The parame-ters of the circuit responses are purposely tuned to biologicallyplausible values. The detailed equations for the componentscan be found in [8], [11]. These parameters are also used forthe experiment in Section IV.
AMPA Synapse: The three types of AMPA synaptic circuitscorrespond to whether their synaptic weights are biased bya global bias, two local current-mode DACs, or two globalcurrent-mode DACs. The excitatory postsynaptic membraneresponse (EPSP) due to the integration of the synaptic currentis quantified for the 3 different types.
All three types of AMPA synapses are calibrated so thatthey generate similar mean EPSP responses. The mean peakEPSP response for these synapses (τexc = 2.820 V and Wexc =2.156 V) are: 23.41nA (CV= 22.30%) for the AMPA synapsewith global bias; 25.47 nA (CV=24.18%) for the AMPAsynapse with local DACs; and 24.32 nA (CV=22.23%) forthe AMPA synapse with global DACs. For local and globalDAC synapses, data is taken for a DAC value of 62. The CVsare very similar because these circuits are the same except forthe additional DAC components.
NMDA Synapse: The NMDA synapses are activated whenthe local dendritic membrane voltage exceeds a thresholdVThnmda at the time of synaptic stimulation. The second-orderdynamics of the NMDA current In(t) is approximated bytwo first-order current equations that describe the rise and falldynamics separately. The equations are based on the same LPFcircuit (highlighted in dark gray in Fig. 2(b)), whose dynamicsIn(t) in response to an arbitrary input Iin(t) follows:
τndIn(t)
dt= In(t)
(Iin(t)
Iτn− 1
)(1)
where τn = CnUTκIτn
is the time constant of the NMDA current.The NMDA responses are activated and quantified by
stimulating AMPA synapses (see Fig. 3). Averaged across96 NMDA synapses, the peak In current is 19.67 nA(CV=9.53%) with mean rise time constants, τNMDA,rise of8.10ms (CV=2.9%) and fall time constants, τNMDA,decay of15.40 ms (CV= 14.69%). (τnmda = 2.929 V, Wnmda = 2.329V, Rnmda = 0.176 V, and Thnmda = 0.17 V).
Dendritic Spike Circuit: Biological dendritic spikes canbe evoked by voltage-dependent ion channels. A dendriticspike is generated in the aVLSI circuit through two currentswith complementary dynamics: A hyperpolarizing current IH ,which models the response of slowly activating K+ channels,and a depolarizing current ID, which mimics the responseof Na+ or Ca2+ channels. The current ID comes from thesubtraction of IH from a constant current [8]. Similar to theNMDA synapses, a threshold current set by VWdsp controlsthe voltage dependency of the dendritic spike generation(Fig. 2(b)).
The measured mean peak IH is 188.45nA (CV=8.31%),with mean τIH ,rise of 3.6ms and τIH ,decay of 3.7ms. These
678
(a)
NMDA
DENDRITIC MEMBRANE & CABLE
SUMMATION
V
Pulse
L
V
V
ID
D
S
τ
L
λ λ
I
V
V
V
II
I
λλ
R T B
I I I I
H
Hden
S
in
dleak
denden
den den den den
l r t b
τ
W
AMPA
I
CI
exc
exc
τaa
a
3/10
3/10
2/1
3/5 1.2/0.6
3/3
Th
τ
W
I
R
IV M10
CI
M6
M3
M4
M5
M7
M8
M9
pw M1
M2
nmda
n
nmda
nmdaden
nmda
τn
in
n
n n
3/3
3/3
3/0.6
2.5/1.8 2/1
3/5
3/10
3/10
3/5
5/5
3/5
τ
Th
RW
V
V
DEN_SPIKE
C
I
pwd
dsp
dsp dsp
τH
H
H
D
dsp 3/3
3/1
3/3
1.2/0.6
3/53/5
3/5 3/10
3/10
3/10 3/10
3/10
5/55/5
5/5
10/10 10/10 10/10 10/10
5/5
(b)
Fig. 2. Architecture of a dendritic compartment. (a) Components within adendritic compartment: 4 excitatory AMPA synapses, 2 excitatory AMPA-NMDA synapses, 2 inhibitory GABA synapses, a voltage-gated dendriticspike generating circuit, and a local cable circuit. (b) Simplified circuitschematics of the components in a dendritic compartment. The top leftshows the circuit for the AMPA channel. The top right schematic shows theNMDA channel circuit. The middle left schematic shows the circuit for thesummation of currents from individual synapses, dendritic spike current, andneighboring compartments. The middle right schematic shows the dendriticspike generation circuit. When the dendritic membrane potential exceedsthe threshold, a pulse is generated and extended through a pulse extendercircuit (in light gray). The pulse width determines the duration of the risingdynamics of the hyperpolarizing current. The corresponding depolarizingcurrent is obtained by subtracting the hyperpolarizing current from the steady-state currrent, IH (see main text). The bottom schematic shows the circuitfor integrating the currents on the dendritic membrane and for copying theintegrated current to neighboring compartments. The synapse and dendriticmembrane circuits are based on the LFP circuits described in [8]. The sizeof each transistor (W/L) is labelled with the unit of µm/µm.
parameters are first obtained for individual circuits over 20trials, and then averaged across all 96 dendritic spike circuits.
(τdsp = 2.882 V, Wdsp = 0.457 V, Rdsp = 0.219 V and Thdsp= 0.396 V).
Dendritic Membrane Circuit: The dendritic membranecircuit shown in Fig. 2(b) follows the following dynamics:
τdendIden(t)
dt= −Iden(t) +
IdleakIdleak + IH
Iin(t) (2)
where τden = CUTκ(Idleak+IH) . This circuit is based on a prior
circuit in [8]. The dendritic membrane time constant is setto a mean value of 10.4 ms (CV=17.44%) across all 96compartments.
0 20 40 60 80 1000
100
200
300
Den
spI
(
nA)
Time (ms)
0
100
200
I (n
A)
0
10
20
I (n
A)
0
20
40
I (n
A)
an
Hde
n
Fig. 3. Circuit responses of individual dendritic components. From top tobottom, the panels show the response traces of the AMPA synaptic currentIa, NMDA synaptic current In, dendritic spike hyperpolarizing current IH ,and the membrane response of a dendritic spike Iden. Each response tracecorresponds to measurements which are first averaged for individual synapsesover 20 trials for all currents except for In (50 trials). Their mean (in black)and std (in gray) values are then calculated across the same type of circuitsin the dendritic array. The black vertical bar on the top of the figure indicatesthe time of the input stimulation.
IV. EXPERIMENT
The next experiment demonstrates a possible advantageof active nonlinearities in neuronal computation, that is,dynamical mismatch reduction. Mismatch-induced responsevariations will accumulate when the different circuit compo-nents are combined in a multi-compartment neuron. To studythe influence of dendritic spikes on the mismatch inducedresponse variation, the variation is quantified from recordedresponses of neurons with a 3-compartment dendritic branch(Fig. 4) under three different conditions: (1) when both activemechanisms are blocked; (2) when only NMDA channelsare unblocked; and (3) when both active mechanisms areunblocked.
The distal compartment C0 is first stimulated so that nosomatic action potential is generated under conditions (1) and(2). In this case, we can characterize the response variationaccumulated in the course of passive signal propagation fromC0 to the soma. As shown in Table I, the accumulatedmismatch leads to a large variation in both the peak somatic
679
0 100 200Time(ms)
0 100 200Time(ms)
0
150
300
Den
driti
c m
embr
ane
valu
e (n
A)
0
150
300
0 100 2000
150
300
valu
e (n
A)
Time(ms)
0
150
300S
omat
ic m
embr
ane
dt
Only AMPA Unblock NMDA Unblock NMDA and DenSp
Fig. 4. Dynamical variance measurements. The dendritic architecture isprogrammed to form a population of 32 neurons, each with a dendriticbranch consisting of 3 compartments. 4 AMPA synapses and 2 AMPA-NMDAsynapses of either C0 or C2 compartments are stimulated with an inter-spikeinterval dt of 0.1 ms. The figure shows the response traces when C0 isstimulated. The columns represent the three different conditions (see text).Each row corresponds to responses from compartments C0, C1, C2, and thesoma respectively. Each trace plots the mean and std of responses across 32neurons. The parameter settings for results in this figure and Fig. 3 are asfollows: Vth = 0.195 V, Vleak = 2.884 V, Vrefr = 2.761 V.
TABLE IMEASURES QUANTIFYING THE AVERAGE SOMATIC RESPONSE ACROSS 32NEURONS FOR THE EXPERIMENT IN FIG. 4. Vmp AND CVVmp DESCRIBE
THE MEAN AND CV OF PEAK MEMBRANE RESPONSES ACROSS THE
ARRAY, AND δt DESCRIBES THE STANDARD DEVIATION OF THE PEAK
RESPONSE TIME OR RESPONSE JITTER TIME ACROSS THE ARRAY.
Stim Vmp CVVmp δtComp (nA) (%) (ms)
Blocked both C0 8.44 40.06 2.7C2 275.07 9.85 1.7
Unblocked NMDA C0 10.22 56.96 10.1C2 277.53 8.62 1.6
Unblocked both C0 287.15 17.81 1.5C2 306.68 3.53 0.1
membrane response (CVVmp ) and the peak response time (δt).By comparison, under condition (3), the dendritic spike whichis generated within each compartment helps to reduce theaccumulation of the response variation from the earlier stagesthus resulting in smaller CVVmp and δt values.
These results suggest that dendritic spikes can help to reducethe accumulation of response variations across dendritic com-partments thus allowing timing information to be conveyedmore precisely along the dendritic branch even in the presenceof systematic mismatch.
V. CONCLUSION
This paper describes a 2D aVLSI dendritic neuron arraywhich includes circuits that implement two widely studiedclasses of regenerative event mechanisms in biological den-
drites: one mediated by NMDA channels and the other byclassical voltage-gated channels. The chip fabric supports theonline configuration of different dendritic architectures for theneurons through a novel bidirectionally programmable cablecircuit. In addition, the programmability of local synapticweights of AMPA synapses through DACs allows users toperform mismatch compensation or online learning tasks. Thischip offers the possibility to investigate the contribution ofdendritic structure to computation both within a single neuronand a small network of neurons.
An example of how active dendrites can contribute tothe dynamical reduction of system mismatch is shown here.This chip also serves as a simulation platform for exploringother questions regarding neuronal computations, such as thefunction of active dendrites in spatio-temporal processing, andthe computational principles that govern dendritic processing.
VI. ACKNOWLEDGEMENTS
We acknowledge M. Bernet for assistance in initial chiptesting; A. Fco. Jimenez Fernandez, R. Berner, and T. Del-bruck for support with the testing setup. We thank membersof the Institute of Neuroinformatics, particularly A. Whatley,who was involved in the development of the PCI-AER board,of its drivers, and software library components, and D. Muirwho developed the spike tool box. This work was supportedpartially by the ETH Research Grant TH-20/04-2.
REFERENCES
[1] R. Silver, K. Boahen, S. Grillner, N. Kopell, and K. L. Olsen, “Neu-rotech for neuroscience: Unifying concepts, organizing principles, andemerging tools,” Journal of Neuroscience, vol. 27, pp. 11807-11819,2007.
[2] G. Indiveri, S-C. Liu, T. Delbruck, and R. J. Douglas, “Neuromorphicsystems,” Encyclopedia of Neuroscience, L. e. i. c. Squire, AcademicPress: pp. 521-528, 2009.
[3] J. Schemmel, D. Bruderle, A. Grubl, M. Hock, K. Meier, and S. Millner,“A wafer-scale neuromorphic hardware system for large-scale neuralmodeling,” Proceedings of the 2010 IEEE International Symposium onCircuits and Systems (ISCAS), pp. 1947-1950, 2010.
[4] J. C. Magee, in Dendrites, G. Stuart, N. Spruston, M. Hausser, Eds.(Oxford Univ. Press, Oxford), pp. 139-160, 1999.
[5] I. Segev and M. London,“Untangling dendrites with quantitative mod-els,” Science, vol. 290, pp. 744-750, 2000.
[6] C. Rasche and R. J. Douglas, “Forward- and backpropagation in a silicondendrite,” IEEE Trans. on Neural Networks, vol. 12, pp. 386-393, 2001.
[7] J. G. Elias and D. P. M. Northmore, “Building silicon nervous systemswith dendritic tree neuromorphs,” in Pulsed Neural Networks, W. Maassand C. M. Bishop, Eds. Boston, MA: MIT Press, ch. 5, pp. 135-156,1999.
[8] J. V. Arthur and K. A. Boahen, “Recurrently connected silicon neuronswith active dendrites for one-shot learning,” International Joint Confer-ence on Neural Networks (IJCNN), pp. 1699-1704, 2004.
[9] E. Farquhar, D. Abramson, and P. Hasler, “A reconfigurable bidirectionalactive 2 dimensional dendrite model,” Proceedings of the 2004 IEEEInternational Symposium on Circuits and Systems (ISCAS), vol. 5, pp.313-316, 2004.
[10] P. Hasler, S. Kozoil, E. Farquhar, and A. Basu, “Transistor channeldendrites implementing HMM classifiers,” Proceedings of the 2007IEEE International Symposium on Circuits and Systems (ISCAS), pp.3359-3362, 2007
[11] Y-X. Wang and S. C. Liu, “Processing of spatiotemporal patterns inactive dendrites,” Neural Comp., vol. 22, pp 2086-2112, 2010.
[12] M. Oster, Y-X. Wang, R. Douglas, and S. C. Liu, “Quantificationof spike-based winner-take-all VLSI network,” IEEE Transactions ofCircuits and Systems I, vol. 55, pp. 3160-3169, 2008.
680