neuromorphic vlsi event-based devices and systems/file/giacomo_indiveri.pdf · 2012-06-27 ·...

Neuromorphic VLSI Event-Based devices and systems

Giacomo Indiveri

Institute of NeuroinformaticsUniversity of Zurich and ETH Zurich

LTU, LuleaMay 28, 2012

G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 1 / 38

Outline

1 “Neuromorphic Engineering”

2 Spike-based sensory systems

3 Spiking Neural NetworksSilicon NeuronsSilicon synapsesWinner-Take-All networksMulti-chip networks

4 Learning

5 Neuromorphic Cognitive Systems

Natural ComputationThe Honeybee

Energy consumption: 10−15 J/op, at least 106

more efficient than digital silicon(20watts vs. 1Mil.watts)

The brain of the workerhoneybee occupies a volumeof around 1mm3 and weighsabout 1mg.The total number of neurons inthe brain is estimated to be950,000

Flies acrobatically

Recognizes patterns

Navigates

Forages

Communicates

Natural ComputationThe Honeybee

Energy consumption: 10−15 J/op, at least 106

more efficient than digital silicon(20watts vs. 1Mil.watts)

The brain of the workerhoneybee occupies a volumeof around 1mm3 and weighsabout 1mg.The total number of neurons inthe brain is estimated to be950,000

Flies acrobatically

Recognizes patterns

Navigates

Forages

Communicates

Neocortex → Neural computation → Silicon→ Behavior

Synaptic Inputs Constant current Synapse

0 0.05 0.1 0.15 0.20

Time (s)

Neuromorphic VLSI systems

[Nuno da Costa, INI, 2008]

Goals:

To exploit the physics of silicon to reproduce the bio-physics of neuralsystems, using subthreshold analog VLSI circuits.

To develop multi-chip spike-based computing systems, using theAddress-Event Representation (AER) and asynchronous digital VLSItechnology.

To automatically configure and “program” neuromorphic processingsystems distributed across multiple chips, to carry out real–timebehavioral tasks.

Goals:

Outline

4 Learning

AER silicon retinasTobi Delbruck

Silicon retina propertieshttp://siliconretina.ini.uzh.ch

(movie)G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 8 / 38

An AER silicon cochleaShih-Chii Liu

Silicon cochlea properties

Outline

4 Learning

Implement neural computation in silicon

Classical neural networks

Post-synaptic Output

Neuromorphic multi-neuron networks

w2 w3 w4 wnw1

Pre-synaptic Inputs

Spiking multi-neuron architectures

Networks of silicon neurons with adaptation,refractory period, etc.

Silicon synapses with realistic temporal dynamics

Winner-Take-All architectures

Spike-based plasticity mechanisms

Outline

4 Learning

Silicon neuronsThe low-power adaptive exponential I&F neuron

Positive Feedback

Refractory Period

Adaptation

VrestVthra

Imem + Imem ≈Ig IinIτ

+ f (Imem)

[Indiveri et al., ISCAS 2010]

The low-power I&F neuronPositive Feedback

0 5 10 15 20 25 30 351

Time (ms)

The low-power I&F neuronSpike frequency adaptation

0 5 10 15 202

Spike count

0 1 2 3 4 50

The low-power I&F neuronBasic response properties

Leaky I&F model

0 0.05 0.1 0.15 0.20

Time (s)

F-F curve (note mismatch)

Outline

4 Learning

Synapses

Real synapses Artificial synapses

Synapses are often modeled asinstantaneous multipliers.

Science and Engineering Visualization Challenge

2005 winner, Graham Johnson, Medical Media, Boulder,

Colorado.

The diff-pair integrator (DPI) circuit

Min Mthr

Isyn(t) = I0e−κ

UT(Vsyn(t)−Vdd )

Ithr = I0e−κ(Vthr−Vdd )

Csynddt

Vsyn =−(Iin− Iτ)

Isyn + Isyn =Ithr Iw

[Bartolozzi and Indiveri, Neural Computation, 2007]

The DPI synapseTemporal dynamics

0 0.5 1 1.5 2 2.5 3 3.50

Time (s)

=300mV

=320mV

=340mV

Outline

4 Learning

Recurrent cooperative-competitive architectures

Hardwired localsynapses

Local excitatoryconnections

Global inhibitoryconnections

[Chicca et al., Nips, 2006]

Local recurrent connectivityWinner-take-All architectures

0 2 4 6 8 100

Time (s)

Input Stimulus

0 50 100Mean f (Hz)

AER INPUT Y

Input signals are encodedwith mean firing rates

Computation andinformation transfer isdata driven

0 2 4 6 8 100

Time (s)

Feedforward Network

0 20 40Mean f (Hz)

Without local connectivityactivated output spike ratesrepresent linearly input spikerates (modulo mismatcheffects)

0 2 4 6 8 100

Time (s)

Feedback Network

0 20 40Mean f (Hz)

With local WTA connectivity thenetwork exhibits:

Selective amplification

Signal normalization

Signal restoration

[Chicca et al., Nips, 2006]

Outline

4 Learning

Spikes and Address-Event Systems

1 2 3 2 3 12

Inputs

Encode Decode

Address Event Bus

SourceChip

Outputs

DestinationChip

Action Potential

Address-Eventrepresentation ofaction potential

0 0.05 0.1 0.15 0.20

Time (s)

Hierarchical or multi-layer networks

The basic problem with these models is, of course, generalization:a look-up table cannot deal with new events, such as viewing a facefrom the side rather than the front, and it cannot learn in the predic-tive sense described earlier. One of the simplest and most powerfultypes of algorithm developed within learning theory corresponds tonetworks that combine the activities of ‘units’, each broadly tuned toone of the examples (Box 1). Theory (see references in Box 1) showsthat a combination of broadly tuned neurons — those that respondto a variety of stimuli, although at sub-maximal firing rates — mightgeneralize well by interpolating among the examples.

In visual cortex, neurons with a bell-shaped tuning are common.Circuits in infratemporal cortex and prefrontal cortex, which com-bine activities of neurons in infratemporal cortex tuned to differentobjects (and object parts) with weights learned from experience, mayunderlie several recognition tasks, including identification andcategorization. Computer models have shown the plausibility of thisscheme for visual recognition and its quantitative consistency withmany data from physiology and psychophysics2–5 .

Figure 2 sketches one such quantitative model, and summarizes aset of basic facts about cortical mechanisms of recognition establishedover the last decade by several physiological studies of cortex6–8. Objectrecognition in cortex is thought to be mediated by the ventral visualpathway running from primary visual cortex, V1, over extrastriatevisual areas V2 and V4 to the inferotemporal cortex. Starting fromsimple cells in V1, with small receptive fields that respond preferably tooriented bars, neurons along the ventral stream show an increase inreceptive field size as well as in the complexity of their preferred stimuli.At the top of the ventral stream, in the anterior inferotemporal cortex,neurons respond optimally to complex stimuli such as faces and otherobjects. The tuning of the neurons in anterior inferotemporal cortexprobably depends on visual experience9–19. In addition, some neuronsshow specificity for a certain object view or lighting condition13,18,20–22.For example, Logothetis et al.13 trained monkeys to perform an objectrecognition task with isolated views of novel three-dimensional objects(‘paperclips’; Fig. 1). When recording from the animals' inferotemporalcortex, they found that the great majority of neurons selectively tunedto the training objects were view-tuned (see Fig. 1) to one of the trainingobjects. About one tenth of the tuned neurons were view-invariant,consistent with an earlier computational hypothesis23.

In summary, the accumulated evidence points to a visual recog-nition system in which: (1) the tuning of infratemporal cortex cells isobtained through a hierarchy of cortical stages that successivelycombines responses from neurons tuned to simpler features; and (2)the basic ability to generalize depends on the combination of cellstuned by visual experience. Notice that in the model of Fig. 2, thetuning of the units depends on learning, probably unsupervised (forwhich several models have been suggested24; see also review in thisissue by Abbott and Regehr, page 796), since it depends only onpassive experience of the visual inputs. However, the weights of thecombination (see Fig. 3) depend on learning the task and require atleast some feedback (see Box 2).

Thus, generalization in the brain can emerge from the linear com-bination of neurons tuned to an optimal stimulus — effectivelydefined by multiple dimensions25,23,26. This is a powerful extension ofthe older computation-through-memory models of vision andmotor control. The question now is whether the available evidencesupports the existence of a similar architecture underlying general-ization in domains other than vision.

insight review articles

Figure 1 Tuned units in inferotemporal cortex. A monkey was trained to recognizea three-dimensional ‘paperclip’ from all viewpoints (pictured at top). The graphshows tuning to the multiple parameters characterizing each view summarized interms of spike rate versus rotation angle of three neurons in anterior inferotemporalcortex that are view-tuned for the specific paperclip. (The unit corresponding to thegreen tuning curve has two peaks — to a view of the object and its mirror view.) Acombination of such view-tuned neurons (Fig. 2) can provide view-invariant, objectspecific tuning as found in a small fraction of the recorded neurons. Adapted fromLogothetis et al.13.

–80 –30 –0 60 100

–180 –120 –60 0 60 120 180

Rotation around y axis (degrees)

Figure 2 A model of visual learning. The model summarizes in quantitative termsother models and many data about visual recognition in the ventral stream pathwayin cortex. The correspondence between the layers in the model and visual areas isan oversimplification. Circles represent neurons and arrows represent connectionsbetween them; the dots signify other neurons of the same type. Stages of neuronswith bell-shaped tuning (with black arrow inputs), that provide example-basedlearning and generalization, are interleaved with stages that perform a max-likeoperation3 (denoted by red dashed arrows), which provides invariance to positionand scale. An experimental example of the tuning postulated for the cells in thelayer labelled inferotemporal in the model is shown in Fig. 1. The model accountswell for the quantitative data measured in view-tuned inferotemporal cortex cells10

(J. Pauls, personal communication) and for other experiments55. Superposition ofgaussian-like units provides generalization to three-dimensional rotations andtogether with the soft-max stages some invariance to scale and position. IT,infratemporal cortex, AIT, anterior IT; PIT, posterior IT; PFC, prefrontal cortex.Adapted from M. Riesenhuber, personal communication.

Categ. Ident.

V4/PIT

NATURE | VOL 431 | 14 OCTOBER 2004 | www.nature.com/nature 769

14.10 Insight 768 Poggio 1/10/04 7:48 pm Page 769

AER INPUT Y

Hierarchical or multi-layer networks

The basic problem with these models is, of course, generalization:a look-up table cannot deal with new events, such as viewing a facefrom the side rather than the front, and it cannot learn in the predic-tive sense described earlier. One of the simplest and most powerfultypes of algorithm developed within learning theory corresponds tonetworks that combine the activities of ‘units’, each broadly tuned toone of the examples (Box 1). Theory (see references in Box 1) showsthat a combination of broadly tuned neurons — those that respondto a variety of stimuli, although at sub-maximal firing rates — mightgeneralize well by interpolating among the examples.

In visual cortex, neurons with a bell-shaped tuning are common.Circuits in infratemporal cortex and prefrontal cortex, which com-bine activities of neurons in infratemporal cortex tuned to differentobjects (and object parts) with weights learned from experience, mayunderlie several recognition tasks, including identification andcategorization. Computer models have shown the plausibility of thisscheme for visual recognition and its quantitative consistency withmany data from physiology and psychophysics2–5 .

Figure 2 sketches one such quantitative model, and summarizes aset of basic facts about cortical mechanisms of recognition establishedover the last decade by several physiological studies of cortex6–8. Objectrecognition in cortex is thought to be mediated by the ventral visualpathway running from primary visual cortex, V1, over extrastriatevisual areas V2 and V4 to the inferotemporal cortex. Starting fromsimple cells in V1, with small receptive fields that respond preferably tooriented bars, neurons along the ventral stream show an increase inreceptive field size as well as in the complexity of their preferred stimuli.At the top of the ventral stream, in the anterior inferotemporal cortex,neurons respond optimally to complex stimuli such as faces and otherobjects. The tuning of the neurons in anterior inferotemporal cortexprobably depends on visual experience9–19. In addition, some neuronsshow specificity for a certain object view or lighting condition13,18,20–22.For example, Logothetis et al.13 trained monkeys to perform an objectrecognition task with isolated views of novel three-dimensional objects(‘paperclips’; Fig. 1). When recording from the animals' inferotemporalcortex, they found that the great majority of neurons selectively tunedto the training objects were view-tuned (see Fig. 1) to one of the trainingobjects. About one tenth of the tuned neurons were view-invariant,consistent with an earlier computational hypothesis23.

In summary, the accumulated evidence points to a visual recog-nition system in which: (1) the tuning of infratemporal cortex cells isobtained through a hierarchy of cortical stages that successivelycombines responses from neurons tuned to simpler features; and (2)the basic ability to generalize depends on the combination of cellstuned by visual experience. Notice that in the model of Fig. 2, thetuning of the units depends on learning, probably unsupervised (forwhich several models have been suggested24; see also review in thisissue by Abbott and Regehr, page 796), since it depends only onpassive experience of the visual inputs. However, the weights of thecombination (see Fig. 3) depend on learning the task and require atleast some feedback (see Box 2).

Thus, generalization in the brain can emerge from the linear com-bination of neurons tuned to an optimal stimulus — effectivelydefined by multiple dimensions25,23,26. This is a powerful extension ofthe older computation-through-memory models of vision andmotor control. The question now is whether the available evidencesupports the existence of a similar architecture underlying general-ization in domains other than vision.

insight review articles

Figure 1 Tuned units in inferotemporal cortex. A monkey was trained to recognizea three-dimensional ‘paperclip’ from all viewpoints (pictured at top). The graphshows tuning to the multiple parameters characterizing each view summarized interms of spike rate versus rotation angle of three neurons in anterior inferotemporalcortex that are view-tuned for the specific paperclip. (The unit corresponding to thegreen tuning curve has two peaks — to a view of the object and its mirror view.) Acombination of such view-tuned neurons (Fig. 2) can provide view-invariant, objectspecific tuning as found in a small fraction of the recorded neurons. Adapted fromLogothetis et al.13.

–80 –30 –0 60 100

–180 –120 –60 0 60 120 180

Rotation around y axis (degrees)

Figure 2 A model of visual learning. The model summarizes in quantitative termsother models and many data about visual recognition in the ventral stream pathwayin cortex. The correspondence between the layers in the model and visual areas isan oversimplification. Circles represent neurons and arrows represent connectionsbetween them; the dots signify other neurons of the same type. Stages of neuronswith bell-shaped tuning (with black arrow inputs), that provide example-basedlearning and generalization, are interleaved with stages that perform a max-likeoperation3 (denoted by red dashed arrows), which provides invariance to positionand scale. An experimental example of the tuning postulated for the cells in thelayer labelled inferotemporal in the model is shown in Fig. 1. The model accountswell for the quantitative data measured in view-tuned inferotemporal cortex cells10

(J. Pauls, personal communication) and for other experiments55. Superposition ofgaussian-like units provides generalization to three-dimensional rotations andtogether with the soft-max stages some invariance to scale and position. IT,infratemporal cortex, AIT, anterior IT; PIT, posterior IT; PFC, prefrontal cortex.Adapted from M. Riesenhuber, personal communication.

Categ. Ident.

V4/PIT

NATURE | VOL 431 | 14 OCTOBER 2004 | www.nature.com/nature 769

14.10 Insight 768 Poggio 1/10/04 7:48 pm Page 769

AER INPUT Y

Outline

4 Learning

Spike-timing dependent plasticity (STDP)

Abbot, Nelson, 2000

1 If an input (pre-synaptic) spike arrives shortlybefore an output (post-synaptic) spike is emitted,the synaptic efficacy is increased.

2 If it arrives soon after the output spike is emitted,the synaptic efficacy is decreased.

STDP is not enoughfor learning complex spatio-temporal patterns

Senn, Biological Cybernetics, 2002[...] additional non linearities are required ifSTDP should be relevant for both encodinginformation represented in a spike correlationcode and a mean rate code without spikecorrelations.

STDP is not enoughfor learning complex spatio-temporal patterns

Spike-based learning mechanisms idealfor VLSI implementations

depend on the neuron’s membranepotential;

synaptic weights have two stable states(bi-stability);

many synapses see the same pre- andpost-synaptic mean activity (redundancy);

LTP/LTD is induced only in a randomsubset of stimulated synapses(stochasticity).

[Fusi et al. 2000]; [Gütig, Sompolinsky 2006]; [Brader et al. 2007]

Spike-driven plasticity in silicon

w2 w3 w4 wnw1

Pre-synaptic Inputs

Pre-synaptic component

Diff-pairIntergator

AER inputspike

V’UP

V’DN

Post-synaptic component

Comparator

I&F CircuitDiff-pair

IntegratorVCa

CurrentComparator

Spike-driven plasticity in silicon

0 0.1 0.2 0.3 0.4

Time(s)

[Mitra et al. 2009]

Stochastic weight updateLTP/LTD probabilities and stop-learning

LTD consolidation

0 0.1 0.2 0.3 0.4

Time(s)

No LTD consolidation

0 0.1 0.2 0.3 0.4

Time(s)

Outline

4 Learning

Distributed event-driven systems

Neuromorphic “cognitive” systems can be assembled by using:1 Full custom hybrid analog/digital neural processing VLSI devices.2 A spike based communication protocol (e.g., the Address-Event

Representation).3 Systematic methods for parameter tuning.4 Methods for implementing state-dependent computation using spiking

neural networks.

State dependent computation

sWTA diagramExcitatory

neurons

Global

Inhibition

Inhibitory

neurons }Nearest-N

Excitation

I E E E E E

sWTA networks as building blocks

Linearbehaviors

Non linearbehaviors

Analog gain Locus invariance Gain control bycommon mode input

Selective amplification Signal restoration Multi-stability

[Douglas and Martin, 2007]

Configure key parameters of the WTA network automatically.Implement “state-holding” elements.Learn network connectivity patterns.

ConclusionsToward neuromorphic cognitive behaving systems

By using event based sensors andspike-based neural processingcircuits it is possible to implementreal-time sensory-motor systems.

By using soft WTA multi-chipnetworks it is possible toimplement real-timestate-dependent computation.

The AER communicationinfrastructure and automatedparameter tuning techniques,allow us to synthesizespike-based neural finite statemachines.

Acknowledgments

The Institute of NeuroinformaticsElisabetta ChiccaStefano FusiChiara BartolozziThe NCS group (http://ncs.ethz.ch/)

Rodney DouglasKevan MartinRichard Hahnloser

Funding sourcesneuroP (257219) ERCSCANDLE (ICT-231168)eMorph (ICT-231467)

nAttention (121713) SNFSoundRec (119973) SNF

neuromorphic vlsi event-based devices and systems/file/giacomo_indiveri.pdf · 2012-06-27 ·...

Documents

project concepts presentation enee 719v-2002 neuromorphic...

neuromorphic electronics - uio

cooperative competitive neural networks a …...diss. eth...

a neuromorphic vlsi head direction cell system

neuromorphic circuits technical report

a wafer-scale neuromorphic hardware system for … · a...

mixed-signal neuromorphic vlsi devices for spiking neural...

low power neuromorphic analog system based on sub … ·...

executionmodel: neuromorphic computation

a neuromorphic vlsi system for modeling the neural · pdf...

synthesizing cognition in neuromorphic electronic...

neuromorphic analogue vlsi - caltechauthorsthe efficiency of...

neuromorphic computing grollier.pdf

spike processing circuit design for neuromorphic...

abstract dissertation: a neuromorphic vlsi navigation

2cm emergent auditory feature tuning in a real-time...

a survey of neuromorphic computing and neural - arxiv · a...

illustrated assembly manual h8015ip-2 - … assembly manual...

danna a neuromorphic computing vlsi...

neuromorphic vlsi models of selective attention: from