1

1

Modeling plasticity at a single neuron and network levels

Aušra Saudargienė

Vytautas Magnus University

Lithuania

3nd Baltic-Nordic Summer School on Neuroinformatics, 15-18 June 2015, Tartu, Estonia

2

Overview

• Forms of synaptic plasticity

• Models of spike-timing-dependent synaptic plasticity:

– phenomenological

– optimal

– biophysical

• Application of learning rules:

– modeling brain in health and disease

– neuronal robot control

3

Synaptic plasticity

• Human brain consists of a trillion (1012) neurons and a quadrillion (1015) synapses

• Brain plasticity – the ability of the brain to modify itself and adapt to

changing environment.

http://www.brainhealthandpuzzles.com/brain_plasticity.html

• Synaptic plasticity - modification of the synaptic strength triggered by the neuronal activity

4

Synaptic strength - weight

http://flipper.diff.org/app/items/info/4864

Presynaptically:• Number of synaptic vesicles n• Probability of neurotransmitter release p

Postsynaptically:• Maximal conductance g associated with one synaptic vesicle

Synaptic weight:

w = n p g

Axon terminal

Denritic spine

5

Forms of synaptic plasticity• Short-term synaptic plasticity

– Induction: presynaptic activity, 0.5 sec

– Maintenance: 1 sec

• Long-term synaptic plasticity

– Induction: correlated pre- and postsynaptic activity, 1-3 sec

– Maintenance: several hours

(O’Connor et al., 2005)

• Long-term potentiation (LTP): Hebb’s rule

– LTP is faster: ~1.3 min after the onset of the stimulation protocol

• Long-term depression (LTD): Stent’s rule

– LTD is slower: ~3.1 min after the onset of the stimulation protocol

6

Hebb‘s rule: LTP

When an axon of cell A is near enough to excite cell B and

repeatedly or persistently takes part in firing it, some growth

processes or metabolic change takes place in one or both

cells so that A‘s efficiency ... is increased.

Donald Hebb (1949)

tt

AB

w

A B

2

7

Stent‘s rule: LTD

When the presynaptic axon of cell A repeatedly and

persistently fails to axcite the postsynaptic cell B while cell B

is firing under the influence of other presynaptic axons,

metabolic change takes place in one or both cells so that A‘s

efficiency ... is decreased.

Gunther Stent (1973)

tt

AB

w

A B

8

(Bi&Poo, 2001)

Pre

Post t

Syna

ptic w

eig

h c

ha

ng

e %

Pre

Post

t

t

t

Long-term depression Long-term

potentiation

Post-Pre Pre-Post

Spike-timing-dependent plasticitySTDP

prepostttT −=

0 <−= prepost ttT 0 >−= prepost ttT

• Spike-timing-dependent synaptic plasticity is a form of bidirectional change in synaptic strength that depends on the temporal order and temporal difference of the pre- and postsynaptic activity.

9

EPSP

Pre neuron

t

t

Induction and measuring synaptic modifications

Post neuron

EPSP

3. Assesment: Presynaptic action potential evokes a stronger EPSP

t

Pre neuron

Post neuron

t

t

Pre neuron

Post neuron

2. Induction: Pre- and postsynaptic neurons are activated simultaneously

1. Assessment: Presynaptic action potential evokes EPSP at a postsynaptic site

10

Biophysics of Spike-timing dependent plasticity

• Pairing pre-and postsynaptic activity at 1-5Hz for 60-100sec

• Presynaptic site:– neurotransmitter release

• Postsynaptic site:– NMDAr, AMPAr channel activation

– Somatic action potential, back-propagating spike

– Voltage-gated calcium channel opening

Stuart and Sakmann, 1994

Pre

Post

t

11

Application of STDP in modelling:

healthy brain

• Neuronal response shift earlier in time (Gerstner and Kistler, 2002)

• Formation of receptive fields in place cells (Gerstner and Kistler, 2002)

• Development of direction selectivity in the visual system (Honda, Urakubo, Tanaka, Kuroda, 2011)

• Temporal sequence learning (Izhikevich, 2007)

• Memory encoding and retrieval in hippocampus (Cutsuridis et al., 2010)

w1

w2

w3

w5

x2

x1

x3

x5

y

w4x4

Neuron receives trains of 5 spikes.

Neuron fires after 4 synapses are activated.

t

t

t

t

t

x1

x2

x3

x4

x5

y

Weight decreaseWeight increase

∆w

prepost

T

T

ttT

T eA

0T eATw

−=

<

≥=∆−

+

−

−+

0,

,)(

τ

τ

STDP

Neuronal response before learning

3

Neuronal response after learning

t

t

t

t

t

x1

x2

x3

x4

x5

y

Weight decrease

∆w

w1

w2

w3

w5

x2

x1

x3

x5

y

w4x4

prepost

T

T

ttT

T eA

0T eATw

−=

<

≥=∆−

+

−

−+

0,

,)(

τ

τ

The first synapses generates a response

STDP

Weight

increase

Neuronal response shift earlier in time

Gerstner and Kistler, Spiking Neuron Models. Single Neurons, Populations, Plasticity

Cambridge University Press, 2002

Neuronal response

before learning

Neuronal response

after learning

Neuron is active

31ms

and

58ms

after stimulus

presentation

Neuron is active

5ms

after stimulus

presentation

Neuron is presented

with 20 inputs

15

Application of STDP models

Parkinson’s disease

http://www.ocduk.org/dbs-caution-compassion-advised

• Parkinson’s disease – a neurogenerative disease

characterized by tremors, rigidity, slowness of movement, stiffness.

• Electrical high frequency deep brain stimulation of

the subthalamic nucleus is the effective surgical treatment for Parkinson's disease.

16

Desynchronization of neural populations

• Parkinson’s disease is characterized by abnormally

strong, pathological neuronal synchronization

• Synaptic coupling is strong and has to be unlearned

Popovych V., Tass,P. Desynchronizing electrical and sensory coordinated reset neuromodulation. Front Hum Neurosc 2012

Normal

Pathological

Weak synaptic coupling

Strong synaptic

coupling

17

Unlearning strong synaptic coupling via STDP

• Network of Hodgkin–Huxley neurons and STDP learning rule

• Stimulation causes an unlearning of pathological synchronization

– Weak or Strong stimulation does not have a long-lasting effect

– Intermediate strength stimulation suppresses synaptic coupling and leads to a long- lasting desynchronization

Popovych V., Tass, P. Desynchronizing electrical and sensory coordinated reset

neuromodulation. Front Hum Neurosc 2012

Syna

ptic c

oup

ling

18

Models of Spike-timing Dependent Plasticity Phenomenological – encode data and intuitions

about synaptic plasticity.(Abbott and Song, 1999; Gerstner et al, 1996; Kempter et al, 1999; Kistler et al,

2000; Song et al. 2000; van Rossum, 2000);

Optimal – use some optimality criterion to deduce the rules of synaptic modifications.(Bell and Parra, 2003; Dayan et al, 2004; Pfister et al, 2004; Toyoizumi et al,

2005);

Biophysical – biophysically realistic, usually based on calcium dynamics, involve detailed biochemical reactions to explain synaptic plasticity.(Bhalla and Iyengar, 1999; Karmarkar and Buonomano, 2001; Abarbanel et al, 2002; d’Alcantara et al, 2003; Shouval et al, 2002, 2005; Saudargiene et al, 2004; Rubin et al, 2005; Badoual et al, 2006; Pi and Lisman 2008; Graupner and Brunel 2007, 2012; Clopath et al 2010; Jin et al 2013);

4

19

Phenomenological models

• Black box approach

• No explicit modelling of biophysical processes

• Used in networks – memory formation, neuronal selectivity

(Abbott and Song, 1999; Gerstner et al, 1996; Pfister and Gerstner, 2006;

Kempter et al, 1999; Kistler et al, 2000; Song et al. 2000; van Rossum, 2000)

tpre

tpost

Pre

Post

T

∆ωT

<

>=∆−

+

−

−+

0T ,

0T , )(

τ

τ

T

T

eA

eATw

prepostttT −=

∆ω

T

20

Optimal models• Derive the optimal STDP function for a given task:

– Maximal information transfer in network of spiking neurons – use information theory to obtain STDP curve(Bell and Parra, 2003; Toyoizumi et al, 2005)

– Optimal filter function - STDP shape represents the filter to remove the noise from the signal and increase the influence of its significant components;(Dayan et al, 2004)

– Optimal firing time of the postsynaptic neuron –maximization of the probability of firing at the defined times leads to the STDP function(Pfister et al, 2004)

21

Biophysical models

• White box approach

• Based on Ca2+ dynamics in spine of a postsynaptic

neuron

• Used to model synaptic plasticity at a single synapse, local mircocircuit

(Abarbanel et al, 2002; Badoual et al, 2006; Bhalla and Iyengar, 1999; Bush and Jin, 2012;

Graupner and Brunel, 2007, 2012; d’Alcantara et al, 2003; Karmarkar and Buonomano, 2001;

Rubin et al, 2005; Shouval et al, 2002, 2005; Saudargiene et al, 2004, 2005);

∆ωNeuronal

activity

[Ca2+]

Malenka and Nicoll, 1999

22

Calcium dynamics in spines

NMDAr channels

Ca2+ pump

Voltage gated Ca2+

channels

Ca2+ buffers

Ca2+ diffusion

http://synapses.clm.utexas.edu/atlas/1_4_1_17.stm

23

)(__ Ca

qp

LCaVGCCCa EVhmgI −=

)(1.0_ NMDANMDANMDACa EVgI −=

Calcium current through voltage-gated calcium channels:

Calcium current through NMDA receptor-gated channels:

Calcium concentration:

Ca

Ca CaCa

Fv

I

dt

Cad

τ][][

2

][ 2

0

2 2 ++ −+−=

+

Decay due to pumping, buffering, diffusion

Calcium concentration

24

Post - Pre

LTD

Pre – Post

LTP

T=-20Post

Pre T

T=20Post

PreT

[Ca2+],

µM

[Ca2+],

µM

t, mst, ms

Amplitude and time course of Ca2+ transient depend on the temporal order and temporal difference of pre- and postsynaptic activity.

LTP threshold θp

Ca2+ transientsSTDP stimulation protocol

LTD threshold θd

5

25

• Weight change is determined by [Ca2+] levels:

– low [Ca2+] levels – no changes

– intermediate [Ca2+] levels between θd and θp – LTD

– high [Ca2+] levels above θp – LTP

Biophysical models

Calcium control hypothesisShouval et al, 2002, 2005

Graupner and Brunel, 2010 26

• Weight change is proportional to the threshold function Ω([Ca]):

• Plasticity outcomes:

– short positive T – LTP

– short negative and large positive T – LTD

Biophysical models

Calcium control hypothesis

])([ Caw Ω≅∆ η

Shouval et al, 2002, 2005

Ω([

Ca

])

27

Kandel, Schwartz, Jessell, 2000

Biochemical mechanisms of

Spike-timing dependent plasticity

Ca2+

Phosphatases Ca2+/CaM KinaseII

Phosporylation

of AMPA receptors

Dephosporylation

of AMPA receptors

Long-term

potentiation

Long-term

depression

• Correlated activity of pre- and postsynaptic

neurons• Glutamate release

• Activation of NMDAr-gated channels

• Increase in calcium concentration in a spine

28

Biophysical models

LTP and LTD enzyme competitionBadoual et al, 2005

LTD enzyme Ph

*2

m

b

aCam

m

m++

*

h

b

aGluh

h

h+

***

hh

PPhm ++

m activated by Ca2+

h activated by GluPh activated by m* and h*

LTP enzyme K

*2

4 K

b

aCaK

k

k++

Reflects activity of

Ca2+/calmodulin-dependent protein kinase II, activated by Ca2+

** KPw h +−=∆

Ca2+

29

Binary synapse model

• Synaptic changes are all-or-none switch-like events

• Two stable states of synaptic transmission efficacy at resting calcium levels:– DOWN, low synaptic strength

– UP, high synaptic strength

• Transitions:– UP-DOWN: LTD

– DOWN-UP: LTP

Graupner and Brunel, 2010

DOWN → UP UP → DOWN

30

Ca2+/calmodulin (CaM)-dependent

protein kinase CaMKII• CaMKII holoenzyme consists of 6 subunits.

• Each subunit can be in:– active, phosphorylated state;

– not active, dephosphorylated state.

• CaMKII can be stable in two states at resting calcium: – weakly phosphorylated state - DOWN

– highly phosphorylated state - UP

• CaMKII switch in a binary synapse:– Transition UP-DOWN: LTD

– Transition DOWN-UP: LTP

Lisman, Schulman & Cline, 2002

6

31

Bistable synapse model based on

CaMKII activation network• Ca2+/calmodulin C activates CaMKII

• PP1 inhibits, dephosphorylates CaMKII

– PP1 influence is increased by calcineurin activation

– PP1 influence is reduced by cyclic-AMP and PKA activation

Graupner and Brunel, 2008

14 differential equations! 32

• Model of CaMKII regulation, mGlu receptor and NMDA

receptor activation

Bhalla and Iyengar, 1999

Biochemical networks of synaptic plasticity: more details

33

Multiple forms of STDP

• STDP depends on:

• Stimulation protocol– Pattern of post activity (single spike, burst)

– Frequency of pairings

– Duration of stimulation

• Properties of neuron – Dendritic plasticity

– Dendritic synapse location

• Neuron type

• Brain region

34

STDP timing windows

Buchannan and Mellor, 2010 Abbott and Nelson, 2000

Hippocampus, excitatory

synapse

Hippocampus, inhibitory synapse

Neocortex,

excitatory synapse

35

STDP model based on kinase and

phosphatase competition

Graupner and Brunel, 2012

• Recent models are able to account for the effect of spike pattern, frequency, duration of stimulation.

• Synaptic efficacy ρ

)(])([])([)1())(1( * tNoisetCatCadt

dddpp +−Θ−−Θ−+−−−= θργθργρρρρ

ρτ

Kinase influence

[Ca2+]>Θp

Phosphotase influence

[Ca2+]>Θd

Absence of

activity

• Synaptic efficacy ρ: two stable states UP and DOWN– transition UP-DOWN: LTD

– transition DOWN-UP: LTP

LTD termLTP term

36

Diversity of STDP curves

Graupner and Brunel, 2012

• Numbers of postsynaptic spikes and duration of the stimulation

qualitatively change the STDP curve.

7

37

NEURON demo Modeling learning in hippocampus• NEURON simulator

• STDP model: synaptic efficacy variables ρUP, ρDOWN

• Associative learning in a CA1 microcircuit

ρUP

ρDOWN

38

Models of LTP and LTD: review

• 117 models analyzed

• Models of molecular mechanisms underlying synaptic plasticity are classified:

– Direction: LTP/LTD

– Complexity: single pathways/simplified processes/signaling networks

– Method: deterministic/stochastic method

– Cell type

39

Neuromodulation of STDP

Markram H , Gerstner W and Sjöström PJ. A History of Spike-timing-dependent Plasticity Synaptic

Neuroscience 2011Pawlak V, Wickens JR, Kirkwood A and Kerr JND. Timing is not everything: neuromodulation opens the

STDP gate. Frontiers in Synaptic Neuroscience 2010.

• STDP depends on:– pre- and postsynaptic activity

– a “third factor” – neuromodulators

dopamine, acetylcholine, etc

• Neuromodulators open the STDP gate:– necessary for plasticity induction or

– change the shape of the STDP

• Time and spatial scale:– Correlated pre-post ativity, STDP – local and fast, ms

– Neuromodulation – global and slow, 3-5sec

40

Dopamine pathways• Dopamine is involved in:

– reward-motivated behavior

– pleasure

– motor control

• Dopamine signal mimics reward prediction error (reinforcement learning theory)

http://www.drugabuse.gov/publications/addiction-science/why-do-people-abuse-drugs/brain-pathways-are-affected-by-drugs-abusehttp://www.drugabuse.gov/publications/addiction-science/why-do-people-abuse-drugs/brain-pathways-are-affected-by-drugs-abuse

41

Dopamine controls STDP in hippocampal neurons

Pawlak et al, 2014

• Dopamine changes the shape of

the STDP window.

• LTP window is wider

• LTD converts into LTP

• Dopamine reduces the number of spike pairs required to induce LTP

from 60 to 10.

Izhikevich E. Solving the distal reward problem through linkage of STDP and dopamine signal. Cerebral

Cortex 2007 42

How do molecules affect

behaviour?

http://molinterv.aspetjournals.org/cgi/content/abstract/3/7/375

http://www.cartoonstock.com/newscartoons/directory/b/behaviour.asp

8

43

Application of STDP-like learning

rule in robotics

Porr, B. and Wörgötter, F. (2003) Isotropic sequence order learning. Neural Comp., 15, 831-864.

Manoonpong, P., Porr, B. and Woergoetter, F. (2007) Adaptive, Fast Walking in a Biped Robot under Neuronal Comtrol and learning. Comp Biology, 3(7),1305-1311.

RunBot - a biped walking robot –

learns to walk on a terrain

Vision

sensorsCollision

sensors

Robot learns to navigate without

collisions

44

collision x0

B.Porr, F.Wörgötter 2003Learning in a navigating robot

yxdt

dwvision

′= µ

Differential Hebbian learning:

T

∆ω

collision x0

vision x1

T

vision x1

T

T<0 T>0

STDP-like weight change

45

Final remarks• Phenomenological models apply the STDP function and

analyze the computational principles and its functional consequences in networks.

• Optimal models derive optimal STDP function for a given task – optimal firing time, maximal information transfer between

neurons, optimal filter function.

• Biophysical models explain STDP using the principles of ion channel dynamics and intracellular processes. Usually contains a large parameter space.

46

Final remarks

• Neuromudulation gates STDP and provides link from correlation-based synaptic plasticity rules to behaviorally

based learning.

• Link between the cellular processes and behaviour is hard to establish. Robotics offer one of the possibilities to analyze the functional consequences of the learning rules.

• Models contribute to understanding of brain functioning in health and disease.

47

Final remarks

• Modeling synaptic plasticity:

– from molecules (nm) to a system level (m);

– from fast electrical events (ms) to slow chemical processes (min, hours);

– from unsupervised learning to reinforcement learning (reward-based).

https://www.humanbrainproject.eu/documents/10180/17648/TheHBPReport_LR.pdf/18e5747e-10af-4bec-9806-d03aead57655

48

Acknowledgement

• Marja-Leena Linne

• Bruce Graham

• Arnd Roth

• Florentin Worgotter

• Bernd Porr