learning crossmodal spatial transformations through stdp gerhard neumann seminar b, ss 06
TRANSCRIPT
Learning crossmodal spatial transformations
through STDP
Gerhard Neumann
Seminar B, SS 06
Overview Network Model Hebbian Learning and STDP Properties of STDP Cortical Maps Learning Spatial Transformations Papers:
[Song99] : S. Song, L. Abbott, Competitive Hebbian Learning through Spike-Timing-Dependent Plasticity
[Song00] : S. Song, L. Abbott, Cortical Development and Remapping through Spike Timing-Dependent Plasticity
[Davison06]: A. Davison and Y.Fregnac, „Learning Crossmodal spatial transformations through spike-timing-dependent plasticity.
Leaky Integrate + Fire Model
Membrane Potential Vj of neuron j:
Input consists of: Background noise Excitatory Input (added): Inhibatory Input (substracted):
Neuron Model Excitatory Input:
Incremented by following each spike Inhibatory Input:
incremented by by every spike
Simplified Version: Direct change in synaptic current More Complex Version
Conductance Based IF Neurons used by Abbott Basically the same results
Hebbian Learning Donald Hebb:
When an axon of cell A is near enough to excite cell B or repeatedly or consistently takes part in firing it, some growth or metabolic change takes place in one or both cells such that A’s efficiency, as one of the cells firing B, is increased.
Correlation based learning
Not a stable rule: Weight normalization needed No Competition
Usually we need a „global competition signal“ Not biologically realistic
Only for feed forward networks: No recurrent connections possible
)()()( tOtOtW jiij
Spike-timing-dependent plasticity
Synaptic plasticity is sensitive to the temporal oder of the presynaptic and postsynaptic spike
Long Term Potentiation (LTP) If pre synaptic spike before post synaptic spike Correlated Input
Long Term Depression (LTD) If post synaptic spike before pre synaptic spike Random Input
Experiments with culture of rat Hippocampal cells
STDP: Time Window Time Window: ~ 20 ms
Hard Bounds or Soft Bounds Model
w… either models the conductance for the synaptic input or directly the change in the synaptic current
Area of Depression must be larger
than area of potentiation Typical Values for:
: 20 ms : 20 ms – 100 ms
STDP: Other Models Other Models:
Symmetric STDP:
Short Time Intervalls: LTP Long Time Intervalls: LTD
Inverse STDP Reversed LTP/LTD
Also recurrent loops are possible Surpresses recurrent loops leading to a stable network
Mean Input to a Neuron should only be sufficient to charge the membrane to a point below or only slightly above the treshold Postsynaptic Neuron fires primarily in response to statistical fluctuations
in the input
Basic Experiments with STDP
For a single Post Synaptic Neuron STDP tends to segragate synaptic weights into
strong and weak groups (~ 50 %). Competitive Nature of STDP
Basic Experiments with STDP
Effects of Different Correlation Times - Dots… - Triangles…
also works for larger Correlation times
Network Model: 1-D stimulus
E.g. location of a touch stimulus Encoded in 1000 Input Neurons
Grid over the input stimulus Firing rate: Gaussian Firing Curve
Maximum at the prefered stimulus location of the cell Population Coding
Use periodic boundary conditions 200 network neurons
Sparse connectivity to the input neurons (20 %, random) Learning procedure
Input: Brief presentations of the stimulus at a random input location
Lasts ~ 20 ms (exponential distribution)
Experiments:
Without any recurrent connections: Each Neuron develops a random input selectivity => Nearby input neurons are correlated
Strengthing of synapses in one group of corr. inputs Surpresses other input (competitive)
Experiments
Add all-to-all recurrent excitatory connections to the output neurons Initiliaze weights with zero Selectivity and Column Structure, all neurons are selective in the
same neighborhood Recurrent Connections are quite weak after convergence
Experiments Seeding the network
Network neurons 81 – 120 were given initial input weights for stimulus locations from 401 to 600
Recurrent synapses are strengthened before ff synapses Only one strongly correlated group of network neurons,
many correlated groups of input neurons Seeded network neurons begin to drive unseeded network Synapse from input neurons 401-600 to unseeded network
become strong Recurrent synapses are weakened again
FF synapses compete with recurrent synapses because of shorter latency.
Seeded network units can be seen as sort of teacher signal
Experiments
Seeding the network
Cortical Map
Until now: Competitive nature of STDP leads to a winner
take it all situation Single column structure forms
We want: A Continuous Map Restrict the spread of selectivity from one neuron
to another Limit the recurrent excitatory connections
between network neurons to local neighborhoods Add an initial seed to the FF connections
Experiment: Refinement of Maps
Seeded Cortical Map Initialize FF connections with a coarse map Map is tightened and refined.
Experiment: Unseeded Case
Without initial seeding, a single column structure forms. Map can also arise from random initial conditions if
inhibitatory connections are introduced All to all uniform connections of fixed strength between network
neurons Different neurons in the network tend to develop different location
preferences Local excitatory connections favor similar preferences => formation of a smoothly changing cortical map
Experiment: Unseeded Case
A Map like structure is formed The Map can be arranged in either direction at any point in
the network (random initial conditions)
Learning Spatial Transformations
Learn the transformation of a 1 or 2 DoF simulated arm from proprioceptive input to the visual location of the hand (end effector)
Network structure: Three populations, each consisting of a one (or 2)
dimensional array of cells Input Population: Proprioceptive Stimulus Training Population: Visual Stimulus (Location) Output Population: Prediction
Connectivity: Input -> Output: All to All, learned with STDP Training -> Output: topographical and local, fixed
Learning Procedure A random location is choosen,
the training reference calculated Use for the input population,
for the training population E.g. Again use Gaussian Firing
Curve
Experiment:
1 DoF arm: Evolution of weights
S-Shaped Band becomes visible in the matrix of synaptic weights
Prediction:
Training Layer removed, sweep through the input space Weight pattern from the input to the output layer remains stable
Good approximation found, extreme values are underestimated
Experiment: Other Functions
Linear Functions Non-Linear Functions
Experiment: 2 DoF
Also for the 2-D case, longer learning time
Influence of the population tuning curve shape
Poor accuracy of trigonometric functions near the extremes Multiple input values
giving a single output value
Input values compete with one another
=> Make width of the gaussian population tuning curve smaller
Influence of spatio temporal learning Drawbacks of the learning procedure:
Assumes that all point in space is visited with equal frequency Tested with normal distribution Still works, slower convergence
Requires dwell times (time the input stays in a particular location) Has to be approximately the same as the STDP time
constants Ignores travel time between input locations, input location
changed smoothly Works if motion is fast enough
Inputs must fire together to produce potentiation, but then quickly stop firing to avoid depression
Influence of the correlation time
Increase or use symmetric STDP
Influence of signal latency
Latency difference between the input and the training signal
For negative latencies (training before input), the negative weight pattern can emerge
Influence of Training Pathways
Precise range and strength of the connections to the training population is not critical
Plasticity also for the training connections Pattern is stable for prespecified initial connections Similar to Map refinement from Song
Learning the training pathways Start with random training weights
Recurrent (local) excitatory and inhibitory connections are needed Similar to Map Learning with random initial conditions from Song Connections develop with an arbitrary phase
Tends to wander as training continues
Experiments
Influence of plastic rules: Symmetric: More robust to temporal structure Asymmetric: Less dependent on the population
timing curve Soft weight Boundaries: Modulation of the weight
pattern by correlation is weaker, but still learnt Influence of network size and connectivity
Behavior of the network is the same by adjusting other parameters
Summary
STDP: Biologically Inspired Correlation Based learning Competitive rule which strengthens connections to
correlated input, weakens connections to random input Population coding + STDP:
Can be used to calculate cortical mappings Can also learn (simple) spatial transformations Quite complex model
Learning can also be done by a simple offline-linear regression
Sensitive to a lot of parameters