modeling small neural networks
DESCRIPTION
Modeling Small Neural Networks. Baktash Babadi [email protected] SCS, IPM Fall 2004. References. Cokh & Segev (1998) Principles of neural modeling, 1998, MIT press Abeles (1991) Corticonics Ermentrout (1998) Neural Networks as pattern forming Systems, Rep. Prog. Phys. …. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/1.jpg)
Modeling Small Neural Modeling Small Neural NetworksNetworksBaktash BabadiBaktash [email protected]@ipm.ir
SCS, IPMSCS, IPMFall 2004Fall 2004
![Page 2: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/2.jpg)
ReferencesReferences
Cokh & Segev (1998) Principles of neural Cokh & Segev (1998) Principles of neural modeling, 1998, MIT pressmodeling, 1998, MIT press
Abeles (1991) CorticonicsAbeles (1991) Corticonics
Ermentrout (1998) Neural Networks as Ermentrout (1998) Neural Networks as pattern forming Systems, Rep. Prog. Phys.pattern forming Systems, Rep. Prog. Phys.
……
![Page 3: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/3.jpg)
Studying the Small Neural Studying the Small Neural Networks (1)Networks (1)
The dynamics of single neurons is not The dynamics of single neurons is not taken to be importanttaken to be important
The strength of synaptic connections is the The strength of synaptic connections is the important parameters important parameters
The dynamical state of the networks is the The dynamical state of the networks is the focus of attentionfocus of attention
![Page 4: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/4.jpg)
Studying the Small Neural Studying the Small Neural Networks (2)Networks (2)
Network Architecture:Network Architecture:
Vi
Vj
Wi j
Ji j
![Page 5: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/5.jpg)
Single Neuron network (1)Single Neuron network (1)
The neuron model:The neuron model:
Architecture:Architecture:
x
jjjii
i
exg
VWgVdt
dV
1
1)(
) ( ,
W
![Page 6: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/6.jpg)
Single Neuron network (2)Single Neuron network (2)
weightSynaptic:
Threshold:
W
xexg
VWgVdt
dV
1
1)(
).( The model:The model:
Parameters:Parameters:
Steady States:Steady States:).( ** VWgV
![Page 7: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/7.jpg)
Single Neuron network (3)Single Neuron network (3)
Steady States:Steady States:
).( ** VWgVVy
).( VWgy
WW
![Page 8: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/8.jpg)
Bistability :Bistability :
Single Neuron network (4)Single Neuron network (4)
![Page 9: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/9.jpg)
Two Neuron Networks (1)Two Neuron Networks (1)
Architecture :Architecture :
Equations :Equations :
V1 V2W11 W22
W12
W21
xexg
VWVWgVdt
dV
VWVWgVdt
dV
1
1)(
) (
) (
222,212,122
2
121,211,111
1
![Page 10: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/10.jpg)
Two Neuron Networks (2)Two Neuron Networks (2)
Nullclines:Nullclines:
Shape of Nullclines:Shape of Nullclines:
Sigmoid:Sigmoid:
CubicCubic
xexg
VWVWgVdt
dV
VWVWgVdt
dV
1
1)(
) (
) (
222,212,122
2
121,211,111
1
) (
) (
222,212,12
121,211,11
VWVWgV
VWVWgV
![Page 11: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/11.jpg)
Analyzed by Beer (1995), Ermentrout (1998):Analyzed by Beer (1995), Ermentrout (1998): Minimum number of fixed points: 1Minimum number of fixed points: 1
Maximum number of fixed points: 9Maximum number of fixed points: 9
The cubic nullclines have three rims The cubic nullclines have three rims (outer/inner/outer):(outer/inner/outer):
Intersections (fix points) :Intersections (fix points) :Outer-outer : StableOuter-outer : Stable
Inner-Inner: UnstableInner-Inner: Unstable
Inner-outer: Saddle pointInner-outer: Saddle point In general 13 dynamical states are possible in the two In general 13 dynamical states are possible in the two
neuron network.neuron network.
Two Neuron Networks (2)Two Neuron Networks (2)
![Page 12: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/12.jpg)
Example 1: Two stable fix points, one saddle Example 1: Two stable fix points, one saddle point: point:
Two Neuron Networks (3)Two Neuron Networks (3)
![Page 13: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/13.jpg)
Example: 2 saddle points, 3 stable fix Example: 2 saddle points, 3 stable fix points:points:
Two Neuron Networks (4)Two Neuron Networks (4)
![Page 14: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/14.jpg)
Example: 4 stable fix points, 4 unstable fix Example: 4 stable fix points, 4 unstable fix points, 1 saddle point:points, 1 saddle point:
Two Neuron Networks (5)Two Neuron Networks (5)
![Page 15: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/15.jpg)
Example: 4 stable fix points, 4 unstable fix Example: 4 stable fix points, 4 unstable fix points, 1 saddle point:points, 1 saddle point:
Two Neuron Networks (6)Two Neuron Networks (6)
![Page 16: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/16.jpg)
Example: BistabilityExample: Bistability
Two Neuron Networks (7)Two Neuron Networks (7)
![Page 17: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/17.jpg)
Three Neuron Network (1)Three Neuron Network (1)
Architecture:Architecture:
Equations :Equations :
V1 V2
V3
xexg
VWVWVWgVdt
dV
VWVWVWgVdt
dV
VWVWVWgVdt
dV
1
1)(
) (
) (
) (
333,323,213,133
3
232,322,212,122
2
131,321,211,111
1
![Page 18: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/18.jpg)
Chaotic Behavior is possible: Chaotic Behavior is possible:
Three Neuron Network (2)Three Neuron Network (2)
![Page 19: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/19.jpg)
Netlets (1)Netlets (1)
In Netlets (Anninos et al 1970) The number In Netlets (Anninos et al 1970) The number of neurons is high, the number of of neurons is high, the number of connections is low.connections is low. Neuron Model:Neuron Model:
Architecture :Architecture :The number of neurons The number of neurons (N) (N) is highis high
Each neuron receives input from a Each neuron receives input from a
small number of other neurons (small number of other neurons (n=10n=10))
i
ii
jj
jii
h
ha
aWh
if 1
if 0
,
![Page 20: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/20.jpg)
Netlets (2)Netlets (2)
Dynamics:Dynamics: All the synaptic weights are equal All the synaptic weights are equal (W=1).(W=1). The time is taken to be discrete. We assume The time is taken to be discrete. We assume
that we observe the system at time steps that that we observe the system at time steps that are equal to the synaptic delay.are equal to the synaptic delay.
If a neuron fires at time If a neuron fires at time kk, it will be in , it will be in refractory period at time refractory period at time k+1.k+1.
Since the number of neurons is high, Since the number of neurons is high, statistical methods should be usedstatistical methods should be used
![Page 21: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/21.jpg)
Netlets (3)Netlets (3)
kk at time fired that neurons theoffraction The:?1 k
)1( , ,2
)(
: large and smallFor
2
2
2
)(
kkkk
m
nne
mp
n
kk mnmmnm
nmP
km
1)( )!(!
!)(
: at time inputs receivesneuron ay that Probabilit
![Page 22: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/22.jpg)
Netlets (4)Netlets (4)
th
dmmp
thm
)(
step last time the
from inputs receivesit y that probabilit The
1k at time firesneuron ay that Probabilit The
)1(
2
1
1
2)1(
)()1().1(
:1 at time fire that neurons offraction The
2
kk
k
n
nth
x
kk
th
kkk
dxe
dmmp
k
k
k
1
:1 at time firingfor neurons available offraction The
![Page 23: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/23.jpg)
Netlets (5)Netlets (5)
For For N=10N=10, , W=1W=1, th=5:, th=5:
1 2 3 4 5 6 7 8 9 100
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Time
![Page 24: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/24.jpg)
Netlets (6)Netlets (6)
For For N=10N=10, , W=1W=1, th=3:, th=3:
1 2 3 4 5 6 7 8 9 100
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Time
![Page 25: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/25.jpg)
Netlets (7)Netlets (7)
For For N=10N=10, , W=1W=1, th=1:, th=1:
2 4 6 8 10 12 14 16 18 200
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Time
![Page 26: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/26.jpg)
Netlets (8)Netlets (8)The iterative map :The iterative map :
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
α k+1
α k
Th=3
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Th=1α k+1
α k
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Th=5α k+1
α k
![Page 27: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/27.jpg)
QuestionQuestion
How does the dynamics of a Netlet How does the dynamics of a Netlet change if not all the neurons that fired at change if not all the neurons that fired at time time kk, but a fraction of them (c) remain in , but a fraction of them (c) remain in refractory period at time refractory period at time k+1k+1??
![Page 28: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/28.jpg)
Motor Pattern GeneratorsMotor Pattern Generators
The first article in Computational The first article in Computational Neuroscience:Neuroscience: Brown TG:Brown TG: On the nature of the fundamental activity On the nature of the fundamental activity
of the nervous centres; together with an analysis of of the nervous centres; together with an analysis of the conditioning of rhythmic activity in progression, the conditioning of rhythmic activity in progression, and a theory of the evolution of function in the and a theory of the evolution of function in the nervous system.nervous system. J Physiol J Physiol 1914,1914,48:48:18-46.18-46.
The The
![Page 29: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/29.jpg)
Reflex Loops vs. Central rhythm Reflex Loops vs. Central rhythm generationgeneration
Is the rhythmic motor patterns due toIs the rhythmic motor patterns due to reflex loops reflex loops Or a center that generates rhythms Or a center that generates rhythms
spontaneously? spontaneously?
![Page 30: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/30.jpg)
Central Pattern Generators (CPG)Central Pattern Generators (CPG)
Central pattern generators (CPGs) are neural networks that can endogenously (i.e. without rhythmic sensory or central input) produce rhythmic patterned outputs; these networks underlie the production of most rhythmic motor patterns (Marder and Calabrese, 1996; Stein et al., 1997). The first modern evidence that rhythmic motor patterns are centrally generated was the demonstration that the locust nervous system, when isolated from the animal, could produce rhythmic output resembling that observed during flight.
![Page 31: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/31.jpg)
Fictive MovementFictive Movement
A fictive motor pattern is a pattern of motor neuron firing that would, if the motor neurons were still attached to their muscles, result in the motor pattern in question being produced.
![Page 32: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/32.jpg)
Mechanisms of Rhythm GenerationMechanisms of Rhythm Generation
1) Rhythms driven by Pace Maker 1) Rhythms driven by Pace Maker NeuronsNeurons Vertebrate Respiratory SystemVertebrate Respiratory System Pyloric Ganglion of CrustaceanPyloric Ganglion of Crustacean
2) Rhythms resulting from synaptic 2) Rhythms resulting from synaptic interactions of the neurons. interactions of the neurons. Usually emerge from mutually inhibitory Usually emerge from mutually inhibitory
neurons (reciprocal inhibition).neurons (reciprocal inhibition). Called “Half Center Oscillators” Called “Half Center Oscillators”
![Page 33: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/33.jpg)
Half Center OscillatorsHalf Center Oscillators
The mechanisms of The mechanisms of transition between transition between activation and activation and inhibition:inhibition: Spike frequency Spike frequency
adaptationadaptation Escape from Escape from
inhibitioninhibition
![Page 34: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/34.jpg)
Cellular Properties of Neurons in Cellular Properties of Neurons in CPGCPG
a) Spontaneous Rhythmic firinga) Spontaneous Rhythmic firing
b) Plateau Firingb) Plateau Firing
c) Escape from inhibitionc) Escape from inhibition
d) Post inhibitory reboundd) Post inhibitory rebound
e) Delayed Post inhibitory Rebounde) Delayed Post inhibitory Rebound
![Page 35: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/35.jpg)
Example: Somatogastric (STG) Example: Somatogastric (STG) Nervous System of CrustaceansNervous System of Crustaceans
Marder & Abbott 1997, Modeling small Marder & Abbott 1997, Modeling small neural networks:neural networks:
![Page 36: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/36.jpg)
SourceSource
![Page 37: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/37.jpg)
Precise Firing Sequences (PFS)Precise Firing Sequences (PFS)
Prut et al, 1998:Prut et al, 1998:
![Page 38: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/38.jpg)
Synfire ChainsSynfire Chains
The reproducibility of PFSs implies that The reproducibility of PFSs implies that there are synchronous pools of neurons in there are synchronous pools of neurons in the cortex (Abeles 1991).the cortex (Abeles 1991).
![Page 39: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/39.jpg)
The Notion of Synfire ChainsThe Notion of Synfire Chains Based on anatomical and physiological Based on anatomical and physiological
data, Abeles(1991) proposed the data, Abeles(1991) proposed the Synfire Synfire model which is:model which is:
A locally feed-forward neural networkA locally feed-forward neural network With convergent/divergent connectionsWith convergent/divergent connections
pool
……
Link
……
Propagation of synchronous patterns in Synfire Chains is an explanation for precise firing sequences
![Page 40: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/40.jpg)
Dynamics of Firing Patterns in Dynamics of Firing Patterns in Synfire Chains (1)Synfire Chains (1)
If the Synfire notion is true, the neural activity in successive pools must tend to synchronize and remain synchronous
Hermann, Hertz , Prugel-Bennett (1996): In a simple synfire
model of non-leaky integrate-and-fire neurons, the firing patterns in successive pools tend to synchronize.
![Page 41: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/41.jpg)
Pulse Packets in Synfire networksPulse Packets in Synfire networks
In order to study the spike synchronization In order to study the spike synchronization in synfire networks, Aertsen et al (1995) in synfire networks, Aertsen et al (1995) introduced the notion of pulse packet:introduced the notion of pulse packet: A pulse packet is an index of the activity of a A pulse packet is an index of the activity of a
neuron pool and is defined by two neuron pool and is defined by two parameters:parameters:
1) The number of neurons that fire in the pool1) The number of neurons that fire in the pool
2) The standard deviation of the firing times2) The standard deviation of the firing times
![Page 42: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/42.jpg)
Dynamics of Firing Patterns in Dynamics of Firing Patterns in Feed-forward Networks (1)Feed-forward Networks (1)
Diesmann, Gewaltig, Aertsen (1999)Diesmann, Gewaltig, Aertsen (1999)
Bistability in a phase-plane portrait: Dense and highly synchronous pulse packets
will propagate successfully. Sparse and weakly synchronous ones will
eventually dissipate.
![Page 43: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/43.jpg)
Gewaltig, Diesman, Aertsen (2001)Gewaltig, Diesman, Aertsen (2001)
Survival probability of pulse packets in single trials:
Dynamics of Firing Patterns in Dynamics of Firing Patterns in Feed-forward Networks (2)Feed-forward Networks (2)
![Page 44: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/44.jpg)
Dynamics of Firing Patterns in Dynamics of Firing Patterns in Synfire Chains (3)Synfire Chains (3)
Cateau & Fukai (2001) : Using Fokker-Planck equations, the
previous results have been confirmed analytically.
![Page 45: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/45.jpg)
Common Issues Among All the Common Issues Among All the Mentioned StudiesMentioned Studies::
The Synfire activity ends in two scenarios:
Saturation : All the neurons in the final pools fire
Decay: No neuron in the final pools fire (apart from the background activity)
In either case there is a loss of information content
The synaptic weights are uniform
![Page 46: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/46.jpg)
Our QuestionOur Question
Is it possible to avoid full saturation and Is it possible to avoid full saturation and full decay by modifying the synaptic full decay by modifying the synaptic weights of a Synfire network?weights of a Synfire network?
![Page 47: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/47.jpg)
Analysis Assumptions :Analysis Assumptions :
While we do not aim to study the While we do not aim to study the synchronizing property of the network and synchronizing property of the network and only the number of firing neurons is our only the number of firing neurons is our mater of interest, the network is fed with a mater of interest, the network is fed with a fully synchronous pattern.fully synchronous pattern.
No synaptic delay is taken into account.No synaptic delay is taken into account.
![Page 48: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/48.jpg)
Reformulating the problem:Reformulating the problem:
Assume that we feed the net with an input pattern
containing n firing neurons. How should we set the weight parameters ( , ) to avoid full saturation (final n=50) or full decay (final n=0)?
w w
![Page 49: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/49.jpg)
Method of AnalysisMethod of Analysis
Iterative mappingIterative mapping )( 1 nRn
![Page 50: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/50.jpg)
The Iterative MapThe Iterative Map
)( )( 1
0
nRduuGNn
u
2221
1
0 thwn
wnthu
2)(
2
2u
euG
![Page 51: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/51.jpg)
When Mean Weight is positive:
Three attractor states are possible for the number of neurons:
Full saturation / full decay behavior
Only full fade behavior
Attraction to a fixed number of firing neurons
![Page 52: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/52.jpg)
Full saturation and full decayFull saturation and full decay
When :When : 0.001 0.003 ww
![Page 53: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/53.jpg)
Only full decayOnly full decay
When :When : 0.001 0.015 ww
![Page 54: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/54.jpg)
Fixed point AttractorFixed point Attractor
When :When : 0.02 0.003 ww
![Page 55: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/55.jpg)
Three attractor states are possible for Three attractor states are possible for the number of neurons:the number of neurons:
Attraction to a fixed number of firing Attraction to a fixed number of firing neuronsneurons
Oscillation between different number of Oscillation between different number of firing neuronsfiring neurons
Multiple periodic/chaotic behaviorMultiple periodic/chaotic behavior
When Mean Weight is Negative:When Mean Weight is Negative:
![Page 56: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/56.jpg)
Fixed point attractorFixed point attractor
When :When : 0.64 0.3- ww
![Page 57: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/57.jpg)
Cycle AttractorCycle Attractor
When :When : 0.528 3.0 ww
![Page 58: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/58.jpg)
Multiple periodic/Chaotic attractorMultiple periodic/Chaotic attractor When :When :
0.256 3.0 ww
![Page 59: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/59.jpg)
The Bifurcation Diagram for The Bifurcation Diagram for Negative Mean WeightNegative Mean Weight
![Page 60: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/60.jpg)
3D bifurcation diagram ! 3D bifurcation diagram !
![Page 61: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/61.jpg)
3D bifurcation diagram !3D bifurcation diagram !
![Page 62: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/62.jpg)
Two further Questions:Two further Questions:
What if we feed the network with What if we feed the network with asynchronous inputs?asynchronous inputs?
What if the neurons have spontaneous What if the neurons have spontaneous firings?firings?
![Page 63: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/63.jpg)
The Simulation results with The Simulation results with asynchronous inputsasynchronous inputs
![Page 64: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/64.jpg)
The Simulation results with The Simulation results with asynchronous inputs and asynchronous inputs and
spontaneous firingsspontaneous firings
![Page 65: Modeling Small Neural Networks](https://reader036.vdocument.in/reader036/viewer/2022062518/568146bc550346895db3f0af/html5/thumbnails/65.jpg)
Open Questions:Open Questions:
1) The speed of pulse propagation in 1) The speed of pulse propagation in synfire chainssynfire chains
2) Analysis of asynchronous pulse packets 2) Analysis of asynchronous pulse packets in synfire chainsin synfire chains
3) The effect of synaptic plasticity on the 3) The effect of synaptic plasticity on the dynamics of synfire chainsdynamics of synfire chains