1

Awakening Dynamics via Passive Coupling and Synchronization Mechanism in

Oscillatory Cellular Neural/Nonlinear Networks

István Szatmári,

Cellular Sensory Wave Computing Laboratory, Computer and Automation Research Institute, Hungarian Academy of Sciences, Kende u. 13-17, Budapest, H-1111, Hungary

Leon O. Chua

Department of Electrical Engineering and Computer Sciences, University of California at Berkeley, CA 94720, USA

Abstract: We have studied synchronization mechanism in locally coupled nonlinear oscillators. Here, synchronization takes place by passive coupling based on a reaction-diffusion process. We will compare this mechanism to basic synchronization techniques, showing their similarities and specific properties. In addition to synchronization, passive and local coupling can also “awaken” non-oscillating cell circuits and trigger oscillation provided that cells are locally active. This result resembles Turing’s and Smale’s works showing that locally communicating simple elements can produce very different patterns even if separate elements do not show any activity. This property will be demonstrated for two second-order cells and also for a large ensemble of oscillatory cells. In later case, the network of oscillatory cells exhibits very sophisticated spatio-temporal waves, e.g., spiral waves. Keywords: synchronization, oscillation, non-linear dynamics, passive coupling, local activity

Introduction Synchronization is a very natural phenomenon observed in our daily life (including, for example, fireflies flashing and crickets chirping in synchrony, heart cells beating in rhythm). It plays crucial role in many technical applications (quartz watch, atomic clock, oscilloscope, radio, wireless communication, GPS positioning system, etc.) In science, the study of synchronization mechanism has started with the historical observation by Huygens in pendulum clocks. Theory of synchronization implies periodicity of oscillators and nonlinear dynamics. This phenomenon is unique in that it can be observed physically only in nonlinear systems. One striking example in coupled oscillations is the awakening of “dead” cells via passive coupling. This counterintuitive example was shown by Smale for two 4th order cells [1]. Here, however, we show what the reason and mechanism are due to local activity [2-4, 16, 17], and we need only two 2nd order cells.

2

First, we will investigate a basic synchronization mechanism by using a simple electronic circuit. Second, we will analyze how this synchronization takes place in systems consisting of a large number of simple oscillatory cells arranged in a regular way, such as in Cellular Neural Networks [5-7]. Third, we will derive conditions necessary both for oscillation and synchronization. Finally, we will show special cases where connections (communication) among cells not only play a crucial role in the synchronization of cells but, in addition, they can trigger oscillation and awaken previously “dead” cells.

The basic synchronization mechanism via small control signals Consider the circuit shown in Fig. 1. This is a negative resistance converter with a capacitor connected across the input port. It is also known as relaxation oscillator. This circuit oscillates at its natural frequency (fy) determined by its component parameters (RA, RB, Rf, C, and the saturation voltage of the op-amp). For more details, see [8].

Fig. 1 (a) An oscillating RC op-amp circuit and (b) its driving-point characteristic

The driving point characteristic (DP plot) of the circuit is shown in Fig. 1 (b). Since

Ctitvtv cin /)()()( −== �� and C > 0, we have 0)( >tvin� for all t such that i(t) < 0,

and 0)( <tvin� for all t such that i(t) > 0. Hence the dynamic route from any initial point must move toward the left in the upper half plane, and toward the right in the lower half plane, as indicated by the arrow heads in Fig. 1 (b). Observe that there is no stable equilibrium point in the circuit because the zero i current belongs to an unstable equilibrium (opposite arrowheads diverging from zero). This equilibrium point cannot be observed in practice, because any small amount of noise will drive out the circuit from this point. Either the breakpoints QA, and QB

3

cannot be equilibrium points because 0≠i . The dotted arrows show that a sudden instantaneous transition will occur (also called jump)1 at these breakpoints (impasse points). For an RC circuit this transition should be always a vertical jump (assuming in the v-i plane that i is the vertical axis) because the voltage across a capacitance cannot be changed suddenly such that )()( −+ = TvTv cc . Applying jumps at the two impasse points QA, and QB we obtain a closed dynamic route. This means that the solution waveforms become periodic after a short transient time (starting from any initial capacitance voltage) and the op-amp circuit functions as an oscillator. Note that the oscillation is not sinusoidal. Such oscillators are usually called relaxation oscillators. The oscillation frequency will vary depending on parameters as the supply voltage, changes in temperature, noise, etc. There are various methods that can be used for forcing the oscillator frequency fy to “synchronize” with an input signal at a frequency fx. Since the basic principles are the same in all cases, we shall consider only one method, the pulse synchronization method. In the pulse synchronization method the oscillator is synchronized with a voltage pulse vs(t) having the same frequency fx as the input signal. This synchronization signal can be derived from the input signal by some standard waveform operation in order to ensure that the frequency be exactly fx. Let us connect a synchronization voltage source vs(t) in series with the capacitor as shown in Fig. 2(a). Suppose, that vs(t) has a sharp positive pulse with amplitude Es. If vs(t) is as shown in Fig. 2(c), then the DP plot will be shifted to the left by an amount equal to the pulse height Es. Whenever a pulse appears, the capacitor will see a different dynamic route as shown in Fig. 2(b). Observe that if the pulse appears (e.g. at Q1) before the dynamic route reaches the impasse point QA, then a sudden instantaneous transition will occur (i.e. jump) to the new DP route at Q2. Note that the capacitor voltage cannot change suddenly, therefore, the jump should be vertical. After the pulse (Q3) the dynamic route changes to its original position. We can see that a minimum pulse height (Es) and a minimum pulse frequency (fx) (relative to the unsynchronized frequency fy) are required to ensure synchronization. It is easy to see that the frequency of the triggering signal (fx) should be greater than the natural frequency of the oscillator (fy). Then, this jump will occur always close to QA and even a small amplitude of the pulse (Es) will be enough to cause switching. The oscillator will be precisely switched at the frequency of the trigger signal i.e., synchronized to the input frequency (Fig. 3). Synchronization may occur not only when the triggering signal has a slightly greater frequency than the oscillator, but also when the oscillator frequency is subharmonic of the input triggering signal frequency, namely,

xy fn

f1=

1 By inserting a very small linear inductor (representing the inductance of the connecting wires) in series with the capacitor would make it possible to analyze this circuit without these sudden transitions [5].

4

Fig. 2 Oscillator circuit with a triggering signal connected in series with the capacitor, (b) shifted dynamic route of the oscillator corresponding to a square pulse triggering signal, (c) triggering square pulse

This property is the basic operating principle of most frequency dividers of today. By connecting an appropriate number of these dividers in cascade, it is possible to divide accurately an extremely high-frequency pulse train down to any desired output frequency. This property has been used in designing e.g., atomic clocks.

5

][, Vv][),( Vtv

30

15

015−

30−

E−15

30

15

0

15−

30−

CR f=τ

2yT

yT2

3 yTyT2

yy fT /1=

xx fT /1=)(tvs

sE

][, Vv15

15−0

][, st

][, st

][, st

frequencyationsynchronizf

frequencynaturalf

x

y

:

:

yx ff >

][, mAi

Fig. 3 Synchronization mechanism of pulse synchronization method

Oscillatory Cellular Neural/Nonlinear Networks The types of systems considered here are the ones which consist of a large number of cells arranged in a regular way. A Cellular Neural/Nonlinear Network (CNN) is any spatial arrangement of locally-coupled cells, where each cell is a dynamical system evolving according to some prescribed dynamical laws [3,4,6]. Therefore, a CNN is defined by two mathematical constructs:

• the dynamics of the cell • the coupling law relating one or more relevant variables of each cell to all

neighbor cells within a prescribed sphere of influence. We will focus on cells that can exhibit oscillatory phenomena, consequently, circuits made of two energy-storage elements (e.g. two capacitors) are considered here. These types of circuits can be described by a second-order scalar differential equation or equivalently by two first-order scalar differential equations (state equation). We will follow the later one. The second-order autonomous dynamics of a cell is given by

( )( )��

�

==

2122

2111

,

,

xxfx

xxfx�

� (1)

Specifically, an autonomous two-layer Cellular Neural/Nonlinear Network (CNN) model will be used here to model both cells and coupling rules. A similar model was derived in [10] where the complex dynamics of the network was thoroughly analyzed and pattern

6

formation and self-organization capabilities had been demonstrated. Model and dynamics describing a single oscillating CNN cell (without coupling rules and omitting grid indices) are given by

( ) ( ) ( ) ( ) ( )( ) ( ) ( ) ( ) ( )��

�

⋅+⋅++−=⋅+⋅++−=

tytytxtx

tytytxtx

122222

211111

1

1

βαβα

�

� (2)2

with initial condition ( ) ( )0,0 21 xx , and where the output nonlinearity function ( )y x is the piecewise-linear function

( ) ( )1121 −−+⋅= xxxy (3)

The output of a cell is defined as ijy and, therefore, always bounded: 1<ijy . To derive

specific properties of this non-linear network first we will analyze its behavior in the linear region (i.e. each cell operates in the linear region). When all 1<ijx , then

( ) ijij xxy = and the whole system behaves according to the linear system (writing equation for a single cell)

( ) ( )( ) ( ) �

��

⋅+⋅=⋅+⋅=

)(

)(

12222

21111

txtxtx

txtxtx

βαβα

�

� (4)

In a compact form

1 11 1 1 1

2 2 2 22 2

,or ,x x

x x

α β α ββ α β α

� � � �� � � �= ⋅ = ⋅ =� � � �� � � �� � � �

x A x A�

��

(5)

The eigenvalues of the characteristic equation are

( )21 21,2 1 2 1 2 1 2

12 4

α αλ α α α α β β+= ± + − ⋅ + ⋅

In Fig. 4 we can see the possible equilibrium states. We introduced variables

21 αα +=T , trace of matrix A, and 2121 ββαα ⋅−⋅=∆ , determinant of matrix A. Here, we are interested in oscillation only. Oscillations occur if the eigenvalues are complex. We have this possibility in case of unstable focus (stable focus and lossless oscillation do not bear practical issue in our case). Thus the condition for unstable focus in terms of T and ∆ are

�

�

�

>∆

>

4

02T

T. (6)

2 The term 1+� will later be useful to simplify equations

7

The amplitude of the theoretically ever expanding spiral should somehow be controlled. This will exactly be provided by the non-linearity of the CNN.

Fig. 4 Equilibrium state classification diagram in linear case

If we can ensure that there should not be any stable equilibrium in the saturated regime (forced stability) then, according to the Poincaré-Bendixon Theorem, it can be concluded that the system has an unstable focus surrounded by a stable limit cycle. Remarks: 1. All trajectories in the phase space far from equilibrium ( βα ++≅ 1R )obey the dynamic

equation 11 xx −≅� and 22 xx −≅� . Therefore, all solutions are bounded. 2. Trajectories cannot intersect itself (uniqueness property). The expanding spiral must necessarily converge to some limiting closed contour. For more details, see [9]. The system will give rise to an oscillatory periodic steady state behavior. How to avoid forced stability? We need to find locations of all equilibrium points of (2) ( 1 0,x =� and

2 0,x =� ) and ensure that there should not be any equilibrium except the origin. Remark: Since the piecewise-linear output function has three segments, the x1-x2 state space can be partitioned into nine rectangular regions where the state equations are reduced to a linear equation in each region and can trivially be calculated.

8

1 1 1 1 2

2 2 2 2 1

( ) (1 ) ( ) ( )

( ) (1 ) ( ) ( )

x y y

x y y

α βα β

∞ = + ⋅ ∞ + ⋅ ∞ ��∞ = + ⋅ ∞ + ⋅ ∞ �

(7)

To see a simplified example let us have 1 2 ,α α α= = and 1 2 0β β β= − = > . In this case

2T α= and 2 2α β∆ = + , therefore, the eigenvalues are 1,2 jλ α β= ± ⋅ . First, we can see

that 0α > ensures unstable focus (this is equivalent to DP plot having positive slope in the linear region – no stable solution exists in the linear region, thus enabling only binary steady state solutions). Second, we can write equilibrium points in the saturated regions ( 1 21, 1x x> > ), (notation hint: Qs

++ means 1 21, 1x x> > ):

1 1

2 2

1 1

2 2

: 1 : 1,

1 1

: 1 : 1,

1 1

s s

s s

Q x Q x

x x

Q x Q x

x x

α β α βα β α β

α β α βα β α β

++ −−

+− −+

� �= + + = − − − � �

= + − = − − + � �

� �= + − = − − + � �

= − − − = + + � �

(8)

Writing conditions not giving stable equilibrium in all the four cases, we get 1 1,x α β= + − < thus

α β< (9)

The condition ensures that there should not be any stable equilibrium (forced stability) in the saturated regions. Wee still need to check partially saturated regions (i.e.

1 21, 1x x> ≤ or vice versa). Writing equations of partially saturated regions results in

the same α β< condition. Remark: already the previously mentioned 0α > condition ensures that trajectories should leave partially saturated regions as well. Summarizing: we have two conditions to get stable oscillatory periodic solution (limit cycle):

0αβ α

> ��> �

(10)

In terms of T and ∆ , we can write

�

�

�

>∆

>

2

02T

T (11)

This is shown in Fig. 5. All other equilibrium cases, except the unstable focus, are the same in non-linear system as well, additionally, the system settles down a (full or semi-) saturated stable equilibrium point in cases of saddle point and unstable node.

9

Fig. 5 Equilibrium state classification diagram in non-linear case

In arbitrary case we get conditions for oscillation:

( )1 2

21 2 1 2

0

10

4

α α

α α β β

+ > ��− + ⋅ < �

(12)

It can be seen that �1 and �2 will have opposite signs ( 1 2 0β β⋅ < ) and

1 1

2 2

β αβ α

�> �

> �

(13)

Conditions (12) and (13) should be satisfied to ensure limit cycle.

10

Reaction-Diffusion Oscillatory CNN Here, we start our consideration with the following standard Reaction-Diffusion PDE:

( ) xDxfx ∆⋅+=∂∂

t, (14)

where x is the vector of state variables at a given spatial coordinate, D is a constant

diffusion matrix, and ∆ is the continuous Laplace operator (2 2

2 2x y∂ ∂+∂ ∂

) in

two-dimensional case. We will focus on second order oscillatory CNN cells, and mapping this equation into a CNN reaction-diffusion equation, we will get the following equation (omitting grid indices):

1 1 1 1 1 2 1 1

2 2 2 2 2 1 2 2

( ) ( ) (1 ) ( ) ( ) ( )

( ) ( ) (1 ) ( ) ( ) ( )

x t x t y t y t D y t

x t x t y t y t D y t

α βα β

= − + + ⋅ + ⋅ + ⋅ ∆ ��= − + + ⋅ + ⋅ + ⋅ ∆ �

�

� (15)

and detailed for the first state variable with discrete Laplace operator (4-connectivity)

1, 1,

1 1 1 1 1 2 1 1, , 1 1, , 1, , 1

1, 1,

1(1 ) 1 4 1

1f(x)

D ∆y

i j

i j i j i j

i j

y

x x y y D y y y

y

α β−

− +

+

⋅

� �� �� �� �= − + + ⋅ + ⋅ + ⋅ − ∗ � �� �� �� �� �

�� � � �� � � � � ��

� � � � � � � � � � � � � � � � �

(16)

Focusing on the DDD ===−=== 212121 ,, βββααα , parameters �, �, and D completely determine the network settings, i.e. template values. The focus will mainly be on 4-connectivity interactions. Connections among oscillatory cells by means of diffusion usually raise special waves such as spiral waves. For an overview of these most “natural” types of wave, see [11, 12]. Spiral patterns and waves were commonly observed in certain two-dimensional chemical systems such as the Belousov-Zhabotinskii reaction, for an overview, see [13] and also in a variety of biological systems, e.g., [14, 15]. Different CNN models showing this type of complexity are known and various techniques for their analysis have been derived, see e.g., [18-22]. As an example of spiral wave generation, let us check the system composed by oscillatory cells with the following parameters.

� � D 0.5 0.7 0.3

Note that condition in Eq. (10) holds, i.e. individual uncoupled cells have limit cycle (oscillating). Some consecutive snapshots are shown in Fig. 6 and Fig. 7, respectively (sampling is 0.1�CNN).

� � D 0.5 0.7 0.6 and

11

� = 0.5, � = 0.7, D = 0.3

Fig. 6 Spiral wave propagation in coupled oscillatory network. Parameters are � = 0.5, � = 0.7, D = 0.3. Each cell oscillates at a given frequency while their synchronization via coupling (diffusion) ensures that spiral waves should be propagating through the network. � = 0.5, � = 0.7, D = 0.6

Fig. 7 Spiral wave propagation in coupled oscillatory network. Parameters are � = 0.5, � = 0.7, D = 0.6. Cell parameters (�, �) determine the oscillation frequency, while diffusion (D) controls the size and speed of spiral arms.

12

The properties of reaction-diffusion CNN are:

• natural oscillation frequency of individual cells is determined exclusively by parameters � and �. See Fig. 8.

• Diffusion coefficient D increases this natural oscillation frequency. Synchronization takes place via diffusion.

• The increase of diffusion coefficients increases the propagation speed of spiral waves and also the width of spiral wave fronts. See Fig. 9.

Example: Let network parameters be � = 0.5, � = 1.0, D = [0.1, … 1.0], then

• Linear case: angular frequency, � = � = 1.0 1/�CNN. (unstable, exponentially exploding oscillation)

• Nonlinear case, without diffusion, Natural (uncoupled) oscillation frequency, � = 0.57 1/�CNN.

• Synchronized oscillation frequency (with diffusion), � ~ 0.6 – 0.7 1/�CNN. It depends on the diffusion and also on the emerging patterns.

Fig. 8 Relative oscillation frequency of uncoupled nonlinear oscillators depending on parameters � and �. The parameter � stands for self-feedback, while � for cross-feedback. In lossless case (� = 0) � = � and it will decrease if we increase �. Practical implementation requires that � > 0 (condition for stable limit cycle). Oscillations requires that � < �.

13

� = 0.5, � = 0.7, D = 0.1

D = 0.2

D = 0.3

D = 0.4

D = 0.5

D = 1.0

D = 1.5

D = 2.0

D = 5.0

D = 10.0

D = 15.0

D = 20.0

Fig. 9 Spiral wave propagation in coupled oscillatory network with different diffusion coefficients. Diffusion (D) controls the size and speed of spiral arms.

Synchronization After a short transient, cells will be oscillating at the same frequency and their relative phases will be locked, i.e. cells are synchronized. The frequency of oscillation is determined by the cell parameters (� and �), while the synchronization takes place via cell interaction i.e. diffusion (determined by the diffusion term D). Propagating spiral waves will appear and the speed of their spiral arms has the function of

( , ) ( )c Dυ α β λ= ⋅ ,

(17)

where � stands for the oscillation frequency and � is two times the characteristic width of a spiral arm. Once the network converges its limit cycle, then oscillating cells will be remained in synchronization. This means that cell connections will take effect only for a

14

very short time in order to keep the synchronization. The additive term of diffusion will give zero contribution except for a very short time in each oscillating period. At a given point if we set this diffusion term to zero (i.e. the cell connections were switched off and cells oscillated separately), then spiral wave propagation would still be observable. Although cells oscillate separately, they keep their relative phases to each other. Here, phase wave propagation would have the special form of spiral waves. Fig. 10 shows the property of locked phase of two oscillating cells.

0 20 40 60 80 100 120 140 160 180 200-3

-2

-1

0

1

2

3Synchronization of 2 Oscillating Cells

Time[τCNN]

Am

plitu

de

cell1cell2

Fig. 10 Amplitude of two oscillating cells (their relative position is less than the characteristic width of the spiral arm). After roughly 120 �CNN the cells are synchronized, i.e their relative phases are locked. Parameters are [�, �, D] = [0.5, 1.0, 0.3].

Fig. 11 shows the additive term of diffusion to the cell current. In most part of time this term is zero except when high amplitude spikes occur keeping the synchronization. This resembles the detailed mechanism of pulse synchronization method discussed earlier.

15

0 20 40 60 80 100 120 140 160 180 200-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4Diffusion term

Time [τCNN]

Am

plitu

de

Fig. 11 Change of the diffusion term in time of a given cell. It can be seen that after roughly 120 �CNN the cell has been synchronized to its neighbors. The diffusion term is zero in the large pat of time (roughly ~60-70%) except the positions when high amplitude spikes keep this synchronization. These spikes are typical in synchronized circuits. Parameters are [�, �, D] = [0.5, 1.0, 0.3].

Synchronization depends on diffusion: larger D provides shorter transient time. And the number of synchronization events is less. Larger D means the width of spiral arms is larger and also the number of spiral arms is less. Fewer wave front interactions mean that the transient to get synchronized will be shorter. Two phases of synchronization can be distinguished. First, cells closer than the characteristic size of the spiral arm will be synchronized earlier and tend to oscillate together. In this phase interacting spiral arms cause transient thorough the whole network. The second phase starts when these interacting spiral waves are damped down and meta-stability among the spiral arms is formed. Synchronization depends also on network size: smaller the network less number of spiral wave arms can be generated, the less time is needed to get synchronized. Less spiral arms interact to each other. Also an important note: oscillation frequency depends on emerging patterns, too. If, for instance, a simple oscillation source is formed (no spiral arms) at a given location of the network, then overall cell frequencies are higher than if spiral waves were generated. Therefore, oscillation frequency is not completely determined by the three network parameters (�, �, D). A short time interval of a cell dynamics is shown in Fig. 12. The network is already in a quasi-stable state: repeating spiral waves. Figure shows typical dynamics that can be observed in oscillatory circuits with coupling via diffusion. The first state variable is shown with its driving terms. Term -�*X2 drives state of X1, while �*X1 forces X2. This determines oscillation itself. Without nonlinearity (saturation effect is clearly visible on term -�*X2), cells would have an unbounded oscillation but saturation ensures that we

16

have a stable limit cycle (the second important network parameter �, with condition �>�>1 ensures that oscillation does not fade away). Diffusion ensures the synchronization and has little effect on oscillation itself (it changes only the phase). According to measurements, the frequency of oscillation is a bit faster than it would be without diffusion. Fig. 12 shows explanation for this behavior.

180 182 184 186 188 190 192 194 196 198 200-2

-1.5

-1

-0.5

0

0.5

1

1.5

2Synchronization via Diffusion

Time [τCNN]

Am

plitu

de

X1 - state variable -ββββ ••••Y2

Diffusion term

A

BC

D

Fig. 12 Synchronization mechanism among CNN cell oscillators via diffusion. X1: state variable of a CNN cell. Term -�*X2: feedback control from the second state variable forcing oscillation (�*X1 drives X2). Events A, B, C, and D are synchronization “spikes”. Due to the diffusion mechanism, these “spikes” are not sharp at all but they are rather smooth. Depending on cell entering or leaving saturation region and arriving or leaving wave fronts, there are four synchronization events. Network parameters are [�, �, D = 0.5, 1.0, 1.0]

There are four synchronization events: A Cell leaves positive saturation. Incoming white (negative) wavefront. B Cell enters negative saturation. Leaving white (negative) wavefront. C Cell leaves negative saturation. Incoming black (positive) wavefront. D Cell enters positive saturation. Leaving black (positive) wavefront.

• In all the four cases the diffusion term is opposite to the cell state variable, therefore, it speeds up the change, i.e. oscillation will be slightly faster than the natural frequency of uncoupled cells.

17

• Synchronization spikes are smooth, due to the diffusion mechanism. Change of the state variable is also smooth, i.e. there are no sharp points. Here, synchronization behaves merely as a fine tuning.

Position A: Cell enters the linear region. Before this, it was in the positive saturation region and also all its neighbors were in this region. The diffusion term was zero. Before point A, a negative term starts growing but only in case the neighboring cells leave their positive saturation region, i.e. an incoming white wavefront is approaching, therefore, the sum of diffusion coefficients multiplied with the neighbor’s output produces a less positive or even a negative value. The minimum (the peak) of A is exactly when the central cell leaves the positive saturation region. Therefore, the term -4D*y will be less negative and the sum of the diffusion term will increase. Position B: Cell enters the negative saturation region. Meanwhile, neighboring cells tend to become negative as white wavefront passes by. When all neighbors become white the sum of diffusion term will be zero. Position C: At this point an incoming positive (black) wavefront is approaching. The diffusion terms tends to be positive and it will reach its maximum when the cell leaves its negative saturation region and enters a linear one. Position D: Now, the cell enters the positive saturation region and because not all of its neighbors are triggered to black, the diffusion term is negative. It will have the maximum when the cell reaches exactly the saturated region. Diffusion term will diminish when all of its neighboring cell become positively saturated.

Awakening “dead” cells In this section we will discuss the role of diffusion and show that it might play a more crucial role, in addition to synchronization. Stephen Smale shows in his work [1] that inactive so-called “dead” cells might become “alive” by passive coupling. In his example two cells with four enzymes (two 4th order dynamical systems) have a globally stable equilibrium. The cell is “dead” in the sense that the concentrations of its enzymes always tend to the same fixed levels. When two cells like this are coupled in a simple way (diffusion through a membrane), however, the resulting Turing equations are shown to have a globally stable limit cycle. The concentrations of the enzymes begin to oscillate, and the system becomes alive. Here, we will show that instead of four state variables, a second order system shows also this capability, i.e. globally stable system will become active if passive coupling (diffusion) is switched on. We will show this property both for two 2nd order oscillatory CNN cells coupled together and also for a whole network of oscillatory CNN. The key behind this behavior is the concept of local activity. The local activity theorem [11, 12] states that a system cannot exhibit emergence and complexity unless its cells are locally active. This ensures that even a passive coupling might drive a system not only to leave the stable equilibrium but also produce complex patterns, in our case spiral waves via oscillation.

18

We will analyze the model of two 2nd order oscillatory CNN cells, see Fig. 13. State equations of two-parameter case are as follows:

( ) ( )( ) ( )( ) ( )( ) ( )

�

�

�

−⋅+⋅+⋅++−=−⋅+⋅−⋅++−=−⋅+⋅+⋅++−=

−⋅+⋅−⋅++−=

423444

314333

241222

132111

1

1

1

1

yyDyyxx

yyDyyxx

yyDyyxx

yyDyyxx

βαβαβα

βα

�

�

�

�

(18)

1x

2x

3x

4x

D

D

α

α

α

α

β β− β β−

1st cell 2nd cell

Fig. 13 Two second order CNN cells are connected by passive coupling (conductance of the resistive connection equals to the diffusion coefficient, i.e. G=D). In case of parameter values � > � > 0, standalone cells are forced to stable equilibriums. Let coupling be for t<t0 disconnected so cells will be saturated and the system is completely stable. Output values are either +1 or -1. At t= t0, we close the passive coupling and the system will oscillate if the connected output pairs had opposite output values before. Condition for oscillation is � < � +2D.

From the analysis of standalone second order oscillatory CNN cell we know that there is no oscillation in the region of forced stability (see Fig. 5). The condition of parameters for forced stability in a two-parameter case is

0α β> > (19)

If this second order cell is separated (there is no coupling), then the cell will be saturated starting from any initial condition. In the linear region its behavior is oscillatory which would grow exponentially to infinity if there was not any non-linearity. In the saturation region the cell converges to stable equilibrium and the system is completely stable. What can be stated about the equilibrium points of this system? Let us identify the local activity behavior of the equilibrium points. The local activity theorem states [2,3] that if there is at least one equilibrium point for which the cell acts like a source of “small signal” power, i.e. if the cell is capable of injecting a net small-signal average power into the passive (i.e. D ≥ 0) resistive grids, then the cell is said to be locally active. Let Qi be a

19

cell equilibrium point of the second-order CNN cell and the Jacobian matrix associated with the small signal cell coefficients at Qi can be derived as

( )( ) ( )

( ) ( ) �

���

�=

����

�

����

�

�

∂∂

∂∂

∂∂

∂∂

=2221

1211

2

212

1

212

2

211

1

211

,,

,,

aa

aa

xxxf

xxxf

xxxf

xxxf

QJ i (20)

Considering the second-order CNN cell, we can distinguish linear, semi-saturated, and saturated regions. The Jacobian matrices for linear, semi-saturated, and saturated regions are as follows, respectively: a), linear region where 1,1 21 << xandx

( ) �

���

� −=

αββα

ia QJ

b), semi saturated region where 1,1 21 >< xandx

( ) �

���

�

−=

10

βα

ib QJ

c), semi saturated region where 1,1 21 <> xandx

( ) �

���

� −−=

αβ

01

ic QJ , and finally

d), saturated region where 1,1 21 >> xandx

( ) �

���

�

−−

=10

01id QJ

From [2] it can be stated that a two-port reaction diffusion CNN cell with two diffusion coefficients and two state variables is locally active at a given cell equilibrium point Qi if, and only if, any one of the following two conditions hold at Qi:

( )221122211

22

4)2

0)1

aaaa

a

+<

> (21)

We can see that the Jacobian matrix in the saturated region, i.e. ( )id QJ , is passive only. All other regions (linear and semi-saturated) are locally active, i.e. the CNN cell is locally active. Our conclusion is that this two-port CNN cell is although locally active, but the system converges to stable saturated equilibrium points and these equilibrium points are locally passive. The system is completely stable. By using Smale’s phrase, the cell can be categorically called dead. Is it possible to awaken a cell like this at all? We will show the answer is definitely affirmative.

20

What happens if we switch on a passive coupling (diffusion)? Although the resistive connection is passive, the system will leave its equilibrium and starts oscillating (driving the cell to locally active regions). The condition for oscillation is that both cross-connected output pairs shall have opposite sign at time of switching of coupling, otherwise, there would not be any current flow (signal flow). If we examine the saturated regions and write conditions for dynamic active patterns (i.e. cell becomes active and leaves the saturated region we get the necessary condition ( 0>α ensures that there should not be any stable equilibrium points in either the linear or the semi-saturated regions). We need to examine all possible situations, however, we show only one case because each arrangement yields the same condition. Let us examine the equilibrium points in the saturated regions Qs

+-,-+, where 1 2 3 41, 1, 1, 1x x x x> < − < − > . The steady sate values are

,

1

2

3

4

:

1 2

1 2 ,

1 2

1 2

sQ

x D

x D

x D

x D

α βα βα β

α β

+− −+ �= + + − = − − + + �= − − − += + − − �

(22)

Writing conditions not giving stable equilibrium in all the four cases, we get ,

1

2

3

4

:

1 2 1

1 2 1 ,

1 2 1

1 2 1

sQ

x D

x D

x D

x D

α βα βα β

α β

+− −+ �= + + − < = − − + + > − �= − − − + > −= + − − < �

(23)

We are looking for the values where βα > . Finally, the condition for oscillation when at time 0tt = we switch on the connection can be stated as follows:

�

�

�

−=

−=+<<

=

=

)()(

)()(2

402

301

ysignysign

ysignysign

D

tt

tt

βαβ (24)

The next figure shows the parameter field for diffuse oscillation.

21

�

�=�

2D

�=�-2D Limit cycle

Diffuse oscillation

Forced stability

�

� �=�

Limit cycle

Forced stability

�

Fig. 14 Parameter space for diffuse oscillation of two 2nd order CNN cells. Terms �, �, and D stands for self-feedback, cross-feedback, and diffusion coefficient, respectively. Condition for diffuse oscillation is � < � < � +2D. Stand-alone cells would tend to a globally stable solution in the forced stability region. Diffusion splits the regions of the limit cycle and the forced stability. One part of the region of forced stability becomes the region of the diffuse oscillation. In this region, stable solutions become unstable and cells will start oscillating. The actual position of splitting of forced stability depends on the level of diffusion.

The following figures show some typical configurations. Depending on the output of cell pairs, the system will either oscillate or converge to a stable solution. The next figure shows what happens when only one connection has opposite sign of outputs. After a short transient, cells settle down new stable equilibriums.

22

Fig. 15 Cells connected by passive coupling via diffusion. First axis: transient of state variables of the first cell (x1, x2). Second axis: transient of state variables of the second cell (x3, x4). Bottom part: phase portraits of cell’s transient.

The next figure shows what happens when cross-connected cells have opposite sign of outputs. Switching on the connection drives cells out of stable equilibriums and results in oscillating.

Fig. 16 Cells connected by passive coupling via diffusion. First axis: transient of state variables of the first cell (x1, x2). Second axis: transient of state variables of the second cell (x3, x4). Bottom part: phase portraits of cell’s transient.

[�, �, D = 1.0, 0.8, 0.15]

No oscillation because outputs of cell pair (y1,y3) have the same output.

Switch Diffusion ON

1x

3x

2x

4x

[�, �, D = 1.0, 0.8, 0.15 Switch Diffusion ON

Both output pairs (y1,y3) and (y2,y4) are opposite.

1x

3x

2x

4x

23

The following figure shows a similar situation.

Fig. 17 Cells connected by passive coupling via diffusion. First axis: transient of state variables of the first cell (x1, x2). Second axis: transient of state variables of the second cell (x3, x4). Bottom part: phase portraits of cell’s transient.

Op-amp implementation of two-port oscillatory CNN Several possible different operational amplifier (op-amp) implementations of Cellular Neural Networks are known. Here we will show two implementations. The circuits are based on operational amplifiers and RC components. Both circuits are capable to demonstrate the oscillation behavior and awakening oscillation via passive coupling. The circuit shown in Fig. 18 may be the most straightforward cell circuit implementation for CNNs. It consists of the basic CNN cell core, namely, a linear capacitor Cx and a linear resistor Rx, connected in parallel, a linear voltage-controlled current source shown in the left part, and an output sub-circuit with the piecewise-linear output function f(xij) shown in the right part. The dynamics of the CNN cell core is as follows:

and the cell output is )(

,

xy

xx

xxx

vfv

iRv

dtdv

C

=

+−=⋅

The voltage-controlled current source (VCCS) is realized by op-amp A1. It can be shown

that 273

4 vRR

Rix ⋅

⋅−= under the condition

5

76

3

4

RRR

RR += . In this case, the output current

(ix) of the VCCS is independent of the value of the load resistor (Rx), i.e. it works as an ideal VCCS.

[�, �, D = 0.2, 0.18, 0.1 Switch Diffusion ON

1x

3x

2x

4x

Both output pairs (y1,y3) and (y2,y4) are opposite.

24

The piecewise-linear output function is realized by op-amp A2 with the constraint 8 9 10 11

limit8 11

/sat

R R R RE E

R R+ += = , here we assumed limit 1E V= .

Fig. 18 Op-amp implementation of CNN base cell with current generator

The advantages of this implementation are its simplicity, one-to-one mapping of the theoretical model, and diffusion can be implemented by a passive coupling (diffusion coefficient equaling to the conductance of the resistor connected across the cells). Unfortunately, the VCCS is very sensitive to its resistor accuracy, therefore high precision trimmers are required for the precise set up. Therefore, we focused on a different CNN cell circuit implementation shown in Fig. 19. The core of the circuit is the Miller integrator and adder block (left part of the figure). The cell voltage (x1) is the capacitor’s voltage (C1). Observe that the inverting terminal of the op-amp A1 is a virtual ground. The right part of the circuit is the voltage limiter implementing the piece-wise linear output function (same to the previous circuit). The cell equation implemented by this circuit is as follows: The equation perfectly matches to a CNN first-order cell where voltages 1ν and 2ν represent the outputs of other CNN cells.

3

2

2

1

1

111 R

vRv

Rx

dtdx

C −−−=

25

Fig. 19 Op-amp implementation of CNN core cell based on the Miller integrator

Due to its simplicity, this circuit was used for implementing an oscillatory CNN cell as shown in Fig. 20.

Fig. 20 Op-amp circuit implementation of two-port oscillatory CNN

1x

2v

1v

1y

1y−

2y

1y−

2y−

1x

2x 2y

1y 1y−

2y−

26

Dynamics of circuit is as follows: With assumption we can write By introducing time constant (time-scaling factor) RC=τ , the equation becomes dimensionless. with circuit parameters Condition for oscillation 0>> αβ can be written in circuit parameters: The qualitative properties of solutions of this circuit are the same studied earlier except that eigenvalues are time-scaled: with base angular frequency . Realization of diffusion type coupling cannot be implemented by only one resistor but we need a simple differential amplifier.

�

�

�

−−−−−=

−−−−=

22

1

32

2

12

2212

31

2

21

1

11

1111

Ry

Ry

Rx

dtdx

C

Ry

Ry

Rx

dtdx

C

,...11211

11211

RRR

CCC

≡≈≡≈

�

�

�

�

���

�⋅+⋅+−=

�

���

�⋅−⋅+−=

13

12

2

12

11

2

23

11

2

11

11

1

1

1

yRR

yRR

xRCdt

dx

yRR

yRR

xRCdt

dx

( )[ ]

( )[ ]�

�

�

⋅+⋅++−=

⋅−⋅++−=

)()(1)(1

)(

)()(1)(1

)(

1222

2111

tytytxtx

tytytxtx

βατ

βατ

�

�

3

1

2

1

11

,1

,

RR

RR

CR

=

−=

⋅=

β

α

τ

012

1

3

1 >−>RR

RR

[ ]βατ

λ ⋅±⋅= j1

2,1

130

1CR ⋅

==τβω

27

Awakening a whole CNN We will focus on two-parameter oscillation setting and examine their behavior. The conditions for parameters in forced stability region are

0α β> > If cells are separated (no coupling), then each cell will be saturated starting from any initial condition and the system is completely stable. Let us analyze what happens when coupling is switched on via diffusion. The dynamics is

1, 1, 1, 2, 1,

2, 2, 2, 1, 2,

( ) ( ) (1 ) ( ) ( ) ( )

( ) ( ) (1 ) ( ) ( ) ( )ij ij ij ij ij

ij ij ij ij ij

x t x t y t y t D y t

x t x t y t y t D y t

α βα β

= − + + ⋅ + ⋅ + ⋅ ∆ ��= − + + ⋅ − ⋅ + ⋅ ∆ �

�

� or detailed with a 4-

connectivity convolution kernel is

1, 1,

1, 1, 1, 2, 1, , 1 1, , 1, , 1

1, 1,

2, 1,

2, 2, 2, 1, 2, , 1 2, , 2, , 1

2, 1,

1(1 ) 1 4 1

1

1(1 ) 1 4 1

1

i j

ij ij ij ij i j i j i j

i j

i j

ij ij ij ij i j i j i j

i j

y

x x y y D y y y

y

y

x x y y D y y y

y

α β

α β

−

− +

+

−

− +

+

� �� �� �� �= − + + ⋅ + ⋅ + ⋅ − ∗ � �� �� �� �� �

� �� �� �� �= − + + ⋅ − ⋅ + ⋅ − ∗ �� ��� �� �

�

�

�����

.

Now, let us examine what the condition is for dynamic active patterns, i.e. cells become active and leave the saturated regions because cells form unstable patterns. Let rN denote the number of cells having a saturation value opposite to the central cell. The solutions are ( 0=x� )

1

2

1

2

1

2

1

: 1 4 (4 )

1 4 (4 )

: 1 4 (4 )

1 4 (4 )

: 1 4 (4 )

1 4 (4 )

: 1 4 (4 )

s r r

r r

s r r

r r

s r r

r r

s r r

Q x D DN D N

x D DN D N

Q x D DN D N

x D DN D N

Q x D DN D N

x D DN D N

Q x D DN D N

α βα β

α βα β

α βα β

α β

++

−−

+−

−+

�= + + − − + − �

= + − − − + − �

�= − − − + + − − �

= − − + + + − − �

�= + − − − + − �

= − − − + + − − �

= − − + + + − −

2 1 4 (4 )r rx D DN D Nα β��

= + + − − + − �

Conditions for dynamic active patterns are (e.g. if y = 1, then x� should be negative and vice versa.)

28

: 1 4 (4 ) 1

1 4 (4 ) 1

: 1 4 (4 ) 1

1 4 (4 ) 1

: 1 4 (4 ) 1

1 4 (4 ) 1

: 1 4 (4 ) 1

1

s r r

r r

s r r

r r

s r r

r r

s r r

Q D DN D N

D DN D N

Q D DN D N

D DN D N

Q D DN D N

D DN D N

Q D DN D N

α βα β

α βα β

α βα β

α βα

++

−−

+−

−+

�+ + − − + − < �

+ − − − + − < �

�− − − + + − − > − �

− − + + + − − > − �

�+ − − − + − < �

− − − + + − − > − �

− − + + + − − > −+ 4 (4 ) 1r rD DN D Nβ

��

+ − − + − < �

Now, examine the worst case when there is only one neighboring cell which has opposite sign to the central cell, i.e. 1rN = . For all of the four cases we get the following two conditions for active dynamic patterns (no solution in the saturation region).

22

D

D

α βα β

< + ��< − �

Because we are looking for parameters with condition α β> , then we can write it in a compact form:

2Dβ α β< < + (25)

This is exactly the same condition as (24) derived for two cells. The 8-connectivity case yields this condition as well. For a four-parameter case, we can derive a similar condition:

1 1 1 1

2 2 2 2

2

2

D

D

β α ββ α β

�< < + �

< < + � (26)

In the linear (and also in the semi-linear) region there is no stable solution because 0>α . Explanation for this behavior is heuristically the as follows. The diffusion term smoothes the difference among the cells but when cells have similar value, then the term of diffusion is almost zero, therefore, the network starts behaving as a standalone cell. And we already know in this case cells tend towards the saturation region. In the saturation region the only stable solution is if the whole network has the same value in each layer. We distinguish four cases, cells of a layer can be completely saturated at either +1 or -1. In terms of color scheme this means that layer 1 and layer 2 may either be black-black, white-white, black-white, or white-black. Otherwise it is unstable. Cells will oscillate and spiral waves will appear. Fig. 21 shows the snapshot of a network with the following parameters: [�, �, D = 1.0, 0.5, 0.26]. Without diffusion, the system would be completely stable for any initial condition. With diffusion except the previous extreme cases, the

29

system does not have any stable solution. Spiral waves will form. Snapshots of two different parameter cases are shown in Fig. 21 and Fig. 22, respectively.

1-st Layer

Time: 200.0010 20 30 40 50 60

10

20

30

40

50

60

2-nd Layer

10 20 30 40 50 60

10

20

30

40

50

60

Fig. 21 Special role of diffusion: triggers spiral wave generation despite the fact that standalone cells would rather tend to a globally stable equilibrium than oscillating together. Parameters are [�, �, D = 1.0, 0.5, 0.26]. Condition: 2Dβ α β< < +

1-st Layer

Time: 200.0010 20 30 40 50 60

10

20

30

40

50

60

2-nd Layer

10 20 30 40 50 60

10

20

30

40

50

60

Fig. 22 Spiral waves appear where standalone cells would not oscillate. Diffusion makes it possible. Parameters are [�, �, D = 1.0, 0.8, 0.15].

Fig. 23 and Fig. 24 show the transient of an oscillatory CNN cell which would tend to a globally stable equilibrium if no diffusion (passive coupling) exists among cells.

30

0 20 40 60 80 100 120 140 160 180 200-3

-2

-1

0

1

2

3

Time (τCNN)

Sta

te A

mpl

itude

Cell transient

X2 X1

Fig. 23 State variables of an oscillating cell are shown here. An isolated cell has a globally stable equilibrium but diffusion (passive coupling among cells) induces oscillation. Parameters are [�, �, D = 1.0, 0.5, 0.26]. Condition: 2Dβ α β< < +

0 20 40 60 80 100 120 140 160 180 200-3

-2

-1

0

1

2

3

Time(τCNN)

Sta

te A

mpl

itude

Cell transient

X1

X2

Fig. 24 Oscillating cells due to diffusion. Parameters are [�, �, D = 1.0, 0.8, 0.15]. Condition:

2Dβ α β< < +

Let us derive condition for parameter space T − ∆ from condition 2Dβ α β< < + . As we already know, 2T α= and 2 2α β∆ = + . We can write:

31

( )22 2 2 2Dα β α α∆ = + > + − , substituting 2Dβ α> − . Expressing � with T, we get

( ) ( )2 2/ 2 / 2 2T T D∆ > + − . Finally, we get the condition for diffuse induced oscillation:

( )2 212 2

2T D D∆ > − + (27)

Fig. 25 shows equilibrium state classification and region of diffuse induced oscillation. The condition is valid for T>4D. We raised 2Dα − to power 2, thus for region [0, 4D] the former 1/4T2 is valid.

Fig. 25 Parameter space for diffuse oscillation. Stand-alone cells would tend to a globally stable solution in the forced stability region (between blue and red lines), see also Fig. 5. But diffusion splits this region and stable solutions become unstable and cells will start oscillating. The actual position of splitting of forced stability depends on the level of diffusion.

Conclusion Firstly, we have studied synchronization mechanism among cells in reaction diffusion systems and similarities to basic pulse synchronization technique have been shown. Although the diffusion is a kind of a smooth process, synchronization takes place via spikes and there is no signal flow (current flow in our case) among cells in most part of time. Secondly, we have shown that passive coupling among completely stable cells

32

might produce very interesting dynamical behavior, i.e. previously inactive (“dead”) cells will become active and start oscillating. We needed to ensure that the cell should be locally active, except at stable equilibrium points where small signal behavior is locally passive. Even though the presented system was completely stable and the stable equilibrium points were locally passive, the passive coupling (diffusion via resistive grid) was capable to drive out the system from its “dead” state and the system started oscillating. This behavior was shown by connecting two second-order dynamical systems where each standalone system was completely stable. Mechanism was thoroughly discussed and the operational amplifier implementation has verified this phenomenon.

References [1] Smale S. The mathematical model of two cells via Turing's equation. Lectures on

Mathematics in the Life Sciences 1974; 6:17-26. [2] Chua LO. Passivity and Complexity. IEEE Transactions on Circuits and Systems I:

Fundamental Theory and Applications 1999; 46(1):71 – 82. Digital Object Identifier 10.1109/81.739186

[3] Chua LO. CNN: a Vision of Complexity. International Journal of Bifurcation and Chaos 1997; 7(10):2219-2425.

[4] Chua LO. CNN: A Paradigm for Complexity. World Scientific series on Nonlinear Science 1998; 31, World Scientific.

[5] Chua LO, Yang L. Cellular Neural Networks: Theory. IEEE Trans. on Circuits and Systems 1988; 35:257-1272.

[6] Chua LO, Yang L. Cellular neural networks: Applications. IEEE Trans. on Circuits and Systems 1988; 35:1273-1290.

[7] Chua LO, Roska, T. The CNN paradigm. IEEE Trans. Circuits Syst. I 1993; 40: 147-156.

[8] Chua LO. Desoer CA, Kuh ES. Linear and Nonlinear Circuits, McGraw-Hill Companies 1987; ISBN: 0070108986

[9] Chua LO, Roska, T. Cellular neural networks and visual computing - Foundations and applications. Cambridge University Press 2002.

[10] Arena P, Baglio S, Fortuna L, Manganaro G. Self-Organization in a Two-Layer CNN. IEEE Trans. on Circuits and Systems – I. Fundamental Theory and Applications 1998; 45(2):157-162.

[11] Krinsky VI, Ed. Self-Organization: Autowaves and Structures Far from Equilibrium. Springer-Verlag 1984.

[12] Mikhailov AS. Foundation of Synergetics I. Distributed Active Systems. Springer-Verlag 1990.

[13] Tyson J. The Belousov-Zhabotinskii Reaction. Springer-Verlag 1976. [14] Alcantara F, Monk M, Signal propagation during aggregation in the slime mold

Dictyostelium discoideum. Journal of Gen. Microbiology 1974; 85:321-334. [15] Gray RA, Jalife J. Spiral waves and the heart. International Journal of Bifurcation

and Chaos 1996; 6:415–435. [16] Dogaru R, Chua LO. Edge of chaos and local activity domain of Fitzhugh-Nagumo

equation. International Journal of Bifurcation and Chaos 1008; 8(2):211-257.

33

[17] Min L, Crounse K, Chua LO. Analytical criteria for local activity of reaction-diffusion CNN with four state variables and applications to the Hodgkin-Huxley equation. International Journal of Bifurcation and Chaos 2000; 10(6):1295-1343.

[18] Munuzuri AP, Perez-Munuzuri V, Gomez-Gesteira M, Chua LO, Perez-Villar V. Spatiotemporal structures in discretely-coupled arrays of non-linear circuits: a review. International Journal of Bifurcation and Chaos in Applied Sciences and Engineering 1995; 5(1):17-50.

[19] Civalleri PP, Gilli M. A spectral approach to the study of propagation phenomena in CNNs. International Journal of Circuit Theory and Applications 1996; 24:37-48.

[20] Gilli M. Analysis of periodic oscillations in finite-dimensional CNNs through a spatio-temporal harmonic balance technique. International Journal of Circuit Theory and Applications 1997; 25:279-288.

[21] Petras I, Gilli M. Complex dynamics in one-dimensional CNNs. International Journal of Circuit Theory and Applications 2006; 34(1):3-20. (Special Issue on CNN Technology)

[22] Xavier-de-Souza S, Suykens JAK, Vandewalle J. Learning of spatiotemporal behaviour in cellular neural networks. International Journal of Circuit Theory and Applications 2006; 34(1):127-140. (Special Issue on CNN Technology)