the formation of cell assemblies

BULLETIN OF MATHEMATICAL BIOPHYSICS

VOLUME 28, 1961

THE FORMATION OF CELL ASSEMBLIES

HAROLD WHITE COMMITTEE ON MATHEMATICAL BIOLOGY

THE UNIVERSITY OF CHICAGO

A simple model is presented for the formation of functional groups in a random neural net. They show the following characteristics: 1. They can maintain autonomous activity which might serve as temporary memory traces. 2. Early in the process of formation they become resistant to contraction. 3. Later they become resistant to expansion. 4. Nearby groups inhibit one another. 5. Two groups may contain some cells in common.

There are three assumptions underlying this work. The first is that some of the parts of the nervous system that are involved in learning processes can meaningfully be treated as though they were random nets . The character is t ic feature of such a net is that, given any two ce l l s in the net, the probability that an axonal process of the first makes a synaptic connection with the second is a

function only of the dis tance between the two cel ls . The mathematical s tudy of such nets was begun by Shimbel and Rapaport (1948), who have s ince published many papers on the topic. An independent treatment has been given by Beurle (1954). Aspects of the connect ive patterns, in particular the dendritic f ields, of cel ls in the cerebral cortex, have been d i scussed by Sholl (1956).

The second assumption is that learning in the mature mammal occurs in two phases . Firs t an experience creates a temporary trace within the organism. Then this temporary trace effects a permanent change. Properly this should be called a theory rather than an assumption, for it has been supported by strong evidence. E.g., Duncan (1949) studied the retroact ive interference of electro-

coma on avoidance learning in rats . Some of the control subjects were shocked through the legs instead of through the ears; this type of shock seemed to be more traumatic but it did not produce

43

44 HAROLD WHITE

unconsciousness and it caused relat ively little interference. Dun- cart's data indicated that it took about fifteen minutes for the permanent effect of a single experience to be fully es tabl ished.

The third assumption concerns the nature of the permanent change. There does not seem to be any good evidence on this matter. A variety of hypotheses have been proposed. We shall adopt the suggestion made by Hebb (1949), Ecc les (1953) and others, that the locus of the permanent change is at the synapse: if one cel l repeatedly fires jus t before another, with which it has a connection, then the influence of the first upon the second becomes more excitatory.

If the permanent change is of this nature, then the temporary trace must cons is t , at leas t in part, of pers is tent act ivi ty in particular loci. The mechanism that has often been proposed (e.g., Rashevsky, 1937) for such a temporary trace is the se l f re-exciting cycle: for example, cell A exci tes B, B exci tes C and then C exci tes A again. This example is over-simplified; in general, a cell needs to be stimulated by a number of others within a suff ic ient ly short interval of time in order to fire. Instead of a simple chain, a group of cel ls with many interconnections is required. It has been shown in isolated s labs of the cerebral cortex of the cat that some cel ls can maintain act ivi ty for short periods by this mechanism of mutual excitat ion (Burns , 1958, page 28). An aggregate of cel ls capable of maintaining act ivi ty has been given the name " c e l l a s s e m b l y " by Hebb (op. cir.). Hebb uses this as a central concept in a broad scheme of psychological theory. He ci tes evidence that in the sensori ly deprived mammal (that presumably has not been able to develop the needed cell assembl ies ) learning is very inefficient.

There are certain diff icult ies with the concept of the cell assembly. Beurle (op. cir.) has shown that act ivi ty will tend to spread in waves across a random net, because of the refractory period of the cel ls . Rosenblat t (1958) argues that if there were a number of cel l assembl ies within a net, with the passage of time these assembl ies would coa lesce , by Hebb 's hypothesis of in- creasing synaptic eff iciency. Milner (1957) has suggested a scheme for the development of cel l assembl ies that meets these obiect ions. It is the purpose of the present work to explore mathe- matically a simpler model. This model, in common with Milner's, involves inhibitory as well as excitatory p rocesses . Evidence on

CELL ASSEMBLIES 45

inhibition in the brain stem and cerebral cortex has been given by Purpura (1958).

Let x be a vector denoting position in a random net. Let f(x) be the input at x: the rate at which impulses from cells outside the net are impinging upon a cell at x. Let g(x) be the output at x: the rate at which a cell at x is firing. Let e(x) be the net excitation at x, due to both the input and the act ivi ty of cells within the net. Each of these quanti t ies, f(x), g(x) and e(x), is a local aver- age, each varies with time and each could have the units of impulses per second. Let p (x) be the density of cells at x. Final ly, let O(x, x') be a parameter denoting the influence of act ivi ty at x" upon a cell at x. Then

= f(z) + ] p (z') g(z') o(z,x') dz', (1) E(X)

where the integral extends over the net. In general O(x, z') depends upon two factors. The first is the

distance between x and x'. In this paper, however, we shall be concerned with a few groups of cells in neighboring or overlapping regions and the spat ial factor will not be of importance. The other factor, which will be important here, is the amount of nearly simultaneous act ivi ty that has taken place at x and x'. Following such a c t i v i t y , O(x,x') increases by Hebb's hypothesis. 0 is a mono- tonic function of time. If there has been no simultaneous act ivi ty at x and x', O(x,x') has its leas t possible value. Since neural inhibition is known to be important, we suppose this minimum value to be negative. The idea that 0 can change from a negative to a positive value does not imply that any single cell changes the nature of its influence from inhibitory to excitatory. It means that ini t ial ly the net effect of activity of the one region upon the other is inhibitory and that with simultaneous act ivi ty the excitatory con- nections can be permanently potentiated so that this net effect becomes excitatory. The initial , minimum value of 0 will be denoted by - ] . It does not seem reasonable for 0 to increase without bounds; we suppose 0 to have a maximum attainable value, Oz.

Essent ia l ly what happens in this model is that the types of activity stimulated by various inputs become increasingly stereo- typed and fixed. As will be shown below, if a particular input pattern has been prolonged and repeated enough to cause suff icient " t r a in ing" among the stimulated cells , then any similar input, even though it may stimulate more or fewer cel ls ; will cause the

46 HAROLD WHITE

same pattern of act ivi ty; the cel ls that have been " t r a i n e d " together (that is, have previously had persis tent simultaneous activity) will now stimulate one another but inhibit other cel ls in the vicinity. An input pattern that is continuously graded over space tends to el ic i t act ivi ty with sharp demarcations. We shall d i scuss discrete groups of ce l l s ; our resul ts will show to what extent the boundaries of the groups are actual ly fixed and to what extent they are variable. If the cells of the ith group have been trained together, but none have been trained with any cel ls in the other groups being considered, equation (1) can be rewritten,

e~ = fl + Oinig~- i nv gv, (2) m

where ni is the number of cel ls in the ith group. It is conceivable that the variation in time of f (x) might be too

sharp to permit the development of anything like a s teady s ta te . However, the analys is of the present paper will be limited to such relat ively simple conditions. In the s teady s ta te , the output at any place is a certain function of the net excitat ion there: g = r (e). This function has been d i scussed by Rapaport (1950). It depends upon the following properties of the individual cell: the refractory period 3, the period of temporal summation a, and the threshold h,

If h > 2, the curve has a sigmoid flexure; for large values of e, the curve is asymptotic to 1/6. The inverse function r gives the amount of excitat ion needed to maintain the output at a level g. In the d iscuss ion below we shall make use of the following linear approximation:

6xI~

4x lO a

Q.

~s 2xlOa

I I I l 0 0 25 5O 75 IO0

9 ( impu lses per second)

F I G U R E l . ~ - - l ( g ) , w i t h t h e l i n e a r a p p r o x i m a t i o n sg + k , for t h e c a s e = 1 0 - 2 s e e . , a --- 10 - 8 s e c . , a n d h -- 5.

CELL ASSEMBLIES 47

r 1 (g) = 8 g + k, (3 )

where 0 < g < 1/3. In order to get a rough idea of the dependence of s and k upon the parameters ~, a and h, r was plotted for

various values of these parameters and it was found that s is about 2 (]~/a) ~ and k is h2"a/40~~ ~

The first case to be d iscussed will be the simplest: a group of n cells which have been trained together, and none others in the vicinity, are stimulated by an input f. Now

e= f + eng . (4)

In Figure 2a, f + 0rig is superimposed upon r The two curves intersect when g = g 0 . If g < g 0 , then f + 0 n g > r that is, the excitation e is greater than the excitation needed to maintain the output g and g will increase. If g > go, then f + Ong < r that is, the excitation e is less than the excitation needed

to maintain the output g and g will decrease. So there is s table equilibrium at go.

When the input ceases , that is, when f falls to zero, then go falls to zero. But if the input is suff icient ly prolonged and repeated, the parameter 0 may attain a value great enough that there are two more points of intersection of Ong and r It can be seen in Figure 2b that zero and g'" are values of stable equilibrium and that g" is a threshold. That is, if g < g" then the output drops to zero, but if g > g" then the output moves toward g" . Sup- pose the group to be init ial ly at rest , with g = 0; an input f can push g over to its other stable value g'" and even after f falls back to zero the ~oup will maintain this level of activity, until it is af- fected by inhibition or fatigue. Rapaport named this the " i g n i t i o n "

,

~0

FIGURE 2a.

0 ~' g"

FIGURE 2b.

48 HAROLD WHITE

phenomenon of random nets (1952). The product On of the present discussion corresponds to the parameter a, the axon densi ty, in Rapaport 's paper. The least value of On such that Ong has a non- zero intersection with r (g) is the least value such that ignition is possible. This critical value will be denoted by c. The prob- lem of determining c as a function of h was studied by Trucco (1952) for a model in which the cells could act only at unit inter- vals and there was no temporal summation. In the present model, c - - s + k ~ .

The sizes of the assemblies that develop depend upon the inputs f. We might try identifying f with, for example, a sensory after-discharge, but there does not seem to be pertinent evidence. ~n this model, the ease with which an assembly can be formed varies with the s ize of the group. On the one hand, the smaller the number n, the greater 0 must become in order that On >-- c, the condition that the group be ignitable. (The least value of n such that this condition can ever be met is c/O,~. For the case illus- trated in Figure 1, c = 36; if for example, 0m - 0.1, then the small- est possible assembly would contain 360 of these cel ls . ) On the other hand, the greater the number n, the more the initial inhibition, - i n , that f has to overcome in training the assembly.

In the second case to be considered, n~ + nu cells have been given some degree of training together and only nl of the cells are stimulated by an input f. At equilibrium,

r = f + O n l g l + ~gn2g2, (5)

r = Onl gl + On2 92. (6)

When the input f drops to zero, the excitation is the same through- out. Therefore, as long as 0 is uniform, the n 1 cells cannot be ig-

nited unless the whole set of nl + nu is. Only if there has been so little training that 0 is still negative and act ivi ty of the nl

ceils inhibits the n2 ceils can an assembly form of the nl cells only.

In the third case nl cells are trained together and an input f st imulates nl + n2 cells simultaneously. Using the linear approximation, the equilibrium conditions are

s g l + k = f + 01 n l g l - in292 , ( 7 )

sg2 + k = / - i n l gl - in2 g2. (8)

CELL ASSEMBLIES 49

The solution of this sy s t em of equation is

g l = ( f - k ) s / A ,

g2 (f - k ) [ * , (O, + i )n , ] /A , ; (9)

h ( s - 01n l ) ( s + in2) - i 2n ln2 .

According to this solut ion , g2 < 0 if 01 nl > s - in1 (assuming that f - k and h are pos i t i ve ) . Of course g2 cannot be negative. This means that if 01 nl > s - in1, the n2 cells are inhibited and g2 = 0. Thus if there has been sufficient training, the assembly is resistant to expansion. However, just as the linear approximation is not valid when it gives gu a negative value, neither is it valid when it gives gl a value greater than 1/3. If f is very large, gl approaches 1/3; then, from equation (8),

= f - k - (i. ( lO) S + i n 2

Thus no matter how well trained the nl cel ls are, no matter how large 01 is, a suff icient ly large f can force simultaneous act ivi ty in the n2 cel ls . If this simultaneous act ivi ty is suff icient ly prolonged and repeated, it will cause incorporation of the n~ cells into the assembly.

To summarize what was found in the second and third cases : After a group of cel ls has had enough activity that 0 > 0, it is impossible for an assembly to be formed from only a part of the group. Later in the training, when 0 attains the value ( s / n ) - ] , it becomes difficult , although not impossible for a larger assembly to form with the given group as a part.

In the fourth case two groups of cel ls , Of s izes nl and n2, have been trained separately and receive an input f s imultaneously. The approximate equilibrium conditions are

s g l + k= f+ 01nl gl - jn2g2, (11)

sg2 + k = f - ] n l g l + 02n2g2. (12)

Solving (11) for gl and (12) for g2,

gl = f - k - 1n2 g2, (13) s - 01 nl

g 2 = f - k - ] h i ~71 (14) s - 02 n2

50 HAROLD WHITE

(Of course these equations have no util i ty if 0 2 s / n for either group.) Considering gl in equation (13) to be a function of g2, it can be seen that the value of 01 affects the gl - intercept and the s lope of the function, but that (as long as 01 < s / n l ) the g2-intercept is unaffected: if g2 > ( f - k ) / ] n 2 , the nl ce l ls receive at leas t as much inhibition as excitat ion and gl = 0. Similarly, when g2 in equation (14) is considered to be a function of gl , the value of 02 affects the g2-intercept but not the gwintercept . All of this is pictured in Figure 3. In 3a, 01 and 02 are both small; in 3b they have larger values .

Suppose g~ and g2 to have values such that the point (g l ,g2) lies above the graph of equation (13). This means that the excitation of the n~ cel ls (the right s ide of equation (11)) is less than that required to maintain gl (the left s ide of equation (11)) and gl will decrease . If (g l ,g2) is below the graph of (13) there is more excitat ion than needed and gl will increase. A similar analys is applies to g2. This is i l lustrated by the arrows in Figure 3. Only the circled points are points of s table equilibrium. The condition that there be a s table point where neither gl nor g2 vanishes is that 0 < ( s / n ) - ] for both groups. If that is the case , prolonged simultaneous stimulation can cause consolidat ion of the two groups. But if the groups have had enough training, act ivi ty in one can completely inhibit the other. The group with the larger value of On tends to monopolize the act ivi ty. Alternatively, if On is about the

g~

f-k J na

f-k $-ezn ~

$-01n I JI11

F I G U R E 3 a .

gl

CELL ASSEMBLIES

f-k g~cL s --~"~ nz ~

f -k f -k.. j nl $-oinl

FIGURE 3b.

51

same for the two groups but f is not uniform, the two groups form a mechanism somewhat like that proposed by Landahl (1938) for dis crimination.

The si tuation is much the same if the two groups are ignitable.

Let 01 = 02 = Ore, n t >_.n2 and 0 m n 2 ~ c . If gl and g2 were almost 1/3, then

0m n2 - i n 1 82 = f - j n l g l + O m n 2 g 2 = [ + . (15)

6

The act ivi ty of a group can remain at almost 1/3 only so long as the excitation is not less than ( s / 6 ) + k = c / 6 . If inhibition causes the excitat ion to fall below this , the activity decreases , the excitation drops further and if the inhibition continues the act ivi ty vanishes. So act ivi ty can continue in the second group when f = 0 only if

Om n2 -- j n l c >- - . (16)

6 6

If assemblies are to endure without too much coalescence, the above condition should be unlikely even when the two groups are very large and about the same size. This implies that Om< J (i.e., that the final value of 0 is not greater than the absolute magnitude of its initial negative value.)

The discussions of cases two, three and four imply that the cells of a given assembly act all together or not at all, that the assem-

52 HAROLD WHITE

blies are atomic. The fifth case shows an except ion to this. Sup-

pose that nl + n2 cel ls have been trained together and that n2 + n3 cel ls are stimulated s imultaneously with an input f. That is, the n2 cel ls const i tu te the intersect ion of the se t of trained cel ls with the se t of ce l ls receiving the input. By equation (2) the values of the exci ta t ion are as follows:

~1 = O ( n l g l + n292) - jn~g3,

~2 /+ O(nlgl + n292) - / n a g s , ~ (17)

sa / - j ( n l g l + n2 92 + n~).

If 0n2 > jna, then On292 > jna ga; the nl ce l ls will be act ivated

and if f is not too large the ns cel ls will be inhibited. On the other hand, the nl cel ls may receive more inhibition than excitation. To invest igate this possibi l i ty , we se t gl -- 0, s2 = sg2 + k and ea -- sga + ]% solve for g9 and ga (as for gl and g2 in case

three) and se t On2 g2 < jna gs:

o n2 ( / - k ) s / A < in8 ( / - ]~)Is - (e + j)n2]/A. (18) Solving the inequali ty for n2,

- - �9 0 9 )

If (19) is sa t i s f ied , then upon the application of this type of input

the nl cel ls will be inhibited and eventual ly an assembly can be formed of the n2 + ns ce l l s . Let the training parameter of this as -

sembly be 0". If 0"n2 < in1 it is sti l l possible for the nl + n2 assembly to be act ivated without act ivat ing the n~ + na assembly. Thus two assembl ies can contain a limited number of cel ls in common without losing their separate ident i t ies .

In the opinion of the author, an attempt direct ly to relate the phenomena d iscussed in this paper with psychologica l evidence would be misleading. If there are such things as cell assembl ies , evidence relating to the " law of mass a c t i o n " indicates that it is the interaction of great numbers of them that influences behavior.

There are about 10 l~ neurons in the nervous system; if one in ten were incorporated into an assembly and the mean assembly s ize were 104, there would be 10 5 assembl ies .

The author grateful ly acknowledges the guidance of Professor H. D. Landahl and the cri t icism of Dr. Peter Greene.

C E L L ASSEMBLIES 53

T h i s r e s e a r c h was s u p p o r t e d by the U.S. Air F o r c e unde r con-

t r a c t A F 49(638)-414 .

LITERATURE

Beurle, R . L . 1954. "Ac t i v i t y in a Block of Cei l s Capable of Regener- at ing I m p u l s e s . " RRE Memo 1042, Ministry of Supply, Malvern, England.

Burns, B . D . 1958. The Mammalian Cerebral Cortex. London: Arnold. Duncan, C . P . 1949. " T h e Retroact ive Effect of E lec t roshock on Learn-

ing. '~ J. Comp. Physiol. Psychot. , 42, 32-44. E c c l e s , J . C . 1953. The Neurophysiological Basis of Mind. Oxford:

Clarendon. Hebb, D. O. 1949. The Organization of Behavior. New York: Wiley. Landahl , H. D. 1938. " A Contr ibut ion to the Mathematical Biophys ics

of Psychophys i ca l D i sc r imina t i on . " Psyehometrika, 3, 107-125. Milner, P . M . 1957. " T h e Cel l Assembly : Mark I I . " Psychol. Rev . , 64,

242-252. Purpura, D. P. 1958. " T h e Organiza t ion of Synaptic E l ec t rogenes i s in

the Cerebral Cor tex ." In The Reticular Formation of the Brain, edi ted by H. H. Jasper . Boston: L i t t l e , Brown.

Rapaport , A. 1950. "Con t r ibu t ions to the P robab i l i s t i c Theory of Neu- ral Nets : I I . " Bull. Math. Biophysics, 12, 187-197.

1952. ~ 'Ignit ion Phenomena in Random N e t s . " Bull. Math. Biophysics, 14, 35-44.

Rashevsky , N. 1937. "Mathemat ica l Biophys ics of Cond i t i on ing . " Psychometrika, 2, 199-209.

Rosenbla t t , F . 1958. The Perceptron. Report No. VG-1196-G-1 , Of- fice of Techn i ca l Services , Washington.

Shimbel, A. and Rapaport , A. 1948. " A S ta t i s t i ca l Approach to the Theory of the Central Nervous Sys tem." Bull. Math. Biophysics, 10, 41-55.

Sholl, D . A . 1956. The Organization of the Cerebral Cortex. New York: Wiley.

Trucco, E. 1952, " T h e Smal les t Value of the Axon Dens i ty for which ' I gn i t i on ' can Occur in a Random Ne t . " Bull. Math Biophysics, 14, 365-374.

RECEIVED 5-20-60

the formation of cell assemblies

Documents