PRACTICAL APPLICATIONOF THE LEARNING CURVE

MODELING

Dr. Igor I. Stepanov, MD, PhD

Leading research fellow, Department of Neuropharmacology,Institute for Experimental Medicine,

the Russian Academy of Medical Sciences,St. Petersburg, Russia

Learning is an important component ofanimal and human behavior for survival inpermanently changing environment.

An informative measure of learning process is the learning curve.

The mathematical analysis of animal and human learning had been undertaking since the end of eighteenth century.

In 1879, the eminent psychologist Hermann Ebbinghaus used a hyperbolic function to model an observed retention curve.

kc*)t(Ln

k100R

The earliest instance of fitting a mathematical

function to psychological data

Learning curve for many simple learning paradigms can be modeled with any negatively accelerated

monotonous smooth function

Hyperbolic function Arc cotangent function

Empirical mathematical functions

Empirical mathematical functions

Logarithmic function

The error function that is the integral of the Gaussian distribution

Differential equationsOther researches proceed from the assumption that the learning rate is proportional to the difference between the physiological limit of associative strength and the current value of associative strength.

This assumption leads to the first order linear differential equation

where a is a constant of proportionality.

)( yyadx

dy

Initial value — y0 might take negative value that is incorrect from psychological point of view.

Xaeyyyy )( 0

Solution of the differential equation is an exponential function

An example of modeling learning 16 Russian words is given on this graph.Here X is a trial number, y0 — the initial number of recalled words before learning at x = 0, y∞ — the asymptotic value of recalled words, a — the learning rate.

System analysis approachto the learning curve modeling

To avoid discrepancy and find most suitable for practical applications model we proceed from the system theory. A system under study is treated as "a black box", its internal structure is unknown, and behavior of the system is analyzed in the terms of input (F) and output (y) signals.

If the input signal F is equal to zero at x<0 and is equal to F1 at x0, it is called “the step function”. During learning, the reinforcement or reward or presentation of a list of words acts as a stepwise input signal, starting from the first trial. In this case X is trial or session number. Thus, we assess a learning measure the first time at the first trial or session.

Transitional process in the first order linear system

Transition of the first order linear system from initial to final status is described by the first order differential equation

Here k is a coefficient of proportionality. When x tends to infinity the

system approaches its asymptotic steady status that is Having defined y∞kF1, we can rewrite the differential equation is the

form

The rate of the transition process in the first order linear system is proportional to the difference between the asymptotic and current value of the system’s output signal. This is the same equation that was proposed by C. Hull, R. Rescorla and A. Wagner and others. However, we got this equation from the system theory.

1kF)x(ydx

)x(dyT

1kF)x(y

))x(yy(T

1

dx

)x(dy

DESCRIPTION OF OUR MODEL

We adopted the first order transitional function for modeling the learning curves in the form

where X is a trial or session number and Y is the quantity of correct responses. The parameters are: B2 — the learning rate; B3 — the value of correct responses at X = 1 (the first session or trial); B4 — the asymptotic value of correct responses at X = Infinity. Coefficient B2 is inverse value of the time constant - T, so the higher B2 value the faster is the learning rate and fewer repetitions is necessary to reach the asymptote.

)1(43 )1(2)1(2 XBXB eBeBY

LEARNING CURVES IN HONEY BEES, Apis mellifera L., EXPOSED TO PESTICIDES

Here is an example how our model can be applied to characterize so-called ‘‘green products’’ such as essential oil based pesticides which often make claims of being harmless to honey bees. Prof. Abramson studied the effect of insect growth regulators Confirm2F (Tebufenozide) and Dimilin (Diflubenzuron) on the acquisition of honey bee Proboscis Extension Response learning.

Effect of the Confirm®2F on bee learning

Circles – control group, B2 = 1.77; B3 = 11; B4 = 94.Squares – Confirm 16 µg/bee, B2 = 1.02; B3 = 3; B4 = 92.Diamonds – Confirm 24 µg/bee, B2 = 0.32; B3 = 11; B4 = 75.Triangles – Confirm 32 µg/bee, B2 = 0.24; B3 = 9; B4 = 74.

Confirm decreases the learning rate in each dose (p < .04) and reduces ability to learn in the dose 24 µg/bee (p = .006).

Effect of the Dimilin® on bee learning

Circles – control group, B2 = 1.03; B3 = 3; B4 = 86.Squares – Dimilin 16 µg/bee, B2 = 0.92; B3 = 1; B4 = 66.Diamonds – Dimilin 24 µg/bee, B2 = 0.69; B3 = 3; B4 = 60.Triangles – Dimilin 32 µg/bee, B2 = 0.50; B3 = 6; B4 = 69.

Dimilin reduces ability to learn in each dose (p < .005) and decreases the learning rate in the dose 24 µg/bee (p = .046) and the dose 32 µg/bee (p = .020)

These results suggest that both Confirm®2F and Dimilin® are dangerous to honey bees though both chemicals are considered ‘‘harmless’’ to honey bees.

LEARNING CURVES IN WISTAR RATS TRAINED IN A 3-ARM RADIAL MAZE

This is an example of a positive food conditioned reflex where the learning curves are descending. A learning paradigm lies in teaching a hungry Wistar rat to visit a bright goalbox to get food — a small piece of cheese, in other words, to run against rat natural preference for darkness.

Each session lasts until 20 pieces of cheese are eaten. Thus getting food is not correct conditioned response. On the other hand, the number of errors - visits to a dark arm, is a stochastic variable that tends to zero during consecutive sessions. So, the learning curve must be a function of the number of errors. This learning paradigm might be used to study subspecies differences in the learning curves. Besides, it is useful for study effect of substances on learning.

A Neural Cell Adhesion Molecule–Derived Peptide Restores Memory Retention Impaired With A25-35 in Rats Trained in a 3-arm Radial Maze

Initial training.Circles — the control group, B2 = 0.60; B3 = 16.69; B4 = 0.99.Squares — the Аβ-injected group, B2 = 0.55; B3 = 15.77; B4 = 1.43. Diamonds — the FGL-injected group, B2 = 1.06; B3 = 18.23; B4 = 1.41. Triangles — the Аβ- and FGL-injected group, B2 = 0.52; B3 = 14.44; B4 = 0.38

Re-acquisition.Circles — the control group, B2 = 0.68; B3 = 3.90; B4 = 1.16.Squares — the Аβ-injected group, B2 = 0.85; B3 = 6.82; B4 = 0.59. Diamonds — the FGL-injected group, B2 = 0.86; B3 = 2.39; B4 = 0.46. Triangles — the Аβ- and FGL-injected group, B2 = 2.22; B3 = 3.50; B4 = 0.79.

1. The Аβ impairs retention (p<.001). 2. The FGL peptide improves retention (p<.02). 3. The FGL peptide restores retention impaired by Аβ up to the control value.

MODELING THE LEARNING CURVES IN HUMANS

Free recall memory tests are mainly used to assess the learning curves in humans. The model's validity and efficiency depends on how a memory test is constructed.

The number of trials should be equal or more five trials to ensure a proper assessment of the asymptotic level of learning.

The number of objects in the list should exceed short-term memory span that is 7±2 by G. Miller (1956). Many individuals are able to learn and recall up to 16–17 words during five trials (Zimprich & Rast, 2009).

Not all memory tests fulfill these requirements. For example, the Wechsler memory scale word list I consists of 12 words, but only 4 trials (Wechsler, 1997); the Brief Visuospatial Memory Test uses only three trials (Benedict, 1997). The California Verbal Learning Test rises above the others, in that 16 words from list A are used with 5 trials (Delis et al., 1987; 2000).

The CVLT/CVLT-II learning curve standard measures versus the model's coefficients

Standard CVLT/CVLT-II learning curve measures include free recall correct scores for each Trial from 1 to 5. CVLT/CVLT-II Our modelEach trial score is an integer value. B3, B4, B2 are stochastic variables.Comparison the same Trial score Comparison of each coefficientbetween two tests is impossible. between two tests is possible.

B3 mainly represents attention span and short-term memory encoding process and might be a measure of short-term memory status. We call it "readiness to learn".

B4 represents long-term memory consolidation process and might be a measure of long-term memory status. We call it "ability to learn".

B2 is the learning rate.

)1(43 )1(2)1(2 XBXB eBeBY

The CVLT learning curves in patients with type 2 diabetes mellitus (T2DM)

Learning curves averaged over total group of healthy participant and patients with T2DM.Circles — healthy participants; B2 = 0.75, B3 = 8.28, B4 = 14.75. Squares — patients with T2DM; B2 = 0.98, B3 = 7.68, B4 = 12.86. Type 2 diabetes mellitus worsens ability to learn (p = .0002).

Thus, modeling the learning curve allows assessing memory impairment in patients with T2DM.

Prof. Antonio Convit, Center for Brain Health, New York University School of Medicine courtesy provided CVLT learning data of healthy participants and patients with T2DM,

The CVLT-II learning curves in patients with multiple sclerosis

Circles — healthy participants; B2 = 0.65; B3 = 7.28; B4 = 14.17.Squares — MS patients; B2 = 0.70; B3 = 6.16; B4 = 12.01.Readiness to learn (p = 0.002) and ability to learn (p = 0.001) are lower in the MS patients though the learning rates do not differ (p > 0.2).

Thus, modeling of the learning curve allows assessing memory impairment in patients with MS.

Prof. Ralph Benedict, Department of Neurology, State University of New York (SUNY) at Buffalo, courtesy provided CVLT-II learning data of healthy participants and patients with multiple sclerosis

Modeling the CVLT-C learning curvesAveraged learning curves over moderate and severe TBI female groupsCircles (the moderate group) — B2 = 0.69, B3 = 7.31, B4 =12.52. Diamonds (the severe group) — B2 = 0.56, B3 = 5.04, B4= 9.41.Coefficient B3 is lower in the severe group (p = .032).

Averaged learning curves over moderate and severe TBI male groupsCircles (the moderate group) — B2 = 0.46, B3 = 5.26, B4 =10.83. Diamonds (the severe group) — B2 = 0.78, B3 = 6.03, B4 = 11.62.Coefficient B3 is higher in the severe group (p = .046).

Prof. Seth Warschausky, Department of Physical Medicine and Rehabilitation, University of Michigan, Ann Arbor courtesy provided CVLT-C learning data of healthy children and children with traumatic brain injury.

Test/retest problems

Repeated memory testing is becoming commonplace in recent years. It is used to establish a baseline against which the effects of neurologic disease, traumatic brain injury or treatment may be assessed.

Repeated testing gives valid results only if another list of words is presented during each retesting. CVLT-II includes one alternate form. However, two forms is not enough, if a patient should be retested more times.

Our modification of Luria's memory test A famous Russian psychologist A. Luria developed a memory test that included ten words with 10 trials. Ten words are only slightly exceed short-term memory span. Ten trials are too many so that the test performance might lead to fatigue in an examinee.

We modified the test as follows:1. Each list consists of 16 words, taken from four

semantic categories: vegetables, animals, ways of traveling, and furniture.

2. Our test includes six lists of 16 words from the same semantic categories.

3. The order of words presentation is changed for each trial.

4. Six trials are used.

Correction of individual learning curves

Individual learning curves are of primary interest for monitoring changes before, during, and after treatment.

Though modeling of raw learning data with our model always is mathematically correct, values of B2 and/or B4 might be not biologically plausible.

There are three classes of “outliers”:

1. Extremely high values of B2,

2. Extremely low values of B2,

3. Extremely high values of B4.

Our method of correcting B2 and/or B4 is based on the assumption that the learning curve is of exponential shape, but any influences (functional or pathological) add some "noise" into the memory system output.

We developed a special algorithm for extracting the component of the learning curve that is exponential.

Extremely high values of the learning rate (B2)

A person recalled 9, 13, 13, 13, and 13 words on the CVLT-II test.

Modeling revealed B2 = 20.19, B3 = 9.0, B4 = 13.0, so that the learning curve is -shaped (solid line).

The first order system reaches 99.3% of its asymptotic value during five time constants. If a participant recalled the maximal number of objects during Trial 2, then T=1 / 5 and B2 = 5. This is a maximal biological plausible value for B2 and each B2 > 5.0 should be corrected.

After correction B2 = 1.07, B3 = 9.0, B4 = 13.49, so that the learning curve returned to its negatively accelerated shape (dotted line).

Extremely low value of B2 matched with high B4 value

A person recalled 5, 8, 10, 11, and 14 words on the CVLT-II test. Modeling revealed B2 = 0.08, B3 = 5.2, B4 = 35.5, so that the learning curve was straight line-shaped (solid line).The number of correctly recalled words cannot exceed the number of objects in a list. Taking into account that B4 value is real, B4 larger than (the number of objects + 1) is treated as an outlier. After correction B2 = 0.38, B3 = 5.0, B4 = 15.2, so that the learning curve returned to its negatively accelerated shape (dotted line).

Spread in raw recall values

A person recalled 10, 15, 11, 16, and 16 words on the CVLT-II test. Modeling revealed B2 = 0.19, B3 = 10.76, B4 = 20.68, so that the learning curve looks like a straight line (solid line).

This is an example of data scattering due to inability of monotone increasing the number of recalled words.

After correction B2 = 0.43, B3 = 10.0, B4 = 16.99, so that the learning curve returned to its negatively accelerated shape (dotted line).

Examples of variability of individual learning curves

during retesting healthy persons

Examples of variability of individual learning curves during retesting patients with cerebrovascular

disease

The averaged learning curve in patients with cerebrovascular disease

Circles – age-matched control participants without cerebrovascular disease; B2=0.85, B3=9.30, B4=15.49.

Diamonds – patients with cerebrovascular disease; B2=0.52, B3=7.05, B4=12.92.

Each participant and patient was tested three times.

Significant impairment of readiness to learn (p = 0.0024) and ability to learn (p = 0.006) are found.

Assessment of arginine vasopressin effect on memory rehabilitation in patients after stroke

A. Before treatment — B2 = 0.55, B3 = 6.56, B4 = 10.14.B. After treatment — B2 = 0.60, B3 = 6.72, B4 = 10.46.There is no significant effect of treatment with arginine vasopressin.

THANK YOU FOR ATTENTION