Brain Research Bulletin, Vol. 3, pp. 265-270. Printed in the U.S.A.

Enhanced Passive Avoidance Learning and Appetitive T-Maze Learning with Post-Trial

Rewarding Hypothalamic Stimulation

JOSEPH P. HUSTON AND CORNELIA C. MUELLER

Institute of Psychology, University of Diisseldorf, Lab. Comp. and Physiol. Psychol., Diisseldorf, West Germany and Institute ofPharmacology, University of Ziirich, Ziirich, Switzerland

(Received 16 January 1978)

HUSTON, J. P. AND C. C. MUELLER. Enhanced passive avoidance learning and appetitive T-maze learning with post-trial rewarding hypothalamic stimulation. BRAIN RES. BULL. 3(3) 265-270, 1978. - Experiments were carried out to investigate the effects of post-trial reinforcing stimulation of the lateral hypothalamus on learning in rats. The reinforcing stimulation was always presented for a duration of 20-30 set (0.2 set on/O.8 set off), and was administered either immediately, 30 set delayed or 300 SW delayed after exposure to the learning situation. In Experiment 1 post-trial stimulation led to improved passive avoidance learning of an alcove-avoidance task when presented 30 set compared to immediately after the footshock. In Experiment 2 reversal learning of a one-way shuttle-box avoidance task was facilitated by 30 set delayed, but not 300 set delayed post-trial reinforcing stimulation. In Experiment 3 appetitive left-right discrimination was investigated using a T-maze task. Thirty set delayed post-trial reinforcing stimulation presented contingent on errors facilitated learning of this task. Together, the 3 studies provide further support for the hypothesis that reinforcers directly influence labile memory processes (such as short-term memory) and thereby improve learning.

Memory Facilitation of learning Passive avoidance Hypothalamic stimulation Self-stimulation

Discrimination learning Reinforcement

SEVERAL studies have demonstrated that post-trial rein- forcement can improve learning of an avoidance response. The first evidence came from a study [5] with mice showing that 1 min access to food provided between 20 and 50 set after the footshock in a step-down avoidance trial improved the performance of this avoidance response; i.e., the mice remained up on the platform longer 24 hr later. This result is counterintuitive from the standpoint of conventional consideration of the nature of positive rein- forcers. Common sense would dictate that the reinforcer, if anything, should attenuate the effect of the preceding punishment, and cause the animal to step off the platform sooner instead of later, and therefore, to exhibit inferior rather than superior performance. This result, therefore, provided supporting evidence for a theory of reinforcement [4, 5, 61, which assumes that reinforcers in the operant conditioning situation strengthen behavior by a direct facil- itative action on immediate memory traces; i.e., by preventing the trace from fading or being disrupted. It follows that the reinforcer presented during any period of iabile (i.e., immediate and short-term) memory processing, such as, during the post-trial period for an avoidance learning trial, should have a similar effect, and consequently facilitate learning. This theory received additional support when it was first shown that post-trial reinforcing hypo-

thalamic stimulation facilitated avoidance learning in a shuttle-box task [ 91.

The experiments reported below were carried out: (a) to provide further tests of the hypothesis that post-trial reinforcing stimulation facilitates learning, (b) to examine the possibility of a time-dependent effect of the reinforcing stimulation on memory processes, and (c) to use different learning siutations, including a conventional passive avoid- ance situation as well as an appetitive learning task, to investigate the generality of the facilitating effect of the post-trial reinforcing stimulation.

EXPERIMENT 1: ALCOVE-AVOIDANCE

The present study was undertaken to examine the effects of post-trial reinforcement on avoidance learning in rats by the use of a conventional passive avoidance paradigm. The passive avoidance task used was the two- compartment test developed by Kurtz and Pearl [8] and BureS and BureSova [ 11. It takes advantage of the fact that rats spend most of their time inside a small box in favor of a large adjoining box. The rats learn to avoid the preferred small box after experiencing in it inescapable electrical footshock.

265

Copyright 0 1978 ANKHO International Inc.-0361-9230/78/0303-0265$00.75

266

Method

Male Sprague-Dawley (Tierspital Zurich) albino rats, weighing 210-350 g, were implanted with bipolar 0.2 mm dia. stainless steel electrodes (Plastic Products Co.) into the lateral hypothalamus (using coordinates: AP 5.4; H -2.6; L 1.5 ; [ 101 ). A total of 46 animals showed self-stimulation for reinforcing 0.2 set trains of 100 Hz sine-wave stim- ulation. The optimal current level for self-stimulation was roughly determined for each animal, and ranged from 14-42 MA, rms.

The rats were kept two per cage on a 12 hr light/12 hr dark cycle, with continuous access to food and water. The experiments were performed during the light phase of the cycle.

The apparatus consisted of a big box (50 x 50 x 35 cm) which was separated by means of a black guillotine door from an adjoining small box (21 x 13 x 16 cm). Both chambers had electrifiable grid floors (6 mm dia. stainless steel bars, spaced 13 mm apart). The small box had black walls and a removable metal cover, whereas the large box was grey and illuminated by overhead fluorescent room lights.

The animals were given three successive daily baseline trials of 5 min duration each. They were placed onto the grid floor of the big box, and the entries and time spent in the little preferred box were recorded. They were then assigned to three groups: two experimental groups (each with 15 rats) received 30 set of reinforcing stimulation (0.2 set on/O.8 set off) commencing either immediately or 30 set after administration of the footshock in the small chamber. A control group (16 rats) was handled exactly like the 30 set delayed post-trial stimulation group, but was not administered electrical brain stimulation.

During the learning trials the following procedure was repeated three times at 48 hr intervals: the rats were placed into the small box with the guillotine door and the cover closed. Then a 1 mA inescapable scrambled footshock of 3 set duration was administered, followed by either immediate, 30 set delayed, or sham stimulation, according

I Acquisition I

HUSTON AND MUELLER

to the group assignment. The post-trial treatments were administered in a cylindrical container (34 cm dia./37 cm high). Twenty-four hours later the rats were retested in the same manner as during the baseline trials; i.e., they were placed into the big chamber, and entries and time spent in the small chamber were recorded for 5 min.

Next, to test for possible differences in resistance to extinction between the three groups, six additional ex- tinction trials (one 5 min trial per day) were administered.

Results

In order to test for possible differences in learning between the three groups, the number of successful learners were compared. The criterion for successful learning in avoiding the small box was arbitrarily taken to be 0 set spent in the little box.

Comparison of numbers of learners revealed that the experimental group that received 30 set delayed post-trial reinforcing stimulation showed significantly superior learning in comparison to the control group (p<O.O2, Krauth-test for time-effect curves [7] ), as well as to the experimental group that received immediate post-trial stimulation (p<O.Ol) (see Fig. 1, acquisition). Also, only the 30 set delayed post-trial stimulation group showed a significant decrease in time spent in the little chamber from the third baseline trial to the first retest trial (p<O.Ol, Wilcoxon matched-pairs signed-ranks test); i.e., it was the only group that exhibited learning after only one trial.

The group that was administered 30 set delayed post- trial stimulation was also more resistant to extinction (see Fig. 1, extinction). This group differed significantly from the control group in extinction trial two (p<O.O3, Fisher’s exact probability test).

The performance of the immediate post-trial rein- forcement group was inferior to the 30 set delayed post- trial stimulation group in acquisition and, furthermore, was less resistant to extinction. This latter result was antic- ipated, since in the immediate post-trial stimulation group the reinforcement was expected to interact more strongly

Extinction

-<5sec - 30 set

Trial

1%. 1. Acquisition and extinction curves of the immediate and 30 set delayed post-trial reinforcement groups and the non-stimulated control group.

FACILITATION OF LEARNING 26-l

with the footshock and to reinforce the presence in the small chamber by virtue of the close temporal contingency.

See results of Experiment 3 for histological data.

EXPERIMENT 2: ACTIVE AVOIDANCE

REVERSAL LEARNING

In a preliminary study [9] we reported that 30 set delayed post-trial reinforcing hypothalamic stimulation led to a significant improvement of passive avoidance learning in rats, using a one-way shuttle-box task. The present experiment was carried out to replicate this finding and to determine a possible time-dependent effect of the delayed reinforcing stimulation on learning.

Method

Of 41 male Sprague-Dawley rats, which underwent the same preexperimental treatments as described in Ex- periment 1, 26 animals were included in the experiment on the basis of reliable self-stimulation and completion of all trials.

The shuttle-box consisted of a black box with an adjoining transparent chamber with each measuring 18 x 25 x 20 cm. Both had electrifiable grid floors (as in Experiment 1) and hinged doors serving as covers.

Post-trial stimulation was administered in a separate grey cylindrical container (34 cm dia./37 cm high).

In the first experimental phase the animals were trained in a one-way active avoidance task. They were placed into the black box where a 1 mA escapable footshock of 3 set duration was applied either within 2 set (Trial l-S), after 5 set (Trials 6-l 1) or after 10 set (Trials 12-18, 20 and 22). The animal’s behavior was recorded in terms of escape or avoidance responses. During Trials 19, 21 and 23 immediate footshock was administered to strengthen the conditioned response.

The animals then were randomly assigned to two experimental groups. including a 30 set (n = 8) and a 300 set (n = 9) delayed post-trial stimulation group; and a control group (n = 9), that received sham stimulation 30 set after the footshock. The stimulation was intermittent, i.e., 0.2 set on/O.8 set off. lasting for 20 sec.

After the conditioned avoidance response was established the procedure was reversed. Now the animals received the 1 mA, 3 set footshock when they entered the previously safe transparent chamber. They were im- mediately removed from the experimental box and placed into the stimulation container, where, after a delay of either 30 or 300 set, 20 set of post-trial reinforcing stimulation (0.2 set on/O.8 set off) was administered. The sham stimulated controls were handled identically as the 30 set delay group but were not administered brain stimulation. Thereafter all animals were returned to their homecages and retested 24 hr later. For each animal the step-through latency (STL), i.e., the time elapsed to enter the transparent chamber with all limbs, was measured. Animals which entered the transparent box received the footshock and the respective post-trial treatment. This procedure was continued over a total of 6 retest trials spaced 24 hr apart.

The arbitrary criterion for learning the passive avoidance response was taken to be 150 set stay in the black chamber. If an animal met this requirement it was removed from the experimental situation. In this case further footshock or brain stimulation was not administered.

A direct test for possible aversive properties of passively administered stimulation was performed prior to the above experiment. The apparatus was a shuttle-box (21 x 60 x 38 cm) consisting of two identical chambers separated by a 3 cm high hurdle. Brain stimulation (0.2 set on/O.8 set off) at the same current levels used during the subsequent experiment was available first in one chamber for 8 min, then in the other chamber for 8 min. Entry of the positive chamber with all four limbs automatically turned the stimulation on, exit from the chamber turned it off. Firstly, if the stimulation had aversive properties, the rat would be expected to prefer the nonstimulated side to the stimulated side. Secondly, if a period of 20 or 30 set of passively administered stimulation had aversive properties, it could be expected that the animal would learn to turn the stimulation off prior to the termination of the 20 or 30 set interval by moving to the non-stimulated side. In this case, a decrease in the number of periods of 20 or 30 set duration or longer from the 1st to the 3rd third of each 8 min trial would indicate that the stimulation had aversive properties.

Results

The active avoidance response, i.e., step-through latency of less than 10 set, was acquired by all animals within 18 trials. The learning curves for the conditioned avoidance response for experimental and control groups are shown in Fig. 2. There are with one exception (Trial 7: 30 set vs 300 set delay group, p = 0.05, one-tailed, Fisher’s exact probability test) no significant differences between groups. As can also be seen in Fig. 2, the majority of animals learned the avoidance response in about half the trials, and thus were overtrained.

Trial

FIG. 2. Acquisition curve of the one-way active avoidance response in terms of percentage avoidance for the 30 set and 300 set delayed

stimulation groups and the control group.

Learning of the passive avoidance response (reversal learning) by the 30 set delayed stimulation group differed from the 300 set delayed stimulation group and from the control group, as illustrated in Fig. 3. Thirty set delayed

268 HUSTON AND MUELLER

H 30 set

IB 300 set

III cant rol

A l-2 3-4 5-6

Trial

FIG. 3. Acquisition of the passive avoidance response in terms of percentage avoidances over the pooled trials for the 30 set and

300 set delayed reinforcement groups and the control group.

reinforcing stimulation facilitated learning significantly compared to the effect of 300 set delayed reinforcing stimulation with respect to acquisition of the avoidance response (arbitrary criterion: 150 set in black chamber), (p<O.OS, Krauth-test for time-effect curves [ 71, one-tailed). No significant differences were observed between 30 set delay and the control group, nor between the 300 set delay and control group, although a tendency towards better learning of the 30 set delayed stimulation group is apparent (Fig. 3).

If we consider the number of successive trials during which the criterion of 150 set was reached and maintained (4 or more out of 6 trials), a significant difference was found in favor of the 30 set delayed stimulation group

compared to the control group (p = 0.05, Fisher’s exact probability test, one-tailed); i.e., 30 set post-trial rein- forcing stimulation facilitated retention of the learned avoidance response.

Table 1 shows the mean step-through latencies (STL) and standard deviations for the two experimental and the control group over the pooled trials.

The shuttle-box choice test did not yield any evidence for aversive effects of the passively administered stim- ulation. For one, the total time that the animals spent on the stimulation-on side was 63.8% (vs 36.2% in the stimulation-off chamber). In addition it was observed that the number of periods of 20 set duration or longer increased from 45 to 60, and the number of periods of 30 set duration or longer increased from 3 1 to 45 from the 1st to the 3rd third of the two 8 min trials taken together. Thus, it can be concluded that the passively administered stimulation was not aversive, but instead reinforcing for the duration of 20 or 30 sec.

For histological data see results of Experiment 3. In summary, the above data show that 30 set delayed

post-trial stimulation facilitated reversal learning of an active avoidance task, whereas 300 set delayed stimulation did not have this effect. This confirms a similar previous finding [9], and in addition, suggests that the post-trial facilitation effect is time-dependent, in so far as the 300 set delay group learned less well than the 30 set delay group.

EXPERIMENT 3: APPETITIVE T-MAZE LEARNING

Since previous experiments have shown that post-trial reinforcing events can facilitate avoidance learning ([ 2, 4, 5, 6, 91, and Experiments 1 and 2), presumably by directly acting on labile memory processes, it was of interest whether similar improvement of learning with post-trial reinforcing stimulation could be shown also in an appetitive task. However, a facilitation of appetitive learning as a result of post-trial reinforcement could be interpreted simply in terms of an addition of two reinforcing events. Therefore we employed a paradigm used successfully by Deweer [3] to show facilitation of learning with post-trial stimulation of the reticular formation; i.e., the animals received post-trial hypothalamic stimulation contingent on each wrong response. A facilitation of appetitive learning with reinforcing stimulation contingent on only wrong responses would allow an interpretation of such facilitation in terms of the reinforcement’s acting directly on memory “traces” [4, 5, 61.

TABLE 1

MEAN STEP-THROUGH LATENCIES (x) AND STANDARD DEVIATIONS (SD) IN SEC FOR THE AVOIDANCE REVERSAL LEARNING TASK

NO. rats Baseline

Trials l-2 3-4 5-6

30 set delayed X 4.6 33.8 104.2 104.7 stimulation 8 SD k2.8 +37.4 267.4 i66.3

300 set delayed X 5.8 21.6 73.3 79.6 stimulation 9 SD 24.5 248.4 1r66.8 k71.5

Control jz 5.0 29.8 71.9 94.5 no stimulation 9 SD 23.8 k51.4 -t 57.7 ? 69.4

FACILITATION OF LEARNING

Method

The animals used in this study were also used in Experiment 1. The conditioned avoidance response that had been established in Experiment 1 was extmguishea prior to the beginning of the present experiment. Stim- ulation current levels now ranged from 14-71 PA, rms.

The T-maze was made out of grey plastic, and consisted of a stem (100 x 10 x 40 cm), a choice area (10 x 10 cm) and two arms (45 x 10 x 40 cm). The left arm was painted black and the right arm white. At the entrance of each arm a removable guillotine door (10 x 40 cm) was mounted and at the end of each arm a round basin (9.2 cm dia.) was placed, into which the food reward (chocolate biscuits) could be dispensed through a 1.5 cm dia. wide tube fixed 5.5 cm above the floor.

The post-trial stimulation was administered in a cylin- drical grey container (34 cm dia., 37 cm high).

Before the experiment was started the rats were food- deprived for several days until they reached 80% of their original body weight. They were thereafter kept on this reduced weight schedule.

At first the animals were trained to run the maze; i.e., to move from the closed end of the stem to the choice area and then into either of the arms, where each run was food-rewarded. Thereafter they were randomly split into two groups: an experimental group (n = 19) which received post-trial 30 set delayed reinforcing stimulation, and a control group (n = 21) which received sham stimulation contingent on each incorrect response (i.e., choice of the arm which was not food-rewarded). Each group was subdivided into a reward-left and a reward-right group to exclude a possible side preference.

The learning experiment was then begun: If an animal chose the correct arm it immediately received the food- reward, and was then removed from the experimental situation. If the animal’s choice was incorrect it was removed from the maze after approximately 20 set and placed into the container where 30 set delayed (from the time of entry of the non-rewarded arm) reinforcing stimulation (0.2 set on/O.8 set off) or sham stimulation was administered for the duration of 30 sec. Then the rats were returned to their homecages. The same procedure was carried out twice a day until 6 trials were completed. Each run served as a learning trial as well as a test trial for the previous one.

Results

Analysis of the data in terms of correct responses (i.e., entry of food-rewarded arm) for each day indicated somewhat improved learning by the post-trial 30 set de- layed stimulation group, although the difference from the control group was not significant. This may be due to the fact that there were three animals which learned the correct response errorlessly in the control group and only one errorless learner in the experimental group. Figure 4 illus- trates the findings.

When the experimental and control groups were com- pared with respect to the number of errors which occurred during the acquisition of the correct response, it was observed that there were six animals in the experimental (n = 18) and one animal in the control group (n = 18) who learned the correct response after committing only one error. This difference is significant (p = 0.03, Fisher’s exact

80

70

60

50 5 269

0 30 set

cl control

* 50%

1 2 3

Day FIG. 4. Acquisition in terms of percent correct choice responses in an appetitive left-right discrimination task for each day (two trials). The experimental group received post-trial 30 set delayed rein- forcing stimulation contingent on errors; a control group was given

sham stimulation.

probability test); i.e., the post-trial 30 set delayed stim- ulation group learned the T-maze faster than the control group.

Post-trial 30 set delayed reinforcing stimulation con- tingent on the wrong response resulted in an improvement of learning the left-right discrimination task in a T-maze.

Histology. The placements of 38 electrodes are re- constructed on plates from Pellegrino and Cushman [ lo] in Fig. 5. Most of the electrode placements are located in the lateral hypothalamus (74%). The remaining tip sites were found in the zona incerta (18%) and in the area between the zona incerta and the capsula interna (8%).

DISCUSSION

Rewarding lateral hypothalamic stimulation adminis- tered 30 set after the training trial led to a facilitation of learning in each of the three experiments reported here. To recapitulate the most salient results: In Experiment 1, using a conventional passive avoidance task (alcove-avoidance task), avoidance of the small preferred chamber was facilitated by 30 set delayed post-trial rewarding stim- ulation, compared to the immediate stimulation and sham stimulation groups. In Experiment 2 the reversal learning of the overtrained one-way active avoidance response can also be considered to be a passive avoidance task. The results confirm the findings of the previous study [ 91 in showing a facilitation of reversal learning with 30 set delayed rein- forcing stimulation, and in addition, show that such stimulation is ineffective when it is delayed by 5 min. This suggests that the facilitatory action of post-trial reinforcers on avoidance learning is time-dependent, and supports an interpretation of such effects in terms of action on short-lasting post-trial labile memory processes [4, 5, 61.

Experiment 3 demonstrated that post-trial reinforcing brain stimulation can also facilitate appetitive learning - in

210 HUSTON AND MUELLER

5.6

6.0

6.2

6.4

FIG. 5. Reconstruction of 38 electrode tip sites on plates from Pellegrino and Cushman [lo].

this case a food-reinforced left-right discrimination. Here it should be emphasized that post-trial reinforcement was administered only after incorrect responses in order to rule oul

1.

2.

3.

4.

5.

6.

as much as possible alternative interpretations of a

facilitating effect of such stimulation in terms of a simple addition of two reinforcers. A facilitation of appetitive learning with a post-trial reinforcer contingent on errors supports an interpretation based on a direct action on memory processes. (A similar paradigm was used by Deweer [ 31 to facilitate discrimination learning with post-trial stimulation in the brain stem reticular formation.) As suggested earlier [4, 5, 61 events could come to serve as reinforcers by preventing labile memory traces from fading, and thereby facilitating the consolidation of memory into long-term storage.

The possibility must be considered that the long trains of experimenter administered post-trial stimulation had aversive properties [ 1 l] and, therefore, that the facilitation of learning in each case came about as a result of addition of punishing stimulation. The results of the test for aversiveness performed in Experiment 2 suggest that this was not the case.

Since our initial demonstration of a facilitation of avoidance learning with post-trial reinforcement, a number of studies have appeared in support of the idea [4, 5, 61 that reinforcers act directly on memory processing. For instance, Destrade et al. [2] have shown a facilitation of step-down avoidance learning in mice with reinforcing (as well as presumably sub-reinforcing) lateral hypothalamic stimulation. Wetzel et al. [ 121 recently reported facilitation of learning with post-trial stimulation of the septal area. White and Major [ 131 reported that post-trial self- stimulation facilitated memory in an appetitive learning task.

ACKNOWLEDGEMENT

This work was supported by Grant No. 3.6610.75. from the Swiss National Science Foundation.

REFERENCES

BureS, J. and 0. Buregovi. Cortical spreading depression as a memory disturbing factor. J. camp. physiol. Psychol. 56: 268-272,1963. Destrade, C.. R. Jafford and B. Cardo. Post-trial hippocampal and lateral hypothalamic electrical stimulation effects on long-term memory and on hippocampal cholinergic mech- anisms. Proc. Acad. Sci. GDR, 1971, in press. Deweer, B. Selective facilitative effect of post-trial reticular formation stimulation in discriminative learning in the rat. Behav. Proc. 1: 243.-251, 1976.

7.

8.

9.

10.

Huston, J. P. and C. Mondadori. Memory and reinforcement: a 11.

model. In: Proc. 2nd Int. Congr. CIANS. Prague, 1975. Activitasnerv. Supp. 19: 17-19, 1977. Huston, J. P., C. Mondadori and P. G. Waser. Facilitation of 12. learning by reward of post-trial memory processes. Experientia 30: 1038-1040, 1974. Huston, J. P., C. Mueller and C. Mondadori. Memory facil- itation by post-trial hypothalamic stimulation and other 13. reinforcers: a central theory of reinforcement. Biobehav. Rev. 1: 143-150,1977.

Krauth, J. Nichtparametrische Ansatze zur Auswertung von Verlaufskurven. Biometr. Zeitsch. 15: 557-566, 1973. Kurtz, K. H. and J. Pearl. The effect of prior fear experience on acquired-drive learning. J. camp. physiol. Psychol. 53: 201-206,196O. Mondadori, C., K. Ornstein, P. G. Waser and J. P. Huston. Post-trial reinforcing hypothalamic stimulation can facilitate avoidance learning. Neurosci. Lett. 2: 183-187, 1976. Pellegrino, L. J. and A. J. Cushman. A Stereotaxic Atlas ofthe Rat Brain. New York: Appleton-Century-Crofts, 1967. Steiner, S. S., R. J. Bodnar, R. F. Ackermann and S. J. Ellman. Escape from rewarding brain stimulation of dorsal brain-stem and hypothalamus. Physiol. Behav. 11: 589-591, 1973. Wetzel, W., T. Ott and H. Matthies. Post-training hippocampal rhythmic slow activity (“Theta”) elicited by septal stimulation improves memory consolidation in rats. Behav. Biol. 21: 32-40,1977. White, N. and R. Major. Dopaminergic basis of self-stimulation induced improvement in memory. Sot. Neurosci. Abstr. 111, 762,1977.