Chapter Outline

Classical ConditioningPavlov and the salivating dogsClassical conditioning: The basics

Classical Conditioning Phenomenaand ApplicationsTiming and the number of conditioning trialsExtinction and recovery of conditioned responsesPredictability of the unconditioned stimulusStimulus generalization and discriminationNature versus classical conditioningOther classical conditioning phenomena

Operant ConditioningB. F. Skinner's magic boxShaping and successive approximationsReinforcement and punishment: The basics

Operant ConditioningPhenomena and ApplicationsDelay of reinforcement and punishmentThe power of partial reinforcementWhen there's a choice:Reinforcement versus punishmentThe power of superstitions and phobiasWhen there is no escapeBehavior modification and token economiesNature versus operant conditioning

Cognitive Learning:A Contemporary ViewRooted in the PastThe case of the ingenious chimpsLearning in the absence of reinforcementCognitive maps, expectancies, and knowledge

SummaryTest Yourself AnswersStudy GuideA Matter of Debate: Spanking Children

Life Span Perspective: Learning and Age

ConditioningThe establishment of a rela-tionship between stimuli andresponses, or vice versa.

Although much learningtakes place in school class-rooms, learning also occurs ina variety of other situations,including the home, the work-place, and social and recre-ational settings.

Our behavior depends to a large extent on the process of learning-a subjectthat has been a central concern of psychology from the very beginning. In-deed, every chapter in this text includes some discussion of the role of learn-

ing. In this chapter, we look at basic principles and phenomena in learning thathave been studied for over a century and continue to be studied, though nowa-days from a different perspective.

During the first half or so of the 20th century, learning processes were consid-ered primarily from the viewpoint of conditioning, which emphasizes the estab-lishment of overt associations between stimuli and responses or between responsesand stimuli. The early learning theorists thought that allbehavior-human or other-wise-could be understood in terms of conditioning. They were wrong, as dis-cussed at various points in the chapter, but their error was mainly one of degree.Just as there are some important differences between humans and other animalswhere learning is concerned, we do have some things in common, and a fair amountof learning can indeed be understood in terms of conditioning. This is especiallytrue when we allow for the often-intricate cognitive processes that occur in be-tween stimuli and responses-processes that are mentioned throughout the chap-ter where they shed light on learning and conditioning phenomena.

As noted in Chapter 1 and discussed further at the end of this chapter, evenduring the heyday of strict behaviorism, some psychologists believed that learn-ing could not be fully understood without considering the subtle workings of thebrain. The rise of cognitive psychology has popularized this view of learning as theacquisition of knowledge-one of the steps in "the human being's active interactionwith information about the world" (Klatzky, 1980). It is this process that allows us

to accomplish the learning necessary, for example, to playa musical instrument,devise a computer program, or write a textbook.

Thus, both approaches to studying learning have been fruitful. Together, theyrepresent what is today a generally accepted definition of leaming: a relativelypermanent change in behavior potential resulting from experience. "Relativelypermanent" means that once something is learned, it typically is not "unlearned"or forgotten. Forgetting is more likely caused by an inability to retrieve somethingfrom memory, as discussed in Chapter 5. "Behavior potential" emphasizes thatlearning does not necessarily yield a change in what a person thinks or does at thetime the learning occurs. For example, specific things that you learn in this textmayor may not have an effect on your behavior as you learn them, but they havepotential for affecting your behavior at some future time. Finally, experience is in-cluded in the definition to rule out changes such as those attributable to biologi-cal development.

In this chapter, we first consider classical conditioning, then turn to operantconditioning, and wind up with a survey of the theories of early researchers whostudied learning processes from a cognitive perspective. Along the way, we ex-plore what conditioning does and does not explain well, along with the role of pre-dispositions-built-in tendencies in learning and behavior that humans as well asmembers of other species display.

CLASSICAL CONDITIONINGMany of us have seemingly inexplicable preferences for cer-tain things, especially certain people. You may be instantly FOCUS QU ESTIONSattracted to men who have beards or men who don't, to smallwomen or women who are tall and broad shouldered. Youmay feel unexplained warmth toward a certain tone of voiceor the way a person walks, gestures, or dresses. You know thatthese matters have nothing to do with what the person is re-ally like, yet you find yourself attracted-or perhaps repelled.

Similarly, many of us are troubled by intense, unreasonable fears. You may beutterly terrified of snakes, dogs, rats, spiders, bees, cockroaches, or other animalsor situations that might be dangerous but much more often aren't. Collectively,such extremely disproportionate fears are known as phobias (see Chapter 13).

Likes, dislikes, fears, and more may be learned in various ways (e.g., seeDe Houwer, Thomas, & Baeyens, 2001), but one basic way is through pairings ofan initially neutral stimulus-one that does not produce a certain response-withanother stimulus that does. To see how this works, we begin with the research ofIvan Pavlov.

LearningA relativelv permanent changein behavior potential as aresult of experience.

• What is classical conditioning and how did Pavlovstudy it?

• What basic procedures are involved in classicalconditioning?

Pavlov and the Salivating DogsPavlov conducted his experiments in the early years of the 20th century in SaintPetersburg, Russia. After spending half of his career as a physiologist, he serendip-itously turned to the study of how "automatic" responses such as reflexes can be-come associated with stimuli other than those that normally produce them (seeChapter 1). Reflexes do not require thought. In humans, one reflex is the knee jerk:When you have your legs crossed, with one foot dangling in the air, a sharp tapjust below the kneecap makes the lower part of your leg jump. Another reflexmakes your pupils smaller whenever a bright light strikes your eyes. It is important

ReflexA built-in or otherwise auto-matic response to a specificstimulus.

Ivan Pavlov (at center) isshown in his laboratorywithhisassistants,hisexperimen-tal apparatus, and one ofthe dogs that served assubjects in his researchonconditioning.

Classical conditioningThe establishment of a rela-tionship between two stimuli,tvpicallv one that evokes areflex response and one thatis initiallv neutral with regardto this response.

FIGURE 4.1 Pavlov's appa-ratus. A tube attached to thedog's salivarv glands collectssaliva, and the number of dropsfrom the tube is recorded ona revolving drum outside thechamber. The experimenter canwatch the dog through a one-waV mirror and deliver food tothe dog's feed pan bV remotecontrol. Thus, there is nothingin the chamber to get the dog'sattention except the food andwhatever other stimulus theexperimenter wishes to present,such as the sound of a bell.Yerkes & Margulis, 1909.

to note that emotional responses can alsobe reflexive. In response to a sudden loudnoise nearby, even a baby will be startledand fearful, producing the bodily changesand cognitive reactions characteristic offear.

Pavlov's subjects were dogs, and his ex-perimental apparatus was the simple but ef-fective device illustrated in Figure 4.1. Whenhe began to realize the significance of what

~ is now called classical conditioning, whicha: in essence consists of establishing a leamed~ association between two stimuli, Pavlov be-

gan studying the conditioning of saliva-tion-a reflex action of the mouth's salivary glands. He strapped a dog into theharness shown in Figure 4.1 and then used sounds as a stimulus. (Initially he usedthe beat of a metronome, but later he changed to more manageable bells andtones.) The dog made a few movements called orienting responses, but, as ex-pected, no salivation occurred. The primary stimulus for salivation is the pres-ence of food in the mouth-not the sound of a bell. When food was delivered andthe dog took it into its mouth, drops of saliva flowed freely down the tube shownin the figure.

Now Pavlov set about associating the neutral stimulus of the sound with thereflex action of the salivary glands. When the bell was sounded, he gave the dog asmall amount of food and the dog salivated. After a number of instances in whichthe sound was paired with the food, he sounded the bell without presenting food-and saliva flowed anyway (Pavlov, 1927/1960). Thus, the dog had leamed an asso-ciation between the two stimuli-specifically, that the sound would usually be fol-lowed by food.

Over the years, psychologists have studied classical conditioning with a widevariety of techniques and with various species of animals-mostly rats, pigeons,and rabbits. Such leaming has been shown to be possible even in the primitivesea snail (Carew, Hawkins, & Kandel, 1983). One technique-for conditioning arabbit to blink-is illustrated in Figure 4.2. Similar procedures have been used tostudy the reflexive eyeblink response in humans.

To recording equipment

I

Classical Conditioning: The BasicsTo understand how classical conditioning occurs, we first consider its five basicelements. These elements, illustrated in Figure 4.3, are common to all instances ofclassical conditioning.

1. The food was the unconditioned stimulus (VCS) that produced salivation.Any stimulus that automatically and reliably produces a particular responsecan serve as a DCS.

2. The sound of the bell was initially a neutral stimulus (SN) with respect to sali-vation. But after repeated pairings with the DCS, it became a conditionedstimulus (CS) that also elicited salivation. In principle, any stimulus a subjector participant can detect may be the CS-although, as discussed later in thechapter, certain CSs work better than others with certain DCSs.

3. The response of the salivary glands to the food was the unconditioned re-sponse (VCR) because it occurred without prior leaming.

4. The response of the salivary glands to the sound of the bell was the condi-tioned response (CR), which was leamed. Although both the CR and the DCRare salivation, they differ in two important ways. First, the CR is slower tooccur; for example, the dog doesn't salivate quite as quickly in response to thesound as it does to the food. Second, the CR is "weaker"; the dog doesn't sali-vate as much in response to the sound as it does to the food.

5. The DCS and the CS-to-be are repeatedly presented together; each such pair-ing is called a conditioning trial. What is leamed in classical conditioning is anassociation between the CS and the DCS. To assess whether classical condi-tioning has occurred, the CS is presented by itself. This is called a test trial.

FIGURE 4.2 Conditioningthe eyeblink. The rabbit'shead is held stationary, andone end of a fine string isattached to the upper lid ofone eye. The other end isattached to a sma II potenti-ometer-an electrical devicethrough which eyelid move-ments can be translated intoimpulses and recorded. Thestimulus for the eyeblink reflexis a light puff of air to thesurface of the eye. In variousconditioning experiments,rabbits have learned throughclassical conditioning to blinkin response to totally neWstimuli-including lights,tones, and even vi brationof the animal's abdomen witha hand massager.Adapted from Damian &

Burkhard, 1986.

Unconditioned stimulusIUCS)Any stimulus that automati-cally and reliably produces aparticular response, such asa reflex.

Conditioned stimulus ICS)An initially neutral stimulusthat comes to elicit a responsesimilar to that elicited by aUCS.

Unconditioned responseIUCR)The automatic response to anunconditioned stimulus.

Conditioned response ICR)The learned response to aconditioned stimulus.

FIGURE 4.3 The basicelements 0' classical condi-tioning. Although there maybe many variations in suchspecifics as the timing of thestimuli, the same basic stepsare followed in all instancesof classical conditioning_

--~

(a) Juan, a 4-year-old, be-came nauseated and vomitedwhen he sat in the dentist'schair for the first time. Sincethat episode, he feels queasywhenever he sees anyonein a white coat. What condi-tioning might explain Juan'sreaction?

(b) In the previous ques-tion, what would you call thedentist's garb?

--~

CLASSICAL CONDITIONINGPHENOMENA AND APPLICATIONS

FOCUS QUESTIONS• Why is the timing of the CS and UCS important in

classical conditioning?• Why is it important for the CS to "predict" the UCS?

• How do generalization and discrimination work?• How do biological predispositions affect classical

conditioning?

The rate at which classical conditioning occurs and the strengthand persistence of the conditioned response are affected by a va-riety of factors, including the timing of the es and DeS, the num-ber of conditioning trials, the consistency of the eS-Des pair-ings, and biological predispositions of the learner. These factorsand some interesting phenomena in classical conditioning-

both in laboratory animals and in humans-are discussedin this section.

Extinction and Recoveryof Conditioned Responses

Delayed conditioningThe most effective version ofclassical conditioning, in whichCS onset precedes UCS onsetand the offset of both stimuliis typically at the same time.

Simultaneous conditioningA less effective version ofclassical conditioning, in whichboth CS and UCS onset andoffset occur at the same time.

Trace conditioningA less effective version ofclassical conditioning, in whichCS onset and offset precedeUCS onset.

Delayedconditioning

Timing and the Number of Conditioning TrialsIn general, the conditioning process is most effective when the es (the bell, inPavlov's experiment) is presented just before the ues (the food)-an arrangementcalled delayed conditioning. For many stimuli, a very brief delay of about 0.5 secworks best. Less effective are simultaneous conditioning, in which the es andthe ues are presented at exactly the same time, and trace conditioning, in whichthe es is presented and discontinued before the ues. Each type of conditioningis illustrated in Figure 4.4.

The number of trials necessary to establish a eR varies according to the be-havior being conditioned. For example, fear reactions are readily conditioned inthe laboratory and in everyday life. After only a few trials in which a neutral stim-ulus is paired with a painful ues such as electric shock, the now es will producebodily reactions symptomatic of fear-crouching, trembling, urination, defeca-tion-indicating that the animal has learned that the es signals that a shock iscoming. Human phobias can be acquired in much the same way. If you're phobicof dogs, it could be that as a small child you were repeatedly menaced by a largedog and thereby conditioned to be terrifiedat the very sight of one-perhaps for life.The question of why phobias can be so en-during, however, requires an understandingof operant conditioning, discussed later inthe chapter.

Simultaneousconditioning

Traceconditioning

Once Pavlov had established the conditionedsalivary response, he wanted to find outhow long and under what circumstances itwould persist. When he merely kept sound-ing the bell without presenting food, hefound that within a very short time the flowof saliva in response to the sound began todecrease. Soon it stopped altogether, asshown in Figure 4.5. In the same way, yourdog or cat would eventually stop gettingagitated and salivating at the sound of a canopener if you stopped feeding it cannedfood. In Pavlov's terminology, this disap-pearance of the conditioned response iscalled extinction. When Pavlov occasion-ally followed the sound with food-thus providing the ues some of the time butnot every time-he found that he could make the conditioned response continueindefinitely.

Pavlov also demonstrated another phenomenon related to extinction. He with-held the ues and let the eRdisappear as usual, then gave the dog a rest away fromthe experimental apparatus. After a period of time, he put the dog in the appara-tus again to see if a response to the bell would occur. Under these circumstances,the eR that had seemed to be completely gone reappeared. Pavlov called thisspontaneous recovery; it is a clear illustration that behaviors that cease to occurare not simply unleamed or forgotten.

FIGURE 4.4 The importance of timing in classical conditioning.Each pair of lines represents one conditioning trial. The top line ineach pair represents the stimulus being conditioned, and the bottomline represents the unconditioned stimulus. Stimulus "onset" occurswhen the line goes up; stimulus "offset" occurs when the line goesback down. Thus, in delayed conditioning, CS onset occurs before UCSonset; offset for both stimuli is typically at the same time. In simul-taneous conditioning, CS and UCS onset occur at the same time, asdo CS and UCS offset. In trace conditioning, CS onset and offset bothoccur before UCS onset.

ExtinctionDisappearance of the CR upondiscontinuation of the UCS.

Spontaneous recoveryReappearance of an extinguishedCR after the passage of time.

For most conditioned responses, the cru-cial factor is not the number of times aes is paired with a DeS but rather thereliability with which the es predicts theoccurrence of the DeS (Rescorla, 1988;Rescorla & Holland, 1982). In one exper-iment, dogs were first trained to jumpback and forth in a "shuttle box" to es-cape an electric shock. (A shuttle box issimply a dog-sized enclosure divided inthe middle by a short barrier to distin-guish its two compartments.) After thedogs had learned to jump, they were di-vided into two groups. For one group, atone es and the electric shock Des werepresented randomly, so that a tone mightoccur before a shock just as often as it

occurred after it. For the second group, the tone was usually fol-lowed by a shock. All the dogs were then presented only with thetone eS-that is, with no shock Des. If learning an associationbetween hearing a tone and feeling a shock required only that tonesand shocks occur together, the first group should have learned tojump as quickly as the second group-both groups had experi-enced the same number of tones and shocks. But if learning tojump was easier when the tone usually predicted when the shockwould occur, then the second group should have learned morequickly. The results showed that the dogs in the second group didindeed learn more quickly because of the relatively consistent re-lationship between the es and the Des.

This study suggests that we are more likely to develop a con-~ ditioned response-for example, a dislike for a particular type of.§ person or physical appearance-if our encounters with that stim-~ ulus are consistently followed by unpleasant experiences. If expo--g sure to the person and unpleasant experiences occur together but~ in a random rather than a predictable fashion, it is less likely that.~

~ the conditioned dislike will develop."g~i)l~~:0

,!l! In Pavlov's experiments, there was nothing special about the soundproduced by the bell. He later used many other kinds of stimuliand found that he could just as easily condition the salivary re-sponse to the sound of a tone or the flash of a light. He also dis-covered that a dog that had been conditioned to the sound of abell would also salivate to the sound of a bell with a different tone.

This phenomenon is called stimulus generalization. Once an organism haslearned to respond to a particular stimulus, it tends to display that behavior to-ward similar stimuli as well-although to a progressively lesser extent as the stim-uli become less similar. Thus, a dog or a cat displays a generalized response to thesound of any can being opened or bag being rustled-not just one that containsfood or treats.

ctl.:0: 10(ij(/)

'0(/)Cl.e 50

Predictability 0' theUnconditioned Stimulus

FIGU RE 4.5 The conditioned response: going, going, gone. The graphshows what happened to Pavlov's dogs when the conditioned stimulus of soundwas no longer accompanied by the unconditioned stimulus of food. The condi-tioned salivary response, very strong at first, gradually grew weaker. By theseventh time the bell was sounded, the conditioned response Wasextinguished-at least for this session.

A person who has experienced painful dentalprocedures may develop a conditioned fear ofdentists.

Stimulus generalizationThe tendency of a CR to occu rto CSs that are similar tothe original CS.

Stimulus Generalization and Discrimination

Stimulus generalization is essential to life because we never encounter thevery same stimulus twice. Twoportions of a food you like are never exactly the same;two chairs you sit on are never exactly the same. But as a result of generalization,you know what to do when you encounter the food or a chair.

After Pavlov had established the principle of stimulus generalization, he wenton to demonstrate stimulus discrimination. By continuing to present food whenhe rang the original bell and omitting food when he sounded a different bell, Pavlovsoon trained the dog to salivate much less to the sound of other bells than to thesound of the first bell. The dog had learned to discriminate between the trainingbell and the other ones. You may have noticed that pets have a generalized responseto any sound that might be associated with food, but they also discriminate andrespond less to the rustle of a grocery sack or the opening of a plastic containerthan to the rustle of a food bag or the opening of a can-assuming, of course, thatthe former have not been associated with food.

Pavlov also discovered that by taking advantage of the stimulus discrimina-tion effect he could condition dogs to behave in abnormal ways. First, he projecteda circle and an ellipse on a screen and conditioned dogs to discriminate betweenthe two. Then he gradually changed the shape of the ellipse so that it looked moreand more like a circle. Even when the difference in appearance was very slight, thedogs still discriminated successfully. When the difference became too small for thedogs to see and discrimination was impossible, the dogs acted strangely disturbed.Some became restless and aggressive; others became apathetic and developedmuscle tremors and tics. Experiments like this suggest that some forms of abnor-mal human behavior may result from problems and difficulties that occur throughclassical conditioning-being unable to discriminate an object or a situation leavesyou not knowing what to do, which can be highly frustrating and stressful.

Nature versus Classical ConditioningUp to this point, our discussion of classical conditioning has focused on generalprinciples that might apply to all species and all behaviors. However, the power ofclassical conditioning depends on the degree to which the organism is prepared byits biological makeup to learn to associate a particular pair of stimuli-an issue thatwasn't investigated until the latter part of the 20th century.

Researchers have now shown that it can be extremely difficult or even im-possible to establish certain conditioned responses. For example, it is easy to con-dition a rat to avoid water with a distinctive taste when the drinking of thatwater is followed by sickness induced by the experimenter. Rats are pre wired toassociate taste (and smell) with foods. On the other hand, for rats and other ani-mals that rely on taste, sounds are not a cue associated with eating. So when re-searchers paired a sound with the act of drinking unflavored water and thenmade the rats sick, the rats did not learn to avoid the water when they heard thenoise (Holder, Bermudez-Rattoni, & Garcia, 1988). Rats are, however, prewired toassociate sounds with danger and pain. In the same series of studies, John Gar-cia and colleagues verified this by easily conditioning the rats to avoid plain waterby again pairing it with the sound but this time following the pairing with elec-tric shock.

Thus, psychologists now know that each species is predisposed by heredity tolearn to react to some stimuli but not others. For example, the ease with whichtaste aversions are learned is higWy adaptive for all animals that use taste as a cuefor what to eat. Such aversions are a special case of classical conditioning that"violates" the principles of timing and repeated trials discussed earlier, in thatthey can be learned in a single trial that spans several hours (e.g., see Welzl,D'Adamo, & Lipp, 2001).

Stimulus discriminationThe tendency of a CR to beweaker or not to occur to CSsthat are dissimilar to the origi-nal CS or that have undergoneextinction.

(c) After 5 years, Kate-having been conditioned tofeel weak and dizzy at theprospect of visiting her pedia-trician-no longer gets sucha reaction on entering thedoctor's office. In classicalconditioning, what term isused to describe this turn ofevents?

(d) Duane ate "Zesto" po-tato chips once and happenedto become severely nauseousa couple of hours later. Nowhe won't eat any kind ofpotato chips. What classi-cal conditioning phenomenais Duane displaying?

In evolutionary terms, those species that did not readily learn to avoid foodsthat made them sick were much less likely to survive long enough to reproduce.Birds use visual cues for eating, and experiments have shown that they can easilyrelate sickness to shapes, patterns, and perhaps colors, but not to taste. In humans,too, conditioning is easier and CRs are stronger with certain stimuli. Like rats,humans usually learn taste and food aversions as a result of sickness-not wheneating the food is associated with accidents such as breaking a limb or gettingcut. Also, sickness is much more likely to result in conditioned aversion to the ac-tual foods involved than to aversion to related stimuli such as where the food waseaten (Garb & Stunkard, 1974; Logue, Ophir, & Strauss, 1981).

Other Classical Conditioning PhenomenaWe saw earlier that phobias and other fears can be acquired through classical con-ditioning. Here we look at an example of how this happens, in the context of aclassic (and nowadays unethical) experiment conducted in the early 1920s. Otherphenomena discussed in this section relate to the relationship between condition-ing and illness, immune responses, and drug reactions.

Conditioned Terror: The BabV Who Feared Santa Claus What salivating dogs andblinking rabbits tell us about more complicated forms of human behavior wasdemonstrated in a famous (most would say infamous) experiment performed byJ. B. Watson and Rosalie Rayner and initially reported in 1920. Watson and Rayner'ssubject was an II-month-old boy known as "Little Albert." The experiment wasdesigned to demonstrate that unwarranted and persistent human fears (such asphobias) could be established through classical conditioning. This wasn't knownat the time-psychoanalysts, for example, were offering quite different explanationsfor such fears, based on unresolved Oedipal complexes or the like (see Chapter 9).

Albert initially had no fear of white laboratory rats, one of which served as theCS. Each conditioning trial consisted of one experimenter presenting the rat toAlbert as the other stood behind him and made a loud, frightening noise (the DCS)by hitting a steel bar with a hammer. Before long, the rat alone caused Albert to be-come afraid and cry, and thus the CR was established. To an extent, the fear alsogeneralized to other things such as a dog and a fur coat-as well as a white SantaClaus beard worn by Watson. The experiment was actually relatively haphazard,and Albert's fear wasn't always consistent and predictable, but Watson and Raynerdid for the most part succeed in what they had set out to accomplish: Albert's be-havior closely resembled a phobia for just about anything white and furry (for thedetails of the experiment, see Harris, 1979).

Watson and Rayner intended to remove Albert's fearful behavior through"counterconditioning"-that is, by presenting a treat along with the rat. But at thatpoint Albert's mother, who apparently had been unaware of what was going on,removed him from the study. Nothing is known about Albert or his fears after that.If the fear persisted, Albert may even have come to be afraid of sidewalk or depart-ment store Santa Clauses-without ever knowing why.

Although the Little Albert experiment was categorically unethical according totoday's APAguidelines (see Chapter 1) and indeed anyone's standards, it did castlight on the unexplained fears we often display as adults. At least some of them areconditioned responses, learned in childhood through some long-forgotten pairingof stimuli-perhaps an experience that did not even impress us much at the time.Similarly, the experiment helps explain many of our unreasonable preferences. Aliking for certain kinds of people may go back to a childhood experience in whicha person with that kind of face or body build or mannerisms elicited responses ofwarmth and pleasure.

Conditioned Illness Classical conditioning can cause illnesses that have no physi-cal basis. In one experiment, rats were put into a coma with a heavy dose of in-sulin, producing the severe reaction known as insulin shock (due to depletion ofglucose, the blood sugar used by the brain). The drug was administered with a hypo-dermic syringe while a bright light was on. The association of the syringe and lightCSs with the insulin DCS and the resulting coma resulted in a spectacular kind ofconditioning. When the same light was turned on and the same syringe was usedto inject a harmless salt solution, the animals went into a coma characteristic ofinsulin shock (Sawrey, Conger, & Turrell, 1956). More recent research has demon-strated that by pairing the injection of insulin with normally irrelevant stimuli suchas a strange odor, noise, and light, it is possible to condition a decrease in blood sugarlevel in humans (Fehm-Wolfsdorf, Gnadler, Kern, Klosterhalfen, & Kerner, 1993).

Such experiments indicate the ways in which classical conditioning can pro-duce some of the strange physical symptoms that may bother us as adults. Anasthma sufferer may have been conditioned-not in the laboratory but by somereal-life experience-to have an attack upon walking into a particular room, see-ing a particular person, or even looking at a certain kind of picture on TV.Eventsthat occur in our lives, unimportant in themselves but associated with past expe-riences, may make us develop headaches or become sick to our stomachs. We maysuddenly and inexplicably show all the symptoms of having a cold, or we may ex-perience heart palpitations, high blood pressure, dizziness, or cramps.

Conditioning and Other Physiological Effects Other studies have shown that the im-mune system, which produces antibodies to help fight infection and disease, canbe influenced by conditioning. Rats were conditioned to associate a novel tastewith unpleasant physical reactions to a drug that suppresses the immune system.Later, when the animals were exposed to the novel taste but not to the drug, theproduction of antibodies was suppressed (Ader, Cohen, & Bovbjerg, 1982). Thisresearch demonstrates that each of us is potentially vulnerable to changes in ourresistance to disease as a result of classically conditioned responses occurring with-out our being aware of them.

Conditioning plays a role in the development of some of life's more pleasantphysical responses as well. Think, for example, of your reaction to the mention ofa restaurant in which you have enjoyed a series of memorable dinners. Do you sali-vate? Or think of the pleasure that you might feel in response to a fragrance that youassociate with an attractive friend or romantic partner. Do you become aroused?

The power of such conditioning was demonstrated in a study of male rat pupsthat suckled females whose nipples and vaginal odors had been altered with a dis-tinct lemon scent CS. The rats were then weaned and never again exposed eitherto females or to the lemon scent until they were sexually mature. At that point,they were paired with sexually receptive females. The vaginal areas of some fe-males were treated with lemon scent; those of others were not. The contrast insexual response was dramatic (see Figure 4.6). A male rat placed with a lemon-scented female approached her readily and ejaculated quickly; male rats placedwith females that had no lemon scent were considerably slower to respond (Fil-lion & Blass, 1986).

In another study, 5-day-old rats were exposed to an orange scent followed byan injection of morphine. When they were 10 days old, they showed a marked pref-erence for the scent of orange, a preference that could be reversed by injecting adrug that blocks morphine's effects. Thus, it appeared that previous association ofthe orange scent CS with the morphine DCS caused a release of morphine-likechemicals in the brain (see Chapter 2). This conclusion is strengthened by the datashown in Figure 4.7. The conditioned rats, when later exposed to the orange scentalone, had a higher threshold for pain. These and many studies like them confirmthat early conditioning can have powerful effects on later behavior.

FIGURE 4.6 Early condi-tioning of later sexual be-havior. Male rat pups thatsuckled females treated witha lemon scent became muchmore sexually aroused by thelemon smell when they reachedpuberty.

ExpectancyThe subject's expectationthat the UCS wi II follow theCS-the basis for the cogni-tive explanation of classicalconditioning.

The Power of Expectation: Drug Reactions without Drugs In the view of many psy-chologists, conditioning can be best explained by the learning of expectancies.In other words, what animals and humans learn in classical conditioning is theexpectation that a particular conditioned stimulus will be followed by an un-conditioned stimulus (Bolles, 1972). In humans, for example, it is possible to con-dition the "sweaty palms" measured by the galvanic skin response (GSR) simply

by telling research participants that they will receivea shock every time they hear a tone. It is not neces-sary for the experimenter to deliver the shock; the ex-pectation of the outcome is sufficient to produce theresponse.

It is now believed that Pavlov's dogs learned to sali-vate to the sound of the bell because they learned toexpect the sound to be followed by the food. Indeed,Pavlov himself concluded that all organisms respondnot only to stimuli but also in anticipation of the stimuli.In one series of experiments, Pavlov paired the soundof a bell es with injection of a drug DeS; the drug wasapomorphine, a stimulant that in high doses producessymptoms such as agitation and vomiting. It turned outthat anticipation of the drug on hearing the bell pro-duced the symptoms. After a period of time, even thebell sound wasn't essential. Simply opening the con-tainer of hypodermic syringes was sufficient to pro-duce the symptoms (Siegel, 1983).

A dramatic example of this phenomenon has beenobserved among former drug addicts, who begin toshow withdrawal symptoms and a desire to return todrug use when they are exposed to drug parapherna-lia, other individuals who use drugs, or even the neigh-borhood where they secured their drugs. This is whypeople involved in rehabilitation of drug addicts sug-gest that they move to a new setting where encounters

FIGURE 4.7 When the smell of orange can act as apainkiller. Three groups of lO-day-old rat pups were placedon a heated surface while being exposed to the scent of or-ange, and the delay in their lifting of their paws to escape theheat Was measured. Note that the rats that had been condi-tioned S days earlier to associate orange scent with morphinetook almost twice as long to lift their paws as did those ratsthat had been conditioned with a simple salt solution andthose that had not been conditioned.

18

16en 14""0c0'-' 12Q)

!!!..3: 10ctlQ.

OJc 8~.~ 6>.ctlQi 40

2

0

• What is operant conditioning and how did Skinnerstudy it?

with the cues for drug use are less likely to occur-that is, where the environmentdoes not elicit anticipation of a "high."

OPERANT CONDITIONINGClassical conditioning primarily concerns reflexive behav-ior, but reflexes certainly aren't the only form of behavior FOCUS QUESTIONSthat is involved in learning. If a rat is placed in a cage, forexample, it exhibits many behaviors that seem to be sponta-neous and self-generated, not predetermined responses tospecific stimuli. It may sniff at the cage, stand up to get abetter look at things, scratch itself, wash itself, and touch var-ious parts of the cage. Similarly, babies in a crib display manyspontaneous actions. They move their arms and legs, try to turn over or grasp ablanket or the bars of the crib, turn their heads and eyes to look at various objects,and make sounds with the mouth and vocal cords.

The actions exhibited by the rat or the babies are prewired in part. Rats, babies,and most other animals have a natural tendency to explore their environment-especially if it's a novel one. But exploratory behavior and the like are initiatedvoluntarily instead of reflexively. The organism "operates" on its environment andoften brings about some kind of change in it. Hence, this type of activity is calledoperant behavior, a term coined by B. F. Skinner (1938) (also see Chapter 1). Likeinborn reflexes, operant behavior can be modified through learning. Here, how-ever, it is the relationship between behavior and its consequences-that is, thecontingency-that is of primary interest. Why do you go to school or work (orboth) each day and perform the appropriate behaviors? In Skinner's terms, you doso because you expect a payoff.

This operant conditioning occurs both in the lab and in everyday life. Thatis, in addition to contingencies imposed by experimenters, there are natural con-tingencies imposed by the environment. Animals-including humans-learn toperform behaviors necessary to survival, such as those that produce nourishmentor avoid danger. Even animals that have prewired behaviors associated with sur-vival must still learn where to look for food. And humans, of course, are subject tocountless contingencies imposed by other humans at home, at school, at work-indeed, throughout everyday life. We don't always behave as predictably as otheranimals, though. As discussed later in the chapter, in addition to thinking aboutconsequences, we often think about the contingencies themselves, which canwreak havoc with predictions based on operant conditioning principles alone.

• What basic terms and procedures are involved inoperant conditioning?

B. F. Skinner's Magic BoxSkinner revolutionized the study of learning and conditioning with the special ap-paratus shown in Figure 4.8, designed for rats. (Similar apparatuses have beendevised for many other research animals.) When Skinner first placed a hungry ratin the apparatus, it engaged in many kinds of spontaneous operant behavior as it"checked out" its new surroundings. Eventually, besides doing other things, it hap-pened onto the bar and caused it to move, at which point the apparatus dropped apellet of food into the feeding cup beneath. However, little or no learning took placeat this point. In cognitive terms, we could say that the animal did not "notice" arelationship between the bar and the food, so it simply ate the food and continuedits explorations as before. Eventually, it accidentally pressed the bar again, caus-ing another pellet to drop. After a number of these instances, the rat finally formed

ContingencyThe relationship between be-havior and its consequences.

Operant conditioningThe imposition of contingen-cies, either deliberate ornatural.

FIGURE 4.8 Learning inthe Skinner box. With thissimple but ingenious inven-tion-a box in which a bar-press response automaticallyreleases a pellet of food orother reward-Skinner (shownin the photo) demonstratedmany of the principles ofoperant behavior that survivetoday. In the past, electricrelay equipment controlled theapparatus and a "cumulativerecorder," such as the onein the bottom right corner ofthe photo, tallied responses.Nowadays, computers are usedinstead. Note the steel gridbars the rat is standing on:These can be used to deliverelectric shocks.

Shaping and successiveapproximationsA procedure for quickly estab-lishing a contingency, suchas barpressing by rats or key-pecking by pigeons, by reward-ing successive approximationsto the target behavior.

an association between the act of pressing the bar and the appear-ance of food. It now began pressing the bar as fast as it could (Skin-ner, 1938, 1953).

The Skinner box prompted a host of new studies of learning. Thesestudies showed that operant conditioning had many parallels withPavlov's classical conditioning. Like conditioned reflexes, conditionedoperant behavior is subject to extinction. That is, if the rat no longerreceives food for pressing the bar, it eventually stops. Spontaneous re-covery also occurs: After barpressing is extinguished and the rat is re-moved from the Skinner box for a while, if the rat is then returned, itstarts pressing the bar again.

Experiments with pigeons clearly demonstrated that stimulus gen-eralization occurs in operant conditioning. A pigeon that had learnedto obtain food by pecking at a white key would also peck at an off-white or perhaps even a red or green key. But if only the white key

'" produced food, the pigeon learned stimulus discrimination and ig-I nored the keys of other colors.~l! Shaping and Successive Approximations

On its own, as described earlier, a rat might take quite some time to"figure out" the relationship between barpressing and its consequences-that is,getting food. Therefore, when conducting operant conditioning experiments, psy-chologists usually begin with shaping and successive approximations, a proce-dure that significantly reduces the time required for a rat to learn to press a bar or apigeon to peck a key.This procedure is also used in training animals to perform muchmore complex behaviors, including the spectacular ones you see in stage or aquar-ium acts. It can likewise be effective therapeutically, as discussed in Chapter 14.

As noted earlier, a rat will move around and explore a Skinner box when firstplaced in it. However, instead of waiting for the rat to "blunder" into the bar, theresearcher can shape the rat's behavior by using successive approximations to thetarget behavior of barpressing. The conditioner uses a manual switch to release afood pellet: at first when the rat heads in the direction of the bar, then when ittouches the bar, then when it touches the bar and causes it to move, and finallyonly when it presses the bar sufiiciently to trigger the food release mechanism.When shaping is complete, the automated equipment takes over. With careful at-tention to consistency and timing, a rat's barpressing can be shaped in less thanan hour. In real life, many of our behaviors are shaped by natural contingenciesor simply by feedback. An infant's behavior of eating with utensils is graduallyshaped into perfection by success in getting food from bowl or plate to mouth.Similarly, based on feedback from others and from the sounds produced, a personwho plays a musical instrument gradually makes better and better music.

Note, however, that how the rat presses the bar usually isn't the important thing.Although a rat can be shaped to press the bar in a highly specific manner, such aswith the tip of its left front paw, it is the fact of the barpress than constitutes theoperant. That is, an operant is a class of behaviors that "get the job done," and itdoesn't matter whether the rat presses the bar with its right paw, its left paw, bothpaws, its nose, or even by sitting on it. As Skinner pointed out, this is an impor-tant concept in the generalization of operant conditioning principles across manydifferent situations, behaviors, and organisms. Just as we never encounter the exactsame stimulus twice, we never perform a behavior the exact same way twice. Whatis most important is what satisfies the contingency-along with what variations inthe contingency increase or decrease the likelihood of the behavior. This is why arat pressing a bar might be comparable to a human performing a host of behaviorsthat produce tangible or social reinforcement.

'"gja The technique of shaping is~g used to train animals like'"Co these dolphins to perform£@ tricks.

The same procedure applied to shaping pigeons is illustrated in Figure 4.9. Ahungry pigeon is led step by step, again using food, to perform a behavior that itmight never have performed by accident.

FIGURE 4.9 Shaping a pigeon's behavior. How can a pigeon be taught to peck at that little black dot in the middleof the white circle on the wall of its cage? When first placed in the box, the bird merely looks about at random (AI. Whenit faces the white circle (B), it receives food in the tray below (e). The pigeon is rewarded first for looking at the circle(D), then not until it approaches the circle (E), and then not until it pecks at the circle (F). The final step (not shown)is to deliver the food only when the pigeon pecks at the dot.

FIGURE 4.10 The fourbasic operant conditioningcontingencies. The contin-gency depends on whether thestimulus or event is appetitiveor aversive and whether it ispresented or removed.

Presentor

"add"

Removeor

"subtract"

Positive reinforcementAn operant conditioning con-tingency in which behavioris strengthened because itresults in presentation ofan appetitive stimulus; alsoknown as reward training.

Negative reinforcementAn operant conditioning con-tingency in which behavior isstrengthened because it re-sults in removal of an aversivestimulus; also known as escapeor active avoidance training.

Reinforcement and Punishment: The BasicsSkinner and his many followers popularized the concepts of reinforcement andpunishment and studied them extensively with regard to variations in quality, tim-ing, and consistency. However, the concepts derive from Thorndike's law of effect,introduced in Chapter 1: (l) Any behavior that is followed by a satisfying state ofaffairs tends to be repeated (to be strengthened), and (2) any behavior that is fol-lowed by an unsatisfying or annoying state of affairs tends not to be repeated (tobe weakened or suppressed) (Thorndike, 1911). The first half of the law of effectcorresponds to reinforcement, the second half to punishment.

But there's more. Skinner's early research culminated in a clear distinctionbetween "positive" and "negative" reinforcement (Skinner, 1953). Others lateradded a distinction between positive and negative punishment. All operant condi-tioning is based on these four basic contingencies, as defined in this section and

illustrated in Figure 4.10. It is important tounderstand these contingencies because eachhas quite different effects on behavior. (Inlearning them, bear two things in mind. First,forget about other areas of psychology thatuse positive and negative to apply to emotions,moods, or the like. Here, positive simply meansthat something is added or presented, and neg-ative means that something is subtracted or re-moved. Second, consider only what happenswhen the behavior actually occurs; if youthink in terms of what happens when the be-havior doesn't occur, you're likely to get theseterms backward.)

In positive reinforcement, a "goody"-technically, an appetitive stimulus-ispresented whenever the desired behavior occurs. ("Appetitive" is not restricted tofood; it refers to any stimulus or event the organism likes, seeks, will approach,etc.) The behavior then tends to be repeated, become more likely, or increase infrequency. That is, the behavior is strengthened. Positive reinforcement is alsoknown as reward training, and it is taking place whenever an organism engages ina behavior that produces something it likes and wants. In a Skinner box, a ratpresses a bar to get food. More generally, positive reinforcement occurs with assimple and immediate an activity as opening the refrigerator door to get some-thing to drink or flipping a switch to turn on a light and as complex and prolongedan activity as working for a week or even a month to get a paycheck or completingyears of coursework to earn a diploma.

Negative reinforcement means that something aversive (unpleasant or fear-ful) is removed, goes away, or doesn't happen whenever the desired behavior isperformed. Again, the behavior is strengthened. In the version called escape train-ing, the aversive object or event is actually experienced, and the desired behaviormakes it go away. In contrast, in active avoidance training, the aversive thing isnot actually experienced-the desired behavior prevents it from occurring at all. Arat might press the bar to turn off an electric shock (escape), or if it has some wayof predicting when the shock will occur-such as a light that precedes the shock-it may soon learn to press the bar in advance and not have to endure the shock(active avoidance). However, this particular training can be very difficult to ac-complish with rats because of their built-in responses to pain. It works much bet-ter in an experimental setup such as that illustrated in Figure 4.11. When a dog isplaced in the wired compartment and electric shock is administered, the dog quicklylearns to jump across the hurdle to the other side to escape the shock. If some kindof warning is given, such as a light or tone that occurs before the shock is admin-

Positive PositiveReinforcement Punishment

Negative NegativePunishment Reinforcement

istered, the dog will quickly learn to jump the hurdle when the stimulus occursand avoid the shock entirely. In humans, a simple example of negative reinforce-ment is nagging a child to clean up his or her room; when the room is cleaned up,the nagging stops. Somewhat more complex examples are superstitions and pho-bias, discussed later in the chapter.

Positive punishment is what we normally think of as punishment and is ap-plied to behaviors that we want to eliminate. Something aversive or unpleasantfollows when the behavior occurs, and the behavior tends not to be repeated, tobecome less likely, or to decrease in frequency-that is, it is either weakened orsuppressed. Another term for this type of punishment is passive avoidance trainingbecause the aversive object or event isn't experienced if the subject or participantsimply does nothing (but remember, focus on what happens when the behaviordoes occur). If a rat that had been previously trained to barpress for food was in-stead shocked whenever it pressed the bar, barpressing should decrease. A humanexample is scolding or spanking a child for misbehaving-or having an accidentwhile driving and talking on your cell phone. The use of positive punishment as away of controlling children's behavior and directing their development has longbeen criticized by some psychologists, but certainly not by all, as we see later inthe chapter.

Finally, in negative punishment, a "goody" (or appetitive stimulus) is removedor taken away, and the target behavior is weakened or suppressed. Traditionallythis type of punishment is called omission training. A rat that was previouslytrained to press a bar for food now gets food automatically unless it presses the

FIGURE 4.11 Negativerei nforcement. The dog,placed in a shuttle box, firstlearns to escape and thenlearns to avoid the shockentirelv·

Positive punishmentAn operant conditioning con-tingencv in which behavioris weakened or suppressedbecause it results in presen-tation of an aversive stimulus;also known as passive avoid-ance training.

Negative punishmentAn operant conditioning con-tingencv in which behavioris weakened or suppressedbecause it results in removalof an appetitive stimulus; alsoknown as omission training.

(e) By continuing to playregularly, Marcia gets betterand better at softball. Whatkind of learning is Marciaexperiencing?

(f) What do psychologistscall the kind of condition-ing an elephant undergoesin learning to do complexstage acts?

(g) You're late to work andyour boss docks your pay.What operant conditioningcontingency is in effect?

bar. Parents may use negative punishment with their children without even realiz-ing that they are using punishment. They give their children privileges, allow-ances, and other goodies freely, regarding themselves as being very "rewarding"toward their children. There is no actual contingency in effect until the child mis-behaves-then negative punishment kicks in along lines of "No TV for a week,""There goes your allowance," or "You're grounded." Another example of negativereinforcement is the long-standing time-out procedure that teachers sometimesuse in working with children who "act out" and are disruptive in class. Because itis believed that disruptive behavior is maintained by giggles from classmates andattention from the teacher-even scolding can be reinforcing for children who getlittle other attention from adults-the child is temporarily isolated from the class-room. Thus, attention from the teacher and classmates is removed, and the dis-ruptive behavior declines.

OPERANT CONDITIONINGPHENOMENA AND APPLICATIONS

FOCUS QUESTIONS• How are the timing and consistency of operant

conditioning contingencies important?• Why is reinforcement preferable to punishment?

• What maintains superstitions and phobias?• How do biological predispositions affect operant

conditioning?

As with classical conditioning, many variables determine the ef-fectiveness of operant conditioning, the rate at which the behav-ior occurs, and the persistence of the behavior if the contingencyis no longer in effect (that is, if the reinforcer or punisher is dis-continued). These and other important considerations in oper-ant conditioning are discussed in this section.

Delay of Reinforcement and PunishmentIt has been found in most laboratory animal experiments that immediate re-inforcement produces the most rapid learning. Any delay reduces the amount of

learning, and too long a delay producesno learning at all, as is shown in Fig-ure 4.12. The same holds true for veryyoung children and for extremely men-tally retarded persons. For example, itis difficult to teach an 18-month-old tostay away from electrical plugs and out-lets unless the child is immediately pun-ished for the behavior or distracted andreinforced for some alternative accept-able behavior. "Wait until your father (ormother) gets home" has little, if any, ef-fect on a very young child.

With older children and normaladults, however, delays can be quite longand the contingency is still effective. Asnoted earlier, you may work for a monthor even longer to get a paycheck, and youmay work for years to get a college degree.And with older children, the threat ofpunishment at the end of the day whenone or the other parent gets home from

FIGURE 4.12 Oops ••• the reinforcement came too late. The steep dropin the curve shows how rapidlv learning fell in an experiment in which a foodreinforcer was presented when rats pressed the bar in a Skinner box. Differentrats received reinforcement delavs ranging from a few seconds to about 2 min-utes. Note that there was no learning at all bV the rats that received delavs ofabout 100 seconds.Perin, 1943.

100

75OJ0u(/)

OJ 50c'c~OJ-J 25

0 0 50 100Delay of reinforcement (in seconds)

work can be very effective-perhaps even more effective than immediate punish-ment if the child truly fears what might happen and ponders it all day.

In each case, a good explanation is verbal mediation-"bridging the gap" be-tween behavior and its consequences by thinking about and reminding oneselfthrough "self-talk" that the consequences will eventually occur. Laboratory ani-mals and very young children lack the necessary language and memory skills, sothey consistently behave in accord with the curve in Figure 4.12. Older childrenand adults who have the necessary cognitive skills easily mediate very long timedelays.

The Power of Partial ReinforcementExperimenters have compared the effects of continuous reinforcement (rein-forcement for each instance of a behavior) with those of partial reinforcement(reinforcement on some occasions but not on others). They have found that al-though learning generally takes place more rapidly with constant reinforcement,the behavior is more persistent (that is, more resistant to extinction) with partialreinforcement (e.g., see Robbins, 1971). That is, when reinforcement is discontin-ued, partially reinforced behavior takes a lot longer to extinguish than continu-ously reinforced behavior does.

To understand why, consider the following. You have two TV sets, one thatturns on every time you press the button on the remote (continuous reinforce-ment) and one that-unpredictably-turns on sometimes the first time you pressthe button, sometimes after several presses, and sometimes after half a dozen ormore (partial reinforcement). Now, suppose that something unknown goes wrongwith both sets and prevents them from coming on at all (extinction). Which re-mote will you continue to press longer before giving up and calling a service tech-nician? Clearly, the partially reinforced one. With that remote, it will take you muchlonger to "discriminate" that the set truly is broken-that it isn't just the button onthe remote (e.g., see Capaldi, 1966).

This finding has many applications to real-life situations. For example, in thewild, animals don't get rewarded for every instance of hunting for food. But if ananimal learns that food is occasionally available in a particular place, it will con-tinue to return there "just to make sure," perhaps long after food is unavailable.

In humans, a powerful example is what parents sometimes do when theirchild throws a temper tantrum. Suppose a little boy starts having temper tantrumswhenever he asks for something and his request is denied. His parents try to ig-nore his behavior-but every once in a while, just to quiet him down, they give inand let him have what he wants. What they have done is set up a situation in whichthe operant behavior of temper tantrums (the very thing they would like to elimi-nate) produces the candy, toy, or whatever else the child wants, on a schedule ofpartial reinforcement. When they eventually get around to realizing that they mustignore or otherwise not reinforce the tantrums every single time they occur, theyfind that the tantrums are extremely hard to eliminate. The child has come to ex-pect to be reinforced only on some occasions, and it may take him quite a longtime to figure out that the tantrums don't work anymore. This is even more likelyif the parents-with the best of intentions-hold out longer and longer before oc-casionally giving up and giving in to the child. Here, the parents are shaping pro-gressively more intense tantrums at the same time that they are inadvertentlyteaching the child, "Keep trying long enough and hard enough and it might work."

Psychologists differentiate between two basic types of partial reinforcementschedules, as indicated in Table 4.1. In the first type, called ratio schedules, thereinforcement is delivered only after the subject responds a certain number of times:A rat would receive a food pellet only after pressing the bar several times or more.

Continuous reinforcementReinforcing everv instance ofa behavior.

Partial reinforcementReinforcing onlV some instancesof a behavior.

Ratio scheduleA partial reinforcement sched-ule in which reinforcementoccurs onlV after a number ofresponses.

Partial reinforcement mav beadministered in four basic wavs.

Interval scheduleA partial reinforcement sched-ule in which reinforcementoccurs onlV for the first re-sponse after an amount of timehas elapsed.

Fixed scheduleA partial reinforcement sched-ule in which reinforcementoccurs after a specific numberof responses or for the firstresponse after a specificamount of time.

Slot machines provide a goodexample of variable rein-forcement. Because playerscannot tell when a jackpotwilloccur, they often con-tinue to play until they runout of money.

TABLE 4.1Schedules of Partial ReinforcementType ofSchedule

WhenReinforcement Occurs

Ratio

Fixed For a specific numberof responses

You're paid "piece rate" for, say,every 10 chopping boards you makein a woodworking shop.

You get a payoff after differingnumbers of plays on a slot machine(the average number is based onprobability).

For a varying numberof responses aroundsome average

Interval

Fixed For the first responseafter a specific amountof time has passedsince the last one

For the first responseafter a varying amountof time, on average, haspassed since the last one

Your mail comes at the same timeeach day, so that's when you checkyour box for a magazine or a friend'sletter.

E-mail from a friend may come atany time, so you only occasionallyretrieve one when you check (theaverage is determined by your friend).

In the second, interval schedules, the reinforcement is delivered for the first re-sponse after a certain amount of time has elapsed since the last response: A ratwould receive a food pellet for, say, making a response every few seconds or more.

Both types of schedules are subject to another kind of differentiation. If eitherthe ratio or the interval is the same each time, the schedule is termed a fixedschedule. Thus, the rat might be reinforced for exactly 5 barpresses or after ex-

actly 30 seconds. On the other hand, if the ratio or interval changes unpredictablyaround some "average" value, the schedule is termed a variable schedule. Here,the rat might be reinforced for anywhere from 1 to 9 barpresses, in the long runaveraging 5, or might have to wait anywhere from 5 to 55 seconds for a barpressto be reinforced, in the long run averaging 30.

Either kind of variable schedule produces greater resistance to extinction thaneither kind of fixed schedule. Predictability is the key, as in the example of parentswho occasionally reinforce temper tantrums. On a variable ratio schedule in par-ticular, the child cannot tell which instance of a tantrum might be rewarded, andagain may take a long time to learn that none will be-when the parents "change"the contingency to extinction.

When There's a Choice: Reinforcement versus PunishmentSkinner (1953) strongly favored using positive reinforcement as a way of control-ling children and teaching them appropriate behavior. For misbehavior-whichhe viewed as always maintained by consequences of some kind-he recommendedextinction (ignoring the behavior) followed by positive reinforcement for appro-priate alternative behavior that achieves the same end. Thus, for example, ratherthan scold, spank, or isolate a child who behaves aggressively to get a toy awayfrom another child or acts disruptively to get attention from a parent or teacher,Skinner suggested that we ignore such behavior while at the same time reinforcingthe child for using appropriate ways to get what she or he wants. Praise and other-wise reinforce the child for politely asking for a toy and for taking turns and shar-ing; likewise, reinforce a child for properly asking questions and doing work thatresults in attention from adults.

Beyond the much more pleasant atmosphere and "good feelings" that positivereinforcement tends to foster, Skinner and others emphasized its advantage ofbeing more informative-that is, it tells the child or adult or pet specifically whatto do, whereas punishment conveys only what not to do. Thus, punishment alonemay eliminate or suppress one misbehavior, only to allow another misbehavior tocrop up in its place. A child who is punished for aggressively going after a toy maybecome sneakier and try to steal the toy. An adult who is imprisoned for muggingpeople to get money may turn to burglary upon release. A dog who is hit with arolled-up newspaper for urinating on the rug in one part of the home may simplyurinate in another room. It is much better, Skinner and others have said, to rein-force appropriate behaviors by teaching children appropriate ways to get whatthey want, by teaching people who engage in criminal behavior how to earn moneylegitimately, and by shaping a dog to go to the door and "ask" to go outside-andinitially following the desired behavior with treats and praise.

Beyond teaching only what not to do, punishment has a number of other well-documented problems. For one, it may simply teach the child to suppress the be-havior when the parent or other adult is around, the same way you might loweryour speed when a police car is on the highway but exceed the speed limit whenone is not. Other potential effects of punishment-especially corporal or physicalpunishment-on children are discussed further in AMatter of Debate on page 158.

So positive reinforcement is preferable to punishment. However, as you mayhave noticed, the picture painted earlier is somewhat rosy. Some behaviors can'tsimply be ignored, because they are too dangerous and harmful-such as a youngchild playing with knives or matches or running out into the street. A slap on thehand when a child reaches toward a forbidden object may be the only way to pre-vent damage or even serious injury. Likewise, societies worldwide can't ignorecriminal behavior while they try to teach muggers and other criminals alternativeways to obtain money. Use of punishment appears to be unavoidable at times,

Variable scheduleA partial reinforcement sched-ule in which reinforcementoccurs after a varying numberof responses or for the fj rstresponse after a varying amountof time, in each case aroundsome average value.

Spanking ChildrenIn July 2002, psychologist Elizabeth Gershoff of Colum- • It evokes in children anxiety and anger that can seri-bia University's National Center for Children in Poverty ously disrupt family relationships, and it can create apublished a journal article on the effects of corporal pun- vicious cycle within a family in which the child mis-ishment on children. It was a major effort that reviewed behaves, the parent punishes, and the punishmentand analyzed some 6 decades of research, and it was leads to further misbehavior and harsher punishment;picked up and described by various news media under such escalation can lead to physical child abuse by theheadlines like "Spanking Is a No-No" and "Spanking May parents.Make Kids Violent." Here, we briefly describe Gershoff'sfindings and leave the matter for you to decide.

The article began by noting others' findings that al-though eleven nations now ban the use of corporal pun-ishment such as spanking children, support for corporalpunishment remains strong in the United States-with94% of U.S. parents spanking their children by the timethe children are 3 or 4 years old. Moreover, in spite ofhundreds of studies pro and con, psychologists remainsharply divided on the issue. Articles and debates haveyet to resolve the ongoing controversy or even to draw aclear line between what might constitute reasonable andappropriate corporal punishment and what constitutesphysical child abuse. With that as a backdrop, Gershoffwent on to review prior research on the effects of corpo-ral punishment, including spanking. The following is asummary of what she found.

• It can be highly effective in stopping misbehavior inthe immediate sense (which Skinner also acknowl-edged regarding punishment in general).

• Used alone, it does not tend to foster children's "inter-nalization" of moral standards of right and wrong;among other things, it may instead teach them to focuson not getting caught.

• It models aggression (as noted in Chapter 1of this text)and has consistently been shown to be associated witha wide variety of factors related to aggression-includ-ing legitimizing violence as a way of dealing with prob-lems, which can surface later in adult relationshipswith spouse and children.

• It has been strongly implicated in the development ofsubsequent delinquent, antisocial, and criminal be-havior-and, to a lesser extent, adult mental healthproblems.

This is not a pretty picture. The subsequent analysisby Gershoff largely verified each of these points, afterwhich she went on to describe problems with this kindof research-such as the need for further study of the fre-quency and severity of corporal punishment, the effectsof different kinds of corporal punishment (spanking ver-sus slapping, paddling, or whipping), whether the parentsadminister the punishment systematically or impulsively,what other forms of discipline are used in conjunctionwith corporal punishment, and how the child perceivesthe punishment and reacts to it.

Yet another important factor noted was culture andethnicity. In some ethnic groups (as in the United Statesas a whole), corporal punishment of children is viewedas quite acceptable and the child may therefore view it asacceptable as well and not react in the negative waysnoted.

Gershoff was also careful to point out that much ofthe research was correlational in nature, which meansthat clear-cut conclusions about cause and effect cannotbe drawn. As has often been pointed out, for example,"naturally" aggressive children may elicit physical pun-ishment by parents.

Still, the sheer volume of the research pointing the otherway seems overwhelming. Based on your knowledge andpersonal experience, what do you think?

cf ····································To research this topic further, go online with InfoTracand use your own term or the following suggestions:• Gershoff, Elizabeth• spanking children• internalization

although it should always be kept to the minimum necessary and, in the case ofchildren, accompanied by explanations appropriate to the child's level of compre-hension. Finally, like ignoring undesirable behavior, punishment is most effectivewhen combined with reward-when the "wrong" behavior is punished and the"right" behavior is reinforced.

The Power of Superstitions and PhobiasDo you have a "lucky" sweater that you always wear to exams because it helps youget good grades? Do you win more ball games when you wear a certain pair of socksand carefully pull on the right one before the left one? Do you always performsome sort of ritual if you spill salt or break a mirror? Do you avoid walking underladders? That is, do you have superstitions?

Or, as noted earlier in the chapter, are you deatWy afraid of certain insects orother animals or of certain situations, such as being in a confined space or beingout in an open one? That is, do you have phobias?

Superstitions and phobias have some things in common. For one, althoughsuperstitions may be handed down from one generation to the next as "dos" and"don'ts" and phobias may be leamed by observing others behaving in a fearful man-ner, each also can be acquired and maintained through conditioning. For another,they are wholly irrational and yet they persist, often for life-and conditioningprinciples help explain why.

Skinner (1953) described the establishment of superstitious behavior-be-havior that occurs and persists in the absence of any actual contingency-in pi-geons as follows. Give a pigeon food every 15 seconds regardless of what it is doing;that is, reinforce whatever behavior the pigeon happens to be engaged in. Eventu-ally, and at random, some particular behavior will be reinforced more often andthe pigeon will in turn perform it more often, in a cycle that leads to the pigeon'sperforming the behavior much of the time. This might be "hopping from one footto the other and back, bowing and scraping, tuming around, strutting, [or] raisingthe head" (Skinner, 1953).

Thus, the pigeon's behavior has been partially reinforced, and we might saythat the pigeon "thinks" that the behavior is related to getting food. Sometimes thisappears to be the case, as when the food just happens to follow the behavior. Theextension of this principle to some human superstitions should be obvious: If youwear your lucky sweater every time you take an exam or perform the sock ritualevery time you play ball, sometimes it will work-you'll get a good grade or you'llwin. Borrowing from cognitive psychology, we know that we have a tendency toremember occasions when something we do works much better than occasionswhen it doesn't; this tendency is known as the fallacy of the positive case. Thus,through partial positive reinforcement, we become convinced that the supersti-tious behavior we engage in is meaningful and helpful.

Phobias and superstitious behaviors that involve negative reinforcement workdifferently. The common elements are fear, which may have been acquired throughinstances of aversive classical conditioning or in other ways, and negative rein-forcement of behaviors that reduce the fear (Mowrer, 1951). In the case of super-stitions, if you've been taught to be fearful of the imagined consequences of spillingsalt or breaking mirrors, you perform the appropriate ritual and the fear goes away.The ritual behavior thus is negatively reinforced and tends to be repeated. In thecase of phobias, when you encounter the feared object or situation, you "escape"from it by running away, calling for help, or-in the case of dangerous snakes andinsects-perhaps attacking the object and killing it. What these behaviors have incommon is that the fear again goes away, along with the object that caused it, pro-viding negative reinforcement and making it likely that you'll engage in the behav-ior in the future. You may take things a step further and never venture into places(such as fields and forests) where snakes and other dangerous things tend to live-thus avoiding the object entirely.

In each case, the fear may be with you for life because you never "test" the out-come by skipping the superstitious ritual or staying around long enough to dis-cover that the feared object usually isn't harmful. Fear reduction through negative

Superstitious behaviorBehavior that occurs and per-sists in the absence of anyactual contingency.

1•••••• l •••••• ~ ••••• J.•••••J•••••• ~•••••• L ••••• J••••••••••••• ,1 2 3 4 5 6 7 8 9 10

Trials

FIGURE 4.13 Results ofan experiment in learnedhelplessness. The rapid risein the solid line shows howquicklV "normal" dogs learnedhow to cope with an electricshock delivered in a shuttlebox. The dashed line showsthe very different behaviorof animals that had acquiredlearned helplessness---andtherefore seemed incapableof learning how to do anythingabout the shock.

40Ol.~C(ij 30~'0x 20Q)-0E

10

0

Learned helplessnessThe belief that no behaviorwill be effective in escapingor avoiding unpleasant orpainful consequences.

reinforcement makes a powerful and lasting impression indeed, although phobiascan be treated very effectively, as discussed in Chapter 14.

When There Is No EscapeMost of the time, when we expect or encounter unpleasant experiences, we dowhatever we can to avoid them. In addition to extreme cases such as people whoavoid normal activities because of phobias, students manage to avoid taking diffi-cult courses and people who are allergic to ragweed stay away from it in the fall.But there are some conditions in which such coping responses can't occur, and insuch situations we may become apathetic and feel helpless.

This was demonstrated in a series of experiments in which dogs were strappedinto the kind of harness used by Pavlov. In one experiment, one group of dogs re-ceived a series of 64 electrical shocks, lasting 5 seconds and delivered at randomintervals. There was no way the dogs could avoid the shocks or escape from thembefore the 5 seconds were up. The next day, each of the dogs was placed in a shut-tle box. From time to time the light inside the box was dimmed, and a few secondslater a shock was administered through the floor of the compartment in whichthe dog had been placed. The animal could avoid the shock altogether by jumpingover the hurdle into the other compartment when the waming light was dimmed,or it could escape the shock by jumping after the electricity was tumed on. If thedog did not jump into the other compartment, the shock continued-this time fora full 50 seconds.

The results of the experiment,shown in Figure 4.13, were dra-matic. Anumber of dogs were usedin the experiment, and all had 10trials in the shuttle box, duringwhich they could leam to avoid orescape the shock. The comparisongroup-dogs with no prior experi-ence with shock-Ieamed the be-havior quickly. But the previouslyshocked dogs mostly failed to leamat all. Most of them simply acceptedthe shock for the full 50 seconds,making no attempt to leap over the

hurdle and instead crouching and cringing in ways that served only to minimizethe shock a bit.

How can we account for the failure of the experimental dogs to learn to es-cape or avoid a severe and long-lasting shock? The experimenters attributed it towhat they called leamed helplessness. That is, while the dogs were in the har-ness, they leamed that nothing they could do would have any effect on whether orhow long they received a shock. In cognitive terms, they had no hope that theycould do anything about the shock, even when they were moved to the shuttle box,so they didn't try (Maier, Seligman, & Solomon, 1969; Seligman & Maier, 1967).

Humans can also be led to acquire leamed helplessness through simple labo-ratory procedures. In one experiment, college volunteers were subjected to a veryloud (but safe) noise. They were told that they could stop the noise by leaminghow to manipulate some control devices, but actually these devices had no effect.Later, when they were placed in another situation where it would have been easy tomove a control lever and tum off the noise, the participants made no effort and sim-ply put up with the noise until the researcher stopped the experiment (Hiroto, 1974).

- Control group•••• Experimental group

(learned helplessness)

Children who are continually yelled at or spankedno matter what they have done may very well acquirelearned helplessness. They may decide that they haveno control over when, how, or why they are punishedand may give up trying to learn what their parents aretrying to teach them. In such a case, attempts to pun-ish them into learning the difference between good be-havior and bad become self-defeating. Some theoristsbelieve that children who grow up in deprived, deteri-orating neighborhoods and see no way to escape arealso vulnerable to learned helplessness.

There are antidotes. As noted in Chapter 1, the orig-inal experiments on learned helplessness were ques-tionable with regard to ethics, but they did open up anew line of psychological investigation. Punishment, ithas been found, is not the only possible cause of learnedhelplessness. An even more common cause is repeatedfailure at any of the tasks we face throughout life-in theclassroom, on the job, in social contexts, or in the worldat large. Although everyone experiences failure, not everyone suffers drastic con-sequences. Studies of learned helplessness offer some clues as to when, how, andwhy this often-tragic result is likely to occur-and what might be done about it.

Suppose you're in love, but the object of your affections rejects you (unrequitedlove). It makes a great deal of difference whether you blame yourself, blame heror him, or blame women or men in general. Blaming yourself usually results inloss of self-esteem and is associated with lack of confidence in the future (Garber& Hollon, 1977). Sometimes it produces significant depression (Rizley, 1978). Theparticular way in which you blame yourself is also important. If you merely blameyour behavior in that one particular relationship, your feelings of helplessnesswill probably be less severe. But if you blame yourself in general-perhaps yourown worth and character-you are much more likely to be in trouble (Peterson &Seligman, 1984). Thus, it is better to think "Well, I just did the wrong thing thattime" than to decide "That's the way I am, and it seems I'm just plain undesirable."

Behavior Modification and Token EconomiesParents who want their children to stop throwing temper tantrums and animaltrainers who want their dolphins to jump through hoops have something in com-mon: Both are trying to condition behavior. All of us constantly try to influencebehavior--our own actions as well as those of the people around us (Stolz, Wien-ckowski, & Brown, 1975). We try to lose weight, quit smoking, get higher grades,or perform better on the job (Boice, 1982). This may be a matter of self-discipline,but operant conditioning reminds us that we're also subject to the contingencieswe encounter. We try to influence other people to give us a good grade or a raise,to show us more appreciation and respect, or to stop doing things that annoy us.In so doing, we often informally practice what psychologists call behavior modi-fication, a technique that is based primarily on operant conditioning principles.

As psychologists use the term, behavior modification refers to any deliberateprogram designed to influence and change behavior by manipulating its conse-quences. If a certain type of behavior works-that is, if it results in reinforcementthrough rewards or praise or even just a boost to self-esteem-it is likely to be re-peated. If it does not produce satisfactory results, it will tend to be abandoned. Inthis fashion, it has been possible, for example, to raise the level of children's social

The inevitable failures experi-enced in life can producelearned helplessness undercertain conditions.

Behavior modificationA technique for changingbehavior, based on operantconditioning principles; alsocalled behavior therapy inclinical settings.

Phase I50

.><:21 40~ co <Il~:;::;_<1l:J 0- 301ii»moO:J"O-<Ilo E 20x ~<Il 0"O'l:C <Il-0- 10

0

Phase II Phase III skills (Yule, 1985) and to teach brain-damaged patients to reduce theirsocially inappropriate behaviors

Token economy Token economy (McGlynn, 1990). In general, behav-abandoned restored ior modification-or behavior ther-

apy, as it is called in clinical settings(see Chapter 14)-has thrived overthe years (Sloan & Mizes, 1999).

Experiments in behavior modifi-cation have produced some dramaticresults. One experiment assessed its

20 40 60 effectiveness with a 3-year-old girl inDays nursery school who was too shy and

withdrawn to take part in any of thegroup activities. Instead, she tried to hide by staying on the floor, either motion-less or crawling. How could she be led to get up, start moving around, and jointhe other children? The procedure tumed out to be simple. As long as she was onthe floor, her teachers ignored her. As soon as she got up on her feet, they show-ered her with attention. Given this reinforcement, she quickly became an activemember of the group-eventually, "natural" social reinforcers took over and theextra praise could be discontinued (Harris, Johnston, Kelley, & Wolf, 1965).

The same kind of behavior modification-ignoring undesirable actions andrewarding desirable ones-has been successful in many other situations. One spe-cial kind of behavior modification, in which the reinforcement is a form of paymentfor desirable behavior, is called a token economy. It has been used in mental in-stitutions (as well as many other settings), where it originated as an attempt toimprove the general well-being and daily lives of patients. For dressing properly,eating in an acceptable manner, and working at useful jobs, patients in mentalhealth institutions are rewarded with "tokens" that they can exchange for privilegessuch as treats, movies, rental of radios or TV sets, and greater opportunities forprivacy. The tokens are usually checkmarks on clipboards kept by attendants sothat the patients won't lose them or possibly have them stolen. These token econ-omies have produced some remarkable changes in behavior, such as those illus-trated in Figure 4.14.

Institutional token economies have some critics, however (e.g., see Glynn, 1990).In addition to the enormous investment of time and resources required, there is ahuman rights issue. Granting extras to patients who "go along with the program"means denying them to patients who don't or simply can't. There is also the ques-tion of whether token economies are truly therapeutic and beneficial for the pa-tients in the long run, after they leave the institution and no longer receive explicitrewards for appropriate behavior.

FIGURE 4.14 Behaviormodification revolutionizesa mental institution. Withthe token economy, patientsworked actively at useful jobsand helped run the institution(phase I). When the tokeneconomy Was temporarily dis-continued from day 20 to day40 (phase III, to attest toits effectiveness, the patientsquickly went back to their pas-sive and apathetic ways. Whenthe token economy was rein-stated (phase 1111, their workreturned to its previously highlevel.Adapted from Ayllon & Azrin,

1968.

Token economyA behavior modification proce-dure in which adaptive behav-ior is reinforced with tokensthat can later be exchanged forprivileges and other rewards.

Instinctive behaviorAs defined by ethologists, abehavior that occurs in all nor-mal members of a species, inresponse to specific releasingstimuli, and in essentially thesame way every time.

EthologyThe study of instinctive behav-iors in the lab and in naturalenvi ronments.

Nature versus Operant ConditioningSome of the creatures that inhabit our earth manage to go about their lives with-out learning a lot about consequences. Much of what they need to know to subsistand reproduce is already present in the prewiring of their nervous system-theyrequire only limited experience to deal with life. Guided primarily by instinctivebehaviors, they can find food, build shelters, mate, and survive as species. Ethol-ogy is the study of instinctive behaviors in animals. These behaviors have beenstudied extensively, both in the laboratory and in natural environments, by ethol-ogists such as Konrad Lorenz (1903-1989) (1952). In their scientific definition, tobe instinctive a behavior must meet the following criteria at a minimum:

• It must occur in all normal members of a species.

FOCUS QUESTIONS• Why is it important to consider cognition in studies

of conditioning?• Can learning occur when there is no reinforcement

or punishment?• What is the modern view of processes in learning

and conditioning?

• It must occur in response to demonstrable "releasing" stimuli.

• It must occur in essentially the same way every time.

Thus, there is some room for learning and modification of instinctive behav-ior, but not much. For example, not all birds of a species sing exactly the same song,but what they sing occurs in the same kinds of situations and they all recognizewhat it means-food, safety, sexual interest, danger.

Human beings are quite different. To be sure, there are certain basic behav-iors-sucking is one example-that appear at the very start of life without any ap-parent leaming. Such reflexes, however, are much simpler than instincts. In tum,evolutionary psychologists continue to expand the list of human predispositionsthat might have evolved because they were adaptive, such as certain preferencesexhibited by women and men with regard to physical characteristics and behav-iors of potential mates (e.g., see Buss, 1999a). These, though, are viewed as "whis-perings" in our biological makeup that can easily be overcome by culture andleaming (Barash, 1979). We certainly have no inborn wiring in our nervous sys-tem that can steer us through life on "automatic pilot." We have to learn almostfrom scratch how to find food, keep warm, and build shelters; without this leam-ing over an extended portion of our early lives, we don't survive. We also mustleam to form social relationships and institutions that enrich our lives by provid-ing through joint effort what no individual could manage alone. We may well bepredisposed to form relationships with others-we are social creatures-but theform this behavior takes is largely leamed.

However, in operant conditioning with nonhuman animals-especially non-primates-it is now recognized that instinctive and often powerful biological pre-dispositions must always be taken into account. As noted earlier in the chapter,rats have extreme difficulty leaming to press a bar to escape or avoid shock. Thishas nothing to do with rats' level of intelligence; instead, it's because their built-inresponses to pain include jumping and fleeing as fast as they can, behaviors thatare incompatible with staying in one place and pressing a bar.

Particularly compelling demonstrations of how biological predispositions canbe at odds with principles of conditioning were described by Keller and Marian Bre-land (1961, 1966), whose primary careers involved training animals to performentertaining and often complex stage acts. What they found was that some of thetrick behaviors could be leamed easily by their animals, but then deteriorated be-cause of instinctual drift back to predisposed behavior. For example, their raccoonscould be conditioned to insert tokens into a slot for a food reward, but as the tokensbecame more strongly associated with the food-through classical conditioningpairings in which the tokens became CSs-the raccoons became unable to insertthe tokens and instead reverted to holding on to them and "washing" them, as rac-coons normally do with food. Similar results were obtained with pigs and tokens-pigs instinctively "root" their food and soon began doing this with tokens as well.

COGNITIVE LEARNING: A CONTEMPORARYVIEW ROOTED IN THE PASTFor several decades, the supposed laws of classical and op-erant conditioning dominated psychology's view of leam-ing. The strict behaviorists thought of all human behavioras conditioned by events in the environment. The prevailingbelief was that there is little difference between humans going

(h) Your car's ignition switchis unreliable and unpredicta-ble. Your car starts sometimesthe first time you turn theswitch, sometimes after youturn it two or three times,and sometimes after as manyas nine or ten turns. One dayyour engine's starter goesbad, and you sit there turningthe switch again and againfor an hour before you finallygive up. What operant con-ditioning phenomenon ac-counts for your persistence?

(i) Having been badly stungby bees, your dog now freaksand runs whenever it seesor hears a bee. What operantconditioning contingency ismaintaining this behavior?

about their daily lives and a ratnegotiating its way through a aTmaze," in which it proceeds downa runway to an intersection whereit must turn left or right to obtaina reward. The rat's behavior couldbe predicted if we knew which turnit had previously been rewardedfor making and which turn it hadpreviously been punished for or re-ceived no reward for making. Theview was that we could also pre-dict people's behavior fairly well ifwe simply knew which of their ac-tions had been rewarded in the pastand which had been punished, andin what way and to what extent(Miller & Dollard, 1941; Skinner,1953). It was even thought that hu-man personality could be explained

m entirely in terms of "reinforcement@ (and punishment) history." Nor is

this approach dead-studies of hu-man operant conditioning have continued to be published at a high rate over theyears since (Dymond & Critchfield, 2002).

Even during the heyday of this view, however, there were dissenters. Some ex-perimental results, even with rats, did not mesh with the principles of condition-ing. Even in apparently simple stimulus-response learning, some researchers real-ized that it was essential to account for the perceptions and thoughts that takeplace between the stimulus and the response, as discussed in Chapter 1.

Thus, even in the early days of learning research, there was a concept that isnow widely accepted: Higher organisms are far from passive products of experi-ence and simple associations between stimuli and responses; they are always ac-tively interacting with their environment and thinking about and interpretingwhat they observe. Rather than associations, they acquire knowledge (Gallistel &Gibbon, 2002).

FIGURE 4.15 You can'tkeep a good chimp down.Faced with the problem ofreaching a banana suspendedhigh overhead, the chimpanzeeon the left balanced a longstick beneath it and quicklyclimbed up. In a different sit-uation, the chimpanzee on theright hit upon the "insight" ofpiling three boxes on top ofone another to reach a banana.Kohler, 1925.

InsightCognitive learning as a resultof problem solving.

The Case of the Ingenious ChimpsOne of the first influential experiments along cognitive lines was reported by theGerman psychologist Wolfgang Kohler as far back as the 1920s. Kohler workedwith chimpanzees, creating situations in which they had to demonstrate consider-able ingenuity to get at a banana placed tantalizingly just out of reach. Sometimesthe food was just a little farther than arm's length away from a chimp behind abarrier. Sometimes it was suspended overhead, too high to reach by jumping. Sonear and yet so far.

Could the chimpanzees learn to get the food? As it turned out, they managedto do so in a number of clever ways. The animals behind the barrier figured outthat they could use sticks to rake in the bananas. The animals who saw the foodoverhead hit upon several strategies, two of which are illustrated in Figure 4.15.To KoWer, this kind of learning went far beyond any stimulus-response connec-tions established by conditioning-the chimps had no such prior training. He heldthat the animals had learned through relatively sudden insight (Kohler, 1925), orwhat today would be called cognition in the form of problem solving (see Chap-

ter 6). That is, they evaluated thesituation, called on whatever pastknowledge they could muster, andprocessed all this information interms of cause and effect. It is im-portant to note that they did notsolve the banana problems throughthe trial-and-error approach pro-posed by Thorndike, and they hadno prior operant conditioning forthis kind of task; they solved themby thinking about them until theyhit upon a solution.

- Group 1, reinforced from first day- Group 2, never reinforced

~ •••• Group 3, reinforced after tenth dayo(,)(/)

ClC·c(ijQ)...J

I I I I4 5 6 7 8 9

Practice days

Learning in the Absence of ReinforcementAnother influential series of experiments, performed with rats in mazes (laid outlike the ones you may have worked with a pencil as a child), produced results thatcontradicted the learning theorists' principle that reinforcement is essential tolearning and cognition can be ignored. In one classic version (Tolman & Honzik,1930), the rats were divided into three groups. Group 1 always found a food re-ward at the end of the maze, and their performance gradually improved until theywere running through the maze as fast as they could and making almost no"wrong turns." Group 2 rats were simply placed in the maze and allowed to movearound and explore, with no reward throughout the experiment. They did so, ofcourse, because rats naturally explore any new environment they find themselvesin (and besides, they had nothing else to do). Group 3 was treated the same wayas Group 2 for the first 10 days, but for the remaining days they were switched tothe food reward that Group 1 had been receiving all along.

How the three groups performed, as measured by how direct a route they tookto the end of the maze, is illustrated in Figure 4.16. Note that the rats in Group 1learned gradually and progressively, improving each day right from the beginning.As was also predicted, the never-rewarded rats in Group 2 displayed little learn-ing. But note the behavior of Group 3. For the first 10 days this group also displayedlittle learning, but on the eleventh day the rats immediately began running the courselike veterans now that they were being rewarded. In just wandering around themaze without any reinforcement, they apparently had learned a great deal aboutit. As soon as a reward was provided, they began to demonstrate this knowledge.

The experiment showed that even rats can learn without the immediate rein-forcement that the strict behaviorists believed to be essential to conditioning-infact, without any reinforcement at all. In cognitive terms, the rats moved aroundthe maze and acquired knowledge about it, which they could call on when it wasuseful in helping them get to food as rapidly as possible.

Such experiments, along with many others, suggest that reinforcement is bet-ter viewed as an event that attracts and focuses attention (Rescorla & Holland,1982). Think, for example, of the sudden taste of food when an animal is hungryor an unexpected frown from a teacher when a student has given a wrong answer.Both are events that alert the organism, leading it to concentrate on the behaviorjust performed. The effectiveness of a reinforcing event thus is related to its at-tracting attention to the behavior just displayed-and providing feedback about it.

Yet another basic form of learning that requires no reinforcement and occursin humans and many other species is observationalleaming-or, as some psy-chologists prefer, learning through modeling or learning by imitation. Each termrefers to the process through which we learn by observing the behavior of others.

I I I I I I I10 11 12 13 14 15 16 17

FIGURE 4.16 A funnything happened on the waythrough the maze. The graphshows the behavior of threegroups of rats placed in a maze,two under different conditionsof reinforcement and oneunder no reinforcement. Notein particular the dotted line.Adapted from Tolman & Honzik,

1930.

Observational learningCognitive learning as a resultof watching others perform abehavior. It extends to learn-ing bV listening or reading.

It has been known for some time thatmany animals learn by observation andimitation. In one early experiment, a catwas placed in a Skinner box and condi-tioned to obtain food by pressing the barwhen a light went on. Another cat had beenwatching and was then placed in the box.This second cat began very quickly to pressthe bar when the light went on. Throughobservation, it had learned much fasterthan the first cat (John, Chesler, & Bart-

~ lett, 1968). Subsequent experiments have.§ demonstrated many kinds of observational~ learning by many animals, from mice to-g dolphins.~ With humans, some of the most dra-'"~ matic examples of observational learning,@ such as the classic "Bobo dolls" experi-

ment described in Chapter 1, were demon-strated by Albert Bandura. Cognitive theorists do not think of observational learn-ing as an automatic and unthinking imitation of what we have seen. Instead, theybelieve that we begin in early childhood and continue throughout our lives to ob-serve what goes on around us and to store the information that the observationprovides. We observe what other people seem to value, how they go about gettingwhat they value, their behavior in general, and the results of their behavior. At thesame time, we make judgments. We mayor may not decide to value what theyvalue. We may imitate their behavior, adopt some but not all of it, or reject it en-tirely. As Bandura stated, learning by observation is "actively judgmental and con-structive rather than a mechanical copying" (Bandura, 1974).

Much of the time, we also learn how to do things by observing others, bothdirectly and indirectly. If you can ride a bicycle, swim, propel a ball precisely witha bat or your foot, fish, or dance-you name it-you probably learned the skill withthe help of someone's showing you, as well as telling you, how. That is, in a gen-eral sense, observational learning includes listening and reading and mentally vi-sualizing in addition to watching. It includes the kind of learning you are doing atthis moment. In our information processing, we benefit not just from the knowl-edge stored in our own memories but also, through language, from all the knowl-edge possessed by our fellow human beings-in fact, from all the wisdom accu-mulated throughout history as recorded in our libraries and computers. Indeed,this handed-down knowledge, in all its forms and all its complexity, is the very basisof modem civilization and technology as we know it-for better or for worse.

Children learn from obser-vation of the adults aroundthem.

(j) Josh uses" bad" wordswhenever he gets angry atschool-just as his father doesat home. What kind of learn-ing is Josh demonstrating?

Cognitive Maps, Expectancies, and KnowledgeExperiments such as the ones described in this chapter-Kohler's chimpanzeesand their insight, the rats that learned without reinforcement, the children wholearned how to beat up a Bobo doll-led many psychologists to seek a new approachto understanding learning. Granted, humans as well as lower animals often estab-lish simple stimulus-response and behavior-consequences associations throughclassical and operant conditioning, with reinforcement a part of the process. Muchof life is governed by the consequences of what we do, and there is no denying it(try not paying taxes and see what happens). But in cases in which other kinds oflearning occur, in ways not accounted for by the principles of conditioning, justwhat is it that the organism learns-and how does the learning take place?

Life Span Perspective Learning and Age

Although we are bom with all theneurons we will ever have, the over-all mass of the infant's brain totalsonly about one-fourth that of theadult brain. Initially, the brain growsbigger because its neurons grow insize and-until about age 2-newglia cells are formed and grow. Thestory is a bit different with neuronsynapses. These average about 2,500per neuron at birth, peak at about15,000 each by age 2 to 3, and are"pruned" back throughout the re-mainder of childhood to just thosethat are actively being used-with theeffect that a child's brain has manymore synapses than an adult's braindoes (Gopnik, Meltzoff, & Kuhl,1999).Thus, the brain specializes andbecomes more efficient at processinginformation during childhood-andbeyond. Because of such changes,we should expect that the ease ofleaming new ideas and skills andachieving insights should also changedramatically over the life span.

During the first year of life, beforea child has acquired much in the wayof language, leaming consists pri-marily of acquiring new perceptualskills and new behaviors. If the new

skill is one for which the infant isbiologically prepared, leaming takesplace quickly. An example is visuallyguided reaching for objects. Infantsleam rapidly to reach for a rattlewhen its sounds are close by, and toignore it when the sounds are froma distance. However, if the task is notone for which the infant is innatelyprepared-for example, successfullygrasping and holding the reached-for object-progress is slower. Thus,ease of leaming is clearly a functionof the nature of the task and, in thiscase, muscular maturation as well.

After age 2 or 3, the child can uselanguage and mental symbols andcan relate new experiences to a verylarge and complex set of ideas. Fromage 3 to 7, there is an obvious im-provement in the speed of leamingnew skills and rules. If the task is toreach for the one object of three thatis different from the other two (forexample, a cup but not the two adja-cent glasses), an older child willieamthe new rule much more quickly thana 2-year-old.

Many researchers believe thatfrom adolescence to about age 30 thebrain is in optimal shape for leam-

ing new ideas and manipulating com-plex concepts. It is noteworthy, forexample, that many mathematiciansand physicists make their most im-portant discoveries in their 20s. Manyof the Nobel laureates in physicsmade their important discoveriesbefore they were 30 years old.

New skills and knowledge can beacquired throughout the life span,but the leaming of middle and olderadulthood is more likely to involvethe integration of past experiences,or what many would call wisdom.A 60-year-old cannot leam as quicklyas a younger person. For example,if the task is to leam a series of 100digits by leaming 10 new digits eachday, a 30-year-old can master the as-signment much more quickly thanan older person. But the older personhas accumulated a richer set of expe-riences, permitting greater insightsinto complex events (see Chapter 1l).Freud's most important ideas cameafter he had reached midlife, andGeorge Bemard Shaw wrote someof his best plays in his "later years."

An idea proposed by Tolman (1948), based on his studies of rats in mazes, isthat the rats learn cognitive maps, both in the lab and in their natural environ-ments. In other words, he concluded that although animals do learn responsesthat result in reward, they also acquire knowledge of the spatial features of theirenvironment-of where things are and what leads to what. Reinforcement doesnot produce the learning as much as it causes organisms to use what they haveleamed (this is the same learning-performance distinction that was demonstratedwith children in Bandura's experiments). If a reward is provided in one part of thecognitive map, they will go there. If there is punishment, they will avoid the spot.We do the same.

Subsequent experimenters, working with types of learning that did not posethe spatial problems found in mazes, enlarged Tolman's interpretation into theidea of acquiring expectancies, as pointed out earlier in the chapter. What is im-portant is the subject's anticipation, or belief, about what will take place-both inclassical and operant conditioning and in other forms of learning.

Today's cognitive psychologists have expanded these suggestions into the all-embracing term knowledge. What is learned, according to the cognitive school, isall sorts of knowledge-the "maps" of pathways acquired by Tolman's rats, the ex-pectancies about food acquired by Pavlov's dogs, and a human's knowledge thatMain Street is one block north of Broadway, an aspirin can relieve a headache,

Cognitive mapA mental representationof the spatial features ofan environment.

2 x 4 = 8, "Thanks" in English is "Gracias" in Spanish and "Merci" in French, andthe earth is round.

Cognitive psychologists regard "the human organism as an active seeker ofknowledge and processor of information," with learning being the step in whichwe acquire information that we then modify, manipulate, store, and use in vari-ous ways (Klatzky, 1980). We are not passive pawns of our environment. Learningis one element in a closely related series of processes-such as perception, mem-ory, and language and cognition-that are discussed in other chapters of the text,especially the next two.

Psychologists are also aware that learning abilities differ at various points inour lives, as discussed in the Life Span Perspective on page 167.

Chapter 4 Summary1. Conditioning emphasizes the establishment of as-

sociations between observable stimuli and responses,although underlying cognitive processes are nowallowed for as well.

2. Learning is a relatively permanent change in behav-ior potential resulting from experience.

Classical Conditioning3. Ivan Pavlov is the originator of what is now called

classical conditioning, a form of leaming based pri-marily on stimuli that cause reflexes, such as sali-vation in response to food.

4. In classical conditioning, the stimulus that natu-rally produces the reflex response is the uncondi-tioned stimulus (UCS), which is repeatedly pairedwith an initially neutral stimulus until the latter be-comes a conditioned stimulus (CS). What is leamedin classical conditioning is a CS-UCS association;the original reflex response is the unconditioned re-sponse (UCR), and the response produced by theCS is the conditioned response (CR).

Classical Conditioning Phenomenaand Applications5. Delayed conditioning is the most effective form of

classical conditioning. Less effective are simultane-ous conditioning and trace conditioning.

6. After conditioning, extinction of the CR occurs whenthe UCS is discontinued. Allowing time to pass andretuming the subject to the apparatus is typicallyaccompanied by spontaneous recovery.

7. The crucial factor in classical conditioning is theconsistency with which the CS predicts the occur-rence of the UCS.

8. Stimulus generalization occurs when a stimulussimilar to the original CS also produces a CR. Stim-ulus discrimination, its complement, occurs whendissimilar stimuli produce a lesser CR or none at all.

9. Biological predispositions are often apparent inclassical conditioning. Because of prewiring, someCS-UCSassociations can be established much moreeasily than others and some not at all.

10. Taste aversion experiments provide an example ofhow biological predispositions affect conditioning.The learning of taste aversions is easy for animalsthat naturally associate taste with food but difficultto impossible for animals that use other cues, suchas visual ones, for food.

11. In Watson and Rayner's "Little Albert" experiment,the loud sound was the UCS, the rat was the CS, anda fear response was the UCR and CR.

12. In the experiment on conditioned illness in rats, theinsulin was the UCS, the light and syringe were CSs,and the coma was the UCR and CR.

13. In the experiment on conditioning the immune sys-tem in rats, the drug was the UCS, the novel tastewas the CS, and production of antibodies was theUCRandCR.

14. In the experiment on conditioning sexual behav-ior in rats, normal female odors were the UCS, thelemon scent was the CS, and sexual arousal was theUCR and CR.

15. In the modem view, conditioning can be best ex-plained by the development of expectancies-thatis, what animals and humans learn is the expecta-tion that a particular CS will be followed by a UCS.

16. Classical conditioning by past events may accountfor many of the fears and preferences displayed byhuman adults-and also for physical symptoms suchas unexplained headaches or nausea and the in-tense desire to retum to drug use that is sometimesdisplayed by former addicts.

Operant Conditioning17. Operant behavior "operates" on the environment

in accord with contingencies. Operant conditioningis based on contingencies that are arranged in thelab or occur in real life.

18. The controlled environment of the Skinner box rev-olutionized the study of leaming and conditioning.

19. Parallels between classical and operant condition-ing occur in the areas of extinction and spontane-ous recovery, as well as stimulus generalization anddiscrimination.

20. Shaping and successive approximations is an effi-cient procedure for training subjects to performspecific behaviors.

21. An operant is a class of behaviors-not a specificbehavior.

22. The first half of Thomdike's law of effect corre-sponds to positive reinforcement and negative rein-forcement; the second half corresponds to positivepunishment and negative punishment.

23. In operant conditioning contingencies, positivemeans that a stimulus is presented or "added" andnegative means that a stimulus is removed or "sub-tracted." The effect on behavior is then determinedby whether the stimulus is appetitive or aversive.

Operant Conditioning Phenomenaand Applications24. Laboratory animals, young children, and others who

lack cognitive and language skills cannot mediatedelays in reinforcement or punishment, so delayed

contingencies tend to be ineffective. Older childrenand adults can mediate, and so delays do not voideffectiveness.

25. Learning is faster with continuous reinforcement,but partial reinforcement produces behavior that ismore resistant to extinction. Parents who partiallyreinforce tantrum behavior eventually find it verydifficult to eliminate.

26. Ratio schedules require a number of responses be-fore reinforcement occurs; inteIVal schedules requirethat an amount of time pass before reinforcementoccurs.

27. Fixed schedules require a specific number of re-sponses or amount of time; variable schedules re-quire a number of responses or amount of timethat varies around an average value.

28. Skinner strongly favored positive reinforcement overpunishment, because of the atmosphere it createsand especially because it tells the organism whatto do.

29. Punishment, in contrast to reinforcement, only tellsthe organism what not to do and also has a long listof undesirable side effects-especially if the pun-ishment is corporal.

30. Punishment may be necessary in the case of dan-gerous or harmful behavior, but it is best used inconjunction with positive reinforcement for accept-able behavior.

31. Superstitious behavior persists in the absence of anyactual contingencies.

32. Phobias, as well as some superstitions, are main-tained by escape and avoidance-that is, the reduc-tion of fear through negative reinforcement.

33. Learned helplessness has been demonstrated in dogsby subjecting them to inescapable electric shocks;their reaction has direct parallels to human feel-ings of helplessness and depression.

34. Learned helplessness can be counteracted by pro-viding success experiences.

Chapter 4 Test Yourself Answers(a) Classical conditioning with prior doctors who

meant well but inflicted pain(b) A conditioned stimulus that Juan has learned to

associate with pain(c) Extinction, the eventual disappearance of a

conditioned response(d) A biological predisposition to learn taste aver-

sions after a single instance of sickness, plusstimulus generalization to potato chips otherthan the ones that made him sick

35. Behavior modification refers to any program de-signed to alter behavior for the better by manipu-lating its consequences.

36. Token economies, which reward helpful and adap-tive behavior, can be highly effective, although theysometimes raise ethical issues.

37. Ethology defines instinctive behavior in a highlyspecific and scientific way. Humans have reflexesand perhaps evolutionary "whisperings," but noth-ing in the way of instinctive behaviors as definedby ethologists.

38. In laboratory and stage animals, biological predis-positions have been found to interfere with oper-ant conditioning procedures.

Cognitive learning: A ContemporarvView Rooted in t~e Past39. In the heyday of the view that human behavior and

personality could be explained in terms of rein-forcement history, there were dissenters who in-sisted that an understanding of cognition was alsoessential.

40. Learning by insight was one early demonstrationthat trial-and-error learning and stimulus-responseassociations were inadequate in explaining behavior.

41. Rats' learning of mazes in the absence of reinforce-ment was another difficultphenomenon for the strictbehaviorists to explain.

42. Demonstrations of observational learning by hu-mans and other species also contradicted the ideathat learning could be explained solely in terms ofreinforcement or punishment; thus, the learning-performance distinction emerged.

43. Modern civilization and its technology exist be-cause of direct and indirect observational learning.

44. In addition to being conditioned, we learn cogni-tive maps and expectancies-in general, knowledge.We are active seekers of learning and not passivepawns of our environment.

(e) Operant learning-probably accompanied byobservational learning, as discussed later in thechapter

(f) Shaping and successive approximations, accom-panied by positive reinforcement

(g) Negative punishment; being late results in theremoval of a goody, or appetitive stimulus.

(h) Partial reinforcement, which produces thegreatest resistance to extinction

(i) Negative reinforcement; running makes the bees"go away," thus maintaining your dog's phobia.

(j) Observational learning, which can crop up atthe most inappropriate times

Chapter 4 Conditioning and LearningKey Terms

behavior modification (p. 161)classical conditioning (p. 140)cognitive map (p. 167)conditioned response (CR) (p. 141)conditioned stimulus (CS) (p. 141)conditioning (p. 138)contingency (p. 149)continuous reinforcement (p. 155)delayed conditioning (p. 143)ethology (p. 162)expectancy (p. 148)extinction (p. 143)fixed schedule (p. 156)insight (p. 164)instinctive behavior (p. 162)interval schedule (p. 156)learned helplessness (p. 160)learning (p. 139)negative punishment (p. 153)negative reinforcement (p. 152)observational learning (p. 165)

operant conditioning (p. 149)partial reinforcement (p. 155)positive punishment (p. 153)positive reinforcement (p. 152)ratio schedule (p. 155)reflex (p. 139)shaping and successive approximations (p. 150)simultaneous conditioning (p. 143)spontaneous recovery (p. 143)stimulus discrimination (p. 145)stimulus generalization (p. 144)superstitious behavior (p. 159)token economy (p. 162)trace conditioning (p. 143)unconditioned response (UCR) (p. 141)unconditioned stimulus (UCS) (p. 141)variable schedule (p. 157)

cf The key terms above can be used as search terms inInfo Trac, a database of readings, which can be foundat www.cengage.com/infotrac.

Active Learning Review

1. The early learning theorists thought that all human behavior could be under-stood in terms of conditioning

2. When we take too large a sip of a hot beverage, the experience teaches us tosip slowly in the future-a lasting change in our behavior potential called

3. Learning is a relatively permanent change in behavior potential resulting from_____ . But learning can also be applied to behaviors that originally are experiencenot learned but are inborn.

Classical Conditioning4. A reflex is an inborn response to a specific . Whenever a bright light

strikes your eyes, it makes your pupils smaller. This type of inborn and auto-matic response is called a _

5. Although reflexes and instincts are inborn and not , Ivan Pavlov, aRussian scientist, studied a form of learning through which a reflex could bemade to occur in response to a stimulus that originally did not evoke it.

6. The process of learning to associate a reflex with a new stimulus-the kindof learning studied by the Russian scientist -is called classicalconditioning.

7. Pavlov dogs to respond to a sound with a salivary reflex, whichoriginally occurred only in response to food in the dog's mouth. This proce-dure is called conditioning.

reflexlearned

unconditioned stimulusUCSreflex

unconditioned stimulus,UCSconditioned stimulusCS

unconditionedconditionedclassical conditioningunconditioned stimulusUCS, unconditionedresponse, UCRconditioned stimulus, CSconditioned response, CR

conditionedunconditioneddelayed conditioning

delayed conditioningsimultaneous conditioning

extinctionUCS, CS

spontaneous recoveryconditioningclassical

8. In the procedure for associating a with a new stimulus, the stimu-lus that naturally sets off the reflex is called the unconditioned stimulus, or VCS.

9. In Pavlov's experiments, food was the natural stimulus for the reflex of saliva-tion. In classical conditioning terms, food was the , or

10. The unconditioned stimulus naturally produces a . In classical con-ditioning terms, this unleamed reflex to a stimulus is called the unconditionedresponse, or VCR.

11. In Pavlov's experiment, salivation was the , or _12. Classical conditioning involves establishing an association between a new,

initially neutral stimulus and an "or . Whenthe new stimulus elicits a response such as salivation, it is called a conditionedstimulus, or CS. The sound of a bell was often used as the .or , in Pavlov's experiments.

13. Each pairing of stimuli such as a bell and food is called a conditioning trial.Eventually, after several , the bell presented alone produced sali-vation-a behavior that is called the conditioned response because it is leamed.

14. When salivation occurs as an unleamed reflex to the food stimulus, it is calledthe response, or UCR. When it occurs as a leamed response to thebell stimulus, it is called the response, or CR.

15. In sum, in the form of leaming called , a previouslyneutral stimulus is associated with another stimulus, the _or .,which reflexively produces the , or_____ . When the previously neutral stimulus produces this response, it isnow the , or , and the response is called the__________ .,or _

Classical Conditioning Phenomena and Applications16. Simple associative learning such as is influenced by

several factors. One is the sequence in which the CS and UCS are presented.The most effective sequence is to present the stimulus followedquickly by the stimulus, a sequence called delayed conditioning.

17. Leaming occurs most rapidly under conditions of . Lesseffective are simultaneous conditioning, in which the stimuli are presented atexactly the same time, and trace conditioning, in which there is a time periodbetween the two stimuli.

18. If the CS precedes the UCS, the process is called ; if thestimuli are presented at the same time, it is called ; andif the CS is presented and then removed before the UCS is presented, it iscalled _

19. If the UCS is discontinued-that is, if the CS is no longer followed by the_____ stimulus-the learned weakens and eventually nolonger occurs.

20. In Pavlov's experiments, if the sound is no longer followed by food, the dogwill eventually cease to salivate to the sound. When this happens, the condi-tioned response has undergone _

21. In short, when Pavlov discontinued the , the underwentextinction. After a rest period away from the experimental apparatus, how-ever, a conditioned response may reappear.

22. After a period of rest, if an extinguished CR reappears following the next pre-sentation of the CS, the phenomenon is called _

23. The finding that delayed, simultaneous, and trace differ in effec-tiveness underscores that the crucial factor in conditioning is thereliability with which the CS predicts the occurrence of the UCS.

24. The subject or participant learns that the CS the DCS; thus, themost effective procedure is briefly _

25. Once a CR has been established to a specific stimulus, it may also occur in re-sponse to other stimuli similar to the original one, in a process called stimulusgeneralization. If a dog learns to salivate to the sound of one bell, the dog mayalso salivate to the sound of a different bell through the process of _

26. But if an experimenter continues to present the DCS after the original belland never after the different one, the CR will eventually occur mostly only inresponse to the original bell. This phenomenon is called stimulus _

27. are "prewired" tendencies that can either enhance orinterfere with conditioning. For example, rats readily associate taste and smellwith food, so it is easy to condition a rat to avoid foods or liquids with a dis-tinctive taste by making the rat sick afterward.

28. In general, conditioned taste aversions are learned easily and quickly by manyanimals because of , which enable conditioning tooccur in a single trial that can span several hours. Thus, a rat or a human canlearn a to a certain food in a way that "violates" theprinciples of classical conditioning.

29. In Watson and Rayner's experiment, Albert initially learned to fear a whiterat, but he also became apprehensive about a Santa Claus beard-evidence of__________ . If Albert had then been given experience with Santa'sbeard when it was not paired with the DCS, eventually his apprehension aboutthe beard, but not the rat, would have undergone and Albert wouldhave learned _

30. In real-life situations, it may be that we learn accidental associations through_____ conditioning and then become ill when we encounter the CS. Forexample, we may develop a headache or become sick for no medical reasonwhen a situation includes a previously established stimulus thatwe don't even remember.

31. Based on the principle that classical conditioning works best when the CSreliably the occurrence of the DCS, a cognitive explanation of clas-sical conditioning is that we acquire an expectancy about what will happennext when a given stimulus occurs. Thus, a former drug addict may begin ex-periencing craving and withdrawal effects when simply exposed to drug para-phernalia, because of conditioned _

Operant Conditioning32. In addition to reflex behavior, which is the subject of conditioning,

another form of behavior important in the study of learning is ______ -an activity that is voluntarily initiated by the organism and thatsomehow "operates" on the environment.

33. When a rat is placed in a new environment, it may explore, sniff, and climbabout. Exploring the environment is a form of behavior.

34. A is a relationship between operant behavior and its consequences,and conditioning is the process through which organisms learnabout contingencies-whether in the lab or in real-life settings. A rat learnswhat to do to obtain food and a human learns what to do to obtain a paycheck,through contingencies and operant conditioning.

35. The process by which an animal learns to increase or decrease frequency ofan operant behavior because that behavior is followed by a certain _is called _

36. When a rat in a Skinner box learns to press a bar to produce food, _conditioning has occurred. Through such conditioning, animals can be taught

predictsdelayed conditioning

stimulusgeneralization

discriminationBiological predispositions

extinctionstimulus discrimination

classicaloperantbehavior

operantcontingencyoperant

consequenceoperant conditioningoperant

shaping, successiveapproximationsreinforcementpunishment

rei nforcementpunishment

weakened, suppressedpositivereinforcementpositive punishment

strengthened, negativeweakened, suppressednegative

positive reinforcementnegative reinforcementpositive punishmentnegative punishment

operantconditioning

to perform novel or complex tasks by the process of shaping and successiveapproximations, which consists of rewarding them for the successful comple-tion of each step leading to the target behavior.

37. A rat in a Skinner box might be given a food pellet at first when it heads in thedirection of the bar, then when it touches the bar, then when it touches thebar and causes the bar to move, and finally only when it presses the bar suffi-ciently to trigger the food release mechanism. Such a process would be an ex-ample of and _

38. Translating Thorndike's law of effect into Skinner's terms, we might say that_____ occurs when a behavior leads to a satisfying state of affairs, andthe behavior tends to be strengthened; occurs when a behaviorleads to an unsatisfying or annoying state of affairs, and the behavior tends tobe weakened or suppressed.

39. When a rat presses a bar to obtain food, barpressing is strengthened and thecontingency is . When a rat instead receives an electric shock, bar-pressing is weakened or suppressed and the contingency is _

40. In operant conditioning terminology, positive means that something is pre-sented or "added" when the behavior occurs. Thus, when a rat presses a barand receives food, barpressing is and the contingency is positivereinforcement; when the rat instead receives an electric shock, barpressing is_____ or and the contingency is positive punishment.

41. If a child is praised for doing something good, the contingency is _____ . If a child is scolded for doing something bad, the contingency is

42. In operant conditioning terminology, negative means that something is re-moved or "subtracted" when the behavior occurs. Thus, when a rat presses abar and escapes or avoids shock, barpressing is and the contin-gency is negative reinforcement; when the rat instead has food postponed ortaken away, barpressing is or and the contingency isnegative punishment.

43. If a child is nagged until the child cleans up his or her room, the behavior is_____ and the contingency is reinforcement. If a child losesprivileges as a result of misbehavior, the behavior is or _and the contingency is punishment.

44. In sum, with regard to the consequences of behavior, presenting an appetitivestimulus, or goody, is ; removing an aversive, or unpleas-ant, stimulus is ; presenting an aversive, or unpleasant,stimulus is ; and removing an appetitive stimulus, orgoody, is _

Operant Conditioning Phenomena and Applications45. Delay of reinforcement or punishment greatly interferes with _

_____ of laboratory animals or very young children, but older childrenand normal adults can span very long delays through verbal mediation, orself-talk. Young children are not capable of mediation, whereasolder children and normal adults regularly use it.

46. Learning generally takes place more rapidly with continuous reinforcement, inwhich each instance of the behavior is reinforced. When reinforcement followseach instance of the desired behavior, reinforcement is . But learn-ing is more resistant to extinction if it is acquired with partial reinforcement,which is the delivery of reinforcement on some occasions but not on others.

47. In the wild, animals don't get rewarded for every instance of hunting for food;they only occasionally get rewarded, which is an example of _reinforcement.

48. If parents occasionally give in and, in effect, reward their children's tantrums,they are engaging in _

49. Behaviors are learned more rapidly when reinforcement is , butbehaviors are more resistant to when reinforcement is _There are two types of partial reinforcement schedules. With a ratio schedule,reinforcement is delivered only after the subject responds a certain number oftimes. With an interval schedule, reinforcement is delivered only for the firstresponse after a certain period of time.

50. If a rat receives food only after pressing a bar three times, reinforcement is ona schedule; if the rat receives food every two minutes, reinforce-ment is on an schedule. For both types of schedules, if the ratio orinterval is always the same, the schedule is fixed; if it changes over time, theschedule is variable.

51. Partial reinforcement given for a specific number of responses is on a _schedule; partial reinforcement given for the first response after so much timeis on an schedule. When the ratio or interval is constant, the sched-ule is called ;when it is changing, the schedule is called _

52. Skinner strongly favored using rewards for appropriate behavior combinedwith ignoring inappropriate behavior; thus, he favored . _combined with _

53. Skinner emphasized that reinforcement is preferable in part because it ismore informative; that is, reinforcement teaches what .whereas punishment only teaches what . _

54. In the operant conditioning explanation, superstitious behavior occurs in theabsence of any actual consequences, and phobic behavior occurs because ofprior bad experiences. Wearing a certain sweater to an exam or throwing saltover your shoulder after you spill it is behavior because it has noactual consequences; desperately avoiding certain objects or situations is_____ behavior learned through bad experiences.

55. In the case of phobias, the behavior in response to the feared object or situa-tion has the effect of making it go away and reducing the fear, and a person willbe likely to continue to engage in behavior that has these effects. Phobic be-havior is therefore maintained by _

56. Unavoidable .,especially if it is painful, can lead in both laboratoryanimals and humans to learned helplessness, or a tendency to believe that eventscannot be controlled and to give up trying to learn or act.

57. Frequent experiences with failure or unavoidable consequences can lead toa belief that events cannot be controlled and to giving up, a condition called

58. Principles of operant conditioning-especially positive reinforcement-havebeen applied in many situations to change behavior for the better. Behaviormodification is a general term for applying to behav-ioral change.

59. Deliberately "teaching" people to change their behavior is known as ______ , a special form of which is token economy. This method, used inmental institutions and many other settings, provides _for desirable and adaptive behavior in the form of tokens that patients can ex-change for privileges and other rewards.

60. When the awarding of tokens-typically checkmarks on a clipboard-for goodbehavior is applied on a large scale within an institution, the approach iscalled a '

61. is the study of instinctive behavior, which is defined as behavior thatoccurs in all normal members of a species, occurs in response to demonstra-ble "releasing" stimuli, and occurs in essentially the same way every time.

partial reinforcementcontinuousextinction, partial

ratiointerval

intervalfixed, variable

positive reinforcementextinction

to donot to do

negative reinforcementpunishment

behaviormodificationpositive reinforcement

token economyEthology

speciesstimuli, everytime

predispositionsinstinctsbiological

driftclassical conditioning,biological predispositions

cognitioninsight, cognitive mapsclassical, operant

behaviorobservational

62. To qualify as an instinctive behavior according to ethologists, the behaviormust occur in all normal members of a , must occur in response tospecific , and must occur in essentially the same way ______ . Humans have no behaviors that meet these criteria, although wemay have subtle biological predispositions that can be overcome by cultureand learning.

63. Biological may exist in humans, but these do not qualify as

64. In training animals to perform tricks such as circus acts, predis-positions can get in the way. For example, raccoons can be trained to inserttokens in a slot to obtain a food reward, but the behavior soon deteriorates be-cause of classical conditioning; the raccoons try to wash the tokens instead-as raccoons normally do. That is, the tokens become so closely associated withfood through that instinctual drift occurs and the rac-coons can no longer perform the trick.

65. Instinctual , which can interfere with operant conditioning, is a re-sult of the interplay between and _

Cognitive Learning: A Contemporarv View Rooted in the Past66. Historically, the strict behaviorists thought that essentially all human learning

was acquired through and conditioning. But not alllearned behavior can be explained by these principles, as is illustrated by theexample of insight, or what today would be called cognition in the form ofproblem solving.

67. In one early experiment, chimpanzees-faced with food placed beyond theirreach and given some sticks-"figured out" how to use the sticks to reach thefood. This form of cognition is called _

68. The chimps had never before used sticks to reach food, so this behavior didnot constitute behavior that was by food.

69. In another classic experiment, cognition and learning in the absence of_____ were demonstrated. First, a group of rats were simply allowed toexplore a maze, with no reward provided. Later, when the rats were rein-forced, they displayed knowledge of the maze comparable to that of rats thathad been reinforced all along for finding their way through the maze. The in-terpretation was that the first group of rats, while exploring, had acquiredcognitive maps of the maze.

70. In the absence of any reinforcement, both the chimpanzees and the rats dis-played thinking, or . For the chimpanzees, the cognition was_____ ; for the rats, the cognition was the learning of _

71. Another form of learning that cannot be explained by or _conditioning is observational learning. Humans and many other animals learnsimply by watching and perhaps imitating the they see.

72. In a general sense, learning includes reading and mentally visual-izing in addition to watching, and the knowledge handed down in this mannerover the generations is the basis for modern civilization.

8. Insight and cognitive maps are concepts mostclosely associated witha. strict behaviorism.b. Pavlov.c. cognitive psychology.d. Thorndike.

1. Which statement is not true with regard tolearning?a. Areflex response can become associated with

a stimulus that did not originally producethe response.

b. Learning can occur without observable be-havioral activity.

c. Learning differs from instinctive behaviorbecause of the role of experience.

d. Reflexes and instincts are inborn and are notmodifiable by learning.

9. A concept that is closely related to classical butnot operant conditioning isa. stimulus discrimination.b. spontaneous recovery.c. stimulus generalization.d. none of the above

2. In general, fears can be learned and maintained 10. Pavlov discovered that he could make dogs rest-

through less, destructive, or apathetic by

a. classical conditioning. a. reinforcing these behaviors.

b. stimulus generalization. b. giving the dogs an impossible discrimination

c. operant conditioning. problem.

d. all the above c. giving the dogs an impossible stimulus gen-eralization problem.

3. In Pavlov's experiments, the CR was d. all the abovea. the food.

Phobias are maintained byb. salivation to the bell. 11.

c. the bell. a. positive reinforcement.

d. salivation to the food. b. negative reinforcement.c. positive punishment.

4. In the "Little Albert" experiment, the DCS was d. negative punishment.a. fear of the sight of the rat.

12. Omission training and the time-out procedureb. the rat.c. the loud noise. are examples of

d. Santa's beard. a. positive reinforcement.b. negative reinforcement.

5. Atype of learning that is not initially dependent c. positive punishment.on a specific stimulus to produce a specific re- d. negative punishment.sponse is

13. Learning is likely to occur most rapidly and bea. operant conditioning.b. reflex learning. most persistent if

c. discrimination learning. a. partial reinforcement is used.

d. classical conditioning. b. constant reinforcement is used.c. partial reinforcement is used first, followed

6. Human beings prone to asthma attacks because by constant reinforcement.of an allergy to dust or pollen could suffer an d. constant reinforcement is used first, fol-attack when exposed to a harmless substance lowed by partial reinforcement.because of

The schedule of reinforcement that operates ina. operant conditioning. 14.

b. stimulus discrimination. gambling games such as playing slot machines

classical conditioning. is ac.variable ratio schedule.d. spontaneous recovery. a.

b. variable interval schedule.7. Which of the following illustrates the role of c. fixed ratio schedule.

biological predispositions? d. fixed interval schedule.a. the learning of taste aversionsb. the learning of superstitious behaviorc. stimulus generalizationd. all of the above

15. Older children and normal adults "bridge thegap" and overcome delay of reinforcementthrougha. cognitive maps.b. insight learning.c. verbal mediation.d. trial-and-error learning.

16. Aparent who deliberately ignores a child's tem-per tantrums is attempting to discourage tan-trums througha. negative reinforcement.b. extinction.c. active avoidance.d. learned helplessness.

17. The assumption that behavior is controlled toa considerable degree by its consequencesunderliesa. behavior modification.b. reinforcement contingencies.c. token economies.d. all the above

18. If a light is illuminated in a shuttle box just be-fore an electric shock is delivered, a dog learnsa. escape and then active avoidance.b. escape and then passive avoidance.c. active avoidance and then escape.d. passive avoidance and then escape.

19. Punishment is most effective when combinedwitha. positive reinforcement for appropriate

behavior.b. negative reinforcement for appropriate

behavior.c. extinction of inappropriate behavior.d. not necessarily any of the above

20. Learned helplessness in children may be causedbya. frequent and inconsistent punishment.b. frequent failure.c. continually being pressured to do better than

they are able to do.d. all the above

1. This exercise will require you to be a bit of anactor. It is designed to demonstrate the effects ofpositive reinforcement in a conversational envi-ronment. Read through the entire exercise beforestarting the procedure.

You will need a tape or digital recorder with atleast a 5-minute capacity. You will need a personwho is willing to talk about a subject of interestto her or him on two consecutive days while yourecord what the person says. You will also need astopwatch or other device to time the 5-minutespan.

Through casual conversation, determine anarea in which your participant has an interest. Itcan be almost anything: cars, music, food, poli-tics, books, television, classes 0), fitness, etc. Ex-plain that you will be asking your participant totalk about two areas of interest, one on each day.For example, if your participant is interested incars, then the first day's topic might be U.S. carsand the second day's topic Japanese cars. If thearea of interest is music, on each of the two daysyour participant might discuss a different bandor a different kind of music.

Confirm that your participant is willing to berecorded and is willing to talk for at least 5 fullminutes each day while you listen attentively. En-sure that the participant knows that you will notbe speaking during her or his monologues.

Naturally, do not tell your participant what thisexercise is actually about until the second sessionhas been completed.

After having established what your participantwill be talking about, start the tape recorder,record the start time, and begin. As you listen, dowhat you ordinarily would do while listening at-tentively, but try to minimize smiling and headnodding. If the participant seems to want you totalk, you may want to gesture to your closed lipsas a reminder of how this exercise works.

Having reminded the participant about today'stopic, record a second 5-minute session, withouterasing the first session. This time, however, be-have differently: Every time your participant usesthe personal pronoun I or you, emit a brief smilethat your participant can see. This will requiresome finesse, and you may benefit from a little

practice with someone else beforehand. Your smilemust look natural. You must not leer, stretch yourlips tightly over your teeth, blind your participantwith too broad a smile, or otherwise allow yoursmile to appear contrived. Natural is the key wordhere. Your participant must not realize that youare producing the smiles deliberately.

People are social animals, and smiles tend to actas appetitive stimuli. You will determine how ef-fective your social approval smiles were in re-inforcing the use of the personal pronouns by an-alyzing the two taped segments.

For each taped segment separately, tally thenumber of times the person said either ''I'' or"you." Now, compare the total for the first seg-ment (without smile reinforcement) with that forthe second segment (with smile reinforcement).Answer these questions:

a. What were the numbers for the two sessions?

b. What was the difference in frequency (num-ber) of those personal pronouns between thefirst and the second session?

c. What does this difference suggest? Were thesmiles actually appetitive stimuli or some-thing else?

d. How might knowledge of such tactics affectthe conduct of a salesperson? A teacher? Apol-itician? A sibling? A business client?

e. Some people have involuntary facial expres-sions (tics, smiles, grimaces, and the like).What are the implications of your findingsfor them?

2. Suppose you're a psychologist who helps peoplewith their problems through the use of behaviormodification. Think about the following situations:

a. You're a school psychologist, and a teacher re-ports to you that Sally is a very shy first-gradegirl who rarely plays with other children. Mostof the time she plays alone in the corner ortries to do special favors for the teacher. Whatwould you suggest the teacher do to help thischild become more social and less of a pestfor the teacher?

b. You're in charge of a ward in a hospital formentally retarded children. Many of theseyoungsters have not learned to dress them-selves, eat properly, wash themselves, brush

their teeth, or refrain from hitting other chil-dren. How would you design a token econ-omy to help these children? Describe not onlywhat you would do, using proper learning ter-minology, but why it should work.

c. You're a child psychologist, and some par-ents come to you because their 3-year-old sonthrows a temper tantrum whenever he doesnot get his way. He yells and screams, throwshimself on the floor, kicks, and pounds. Howwould you explain to the parents how suchbehavior might have developed in the firstplace, and what would you suggest be doneabout it?

Bear in mind, of course, that there may be somecircumstances you aren't aware of. And it is un-derstood that you probably have had no actualexperience with such cases. Moreover, althoughactual psychologists would apply some of theprinciples you have studied in this chapter, theymight treat some cases differently from others,depending on their individual characteristics.Thus, you cannot offer universally "correct" an-swers to these questions.

3. There is considerable debate about the ethics ofusing behavior modification in schools, in hospi-tals, and at home. Some people claim that thisconstitutes manipulating individuals against theirwill and that no one has the right to exert suchcontrol over others in a free society. In contrast,those who favor the use of behavioral principlesargue that children who disrupt classrooms andpatients in mental institutions are already co-erced and manipulated in one way or another,whether behavioral principles are used or not.They claim that behavior modification is singledout-and receives more blame than does sendinga child to the principal's office-simply becausebehavior modification is more likely to be effec-tive. Write a discussion presenting details and ex-amples of both these opposing arguments, andthen state your personal position and justify it.

-;'t For quizzing, activities, exercises, and web links, check out thebook-specific website at www.cengage.com/psychology.