Learning

• What is Learning? – Relatively permanent change in behavior that

results from experience (behaviorist tradition)– Can there be learning that does not result in a

change in behavior?

• Types of Learning– Associative Learning (simple, passive, external)– Cognitive Learning (complex, strategic, internal)

Associative Learning

• Classical Conditioning – associating two stimuli

• Operant Conditioning – associating a behavior and its consequences

Classical Conditioning

• Pavlov’s serendipitous discovery

• Associating 2 stimuli– The first stimulus is “neutral” – does not

produce any response– The second stimulus produces a reflex

(unconditioned) response

• After the 2 stimuli become associated, both will produce the unconditioned response

Pavlovian Classical Conditioning

Before Conditioning

During Conditioning

After Conditioning

UCS UCR

Neutral Stimulus No Response

CS CR

CS UCRUCS

Pavlovian Classical Conditioning

Before Conditioning

During Conditioning

After Conditioning

Food (UCS) Salivation (UCR)

Tone (NS) No Salivation

Tone (CS) Salivation (CR)

Tone (CS) Salivation (UCR)Food (UCS)

Classical Conditioning to Cure Bed-Wetting

Before Conditioning

During Conditioning

After Conditioning

Alarm (UCS) Wake up (UCR)

Full Bladder (NS) No waking up

Full Bladder (CS) Wake up (CR)

Full B. (CS) Wake up (UCR)Alarm (UCS)

Further Concepts that Apply to Classical Conditioning

• Generalization: CR is given to stimuli that are similar to the CS

• Discrimination: CR not given to stimuli that are dissimilar to the CS

• Extinction: If the CS is presented repeatedly without being followed by the UCS, the CR will diminish or cease

• Spontaneous Recovery: Following extinction, the CR will spontaneously re-appear after a delay

Classical Conditioning as Simple Associative Learning

• Temporal Contiguity was thought to be sufficient – the CS simply needs to occur immediately prior to the UCS for conditioning to take place

• Equipotentiality: any two stimuli could be associated through conditioning

Equipotentiality Falsified

• Some stimuli are easier to associate than others

• Taste Aversion – only foods become associated with illness, not other stimuli– Garcia & Koelling, 1966 – the “Sweet, bright,

noisy water study”

Garcia & Koelling, 1966

• CS = flavor, light, and click (sweet, bright, noisy water)

• UCS: 2 conditions– Group 1: UCS = illness (from X-rays)– Group 2: UCS = shock

• CR = avoidance (not drinking the water)

• After conditioning, tested which features of the CS were associated with each UCS

Garcia & Koelling: Results

• Both Groups: CS (sweet, bright, noisy) CR (avoidance)

• Group 1(UCS = shock)– Sweet water No avoidance– Bright noisy water Avoidance

• Group 2 (UCS = illness)– Sweet water Avoidance– Bright noisy water No avoidance

Temporal Contiguity is Not Enough

• Contingency: The CS must reliably predict the occurrence of the UCS (Rescorla, 1966)

• Informativeness: The CS must provide new information for predicting the occurrence of the UCS

Contingency (Rescorla, 1966)

• UCS = shock (S), UCR = fear• CS = tone (T)• Training: two conditions

– Random Condition: S TS S T TS S T TS– Contingent Condition: TS TS TS

• Results: Rats learned to fear the tone only in the contingent condition, when the tone predicted the shock

Informativeness: Blocking

• If an organism has already learned that one CS predicts the UCS, that will block the conditioning of a new CS if the new CS does not provide any additional information

• Example: Fear conditioning of a tone blocks conditioning of a light

Blocking

Training 1 Training 2 Test

-none- Tone & Light, shock

(CR = fear)

Light Fear

Tone, shock

(CR = fear)

Tone & Light, shock

(CR = fear)

Light No Fear

Rescorla-Wagner Model (1972)

• A mathematical model of the “strength of association” produced in classical conditioning

• Can account for all of the classical conditioning phenomena we have just seen

• Uses just one single equation!

Rescorla-Wagner Model

ΔVn = c (Vmax – Vn)V = the strength of association between a CS and a

US

ΔVn = the change in the strength of association between the CS and US on a given trial

Vmax = the asymptote for CS-US association strength after learning

c = rate of conditioning (how fast the association is learned)

Rescorla-Wagner Model

00.20.40.60.81

1.2

0 1 2 3 4 5 6 7 8 9 10

Conditioning Trials

Ass

oci

ati

ve S

tre

ng

th

be

twe

en

CS

an

d U

S

Cognitive Interpretation of Classical Conditioning

• Classical Conditioning is more than simple association

• The concept of information could explain contingency and blocking

• They are not just associating stimuli, they are seeking information from one stimulus to predict the occurrence of the other

Operant Conditioning

• The law of effect: behaviors that are followed by good things happen more often

• Association: Things that occur together become associated

Basics of Operant Conditioning

• Operant – freely emitted behavior operating on the organism’s environment; NOT a reflex response

• Reinforcement Contingencies – the consequences that follow a behavior– Reinforcement: increases the frequency of the

behavior– Punishment: decreases frequency of behavior

Reinforcement & Punishment

• Positive reinforcement

• Negative reinforcement

• Positive punishment

• Negative punishment

Reinforcement Schedules

• Continuous vs. Partial• Fixed vs. Variable• Interval vs. Ratio• Examples

– Fixed ratio: vending machine– Variable ratio: slot machine– Fixed interval: checking mailbox– Variable interval: checking email

Explaining Complex Learning with Operant Conditioning

• Secondary reinforcers - association

• Shaping – simple learning in small increments

• Chaining – small increments plus secondary reinforcement

• Language – association and reinforcement (Skinner’s Verbal Behavior, 1957)

Learning that Could not be Explained by Behaviorism

• Latent Learning – learning without reinforcement (Tolman & Honzig, 1930)

• Observational Learning – learning without behaving or being reinforced (Bandura, 1977)

• Overjustification – when rewards decrease the frequency of behavior (but see Eisenberger & Cameron, 1996 for an opposing view)

• Language Acquisition – Chomsky’s critique

Latent LearningTolman & Honzig, 1930

 Group 1: never a food rewardGroup 2: always a food rewardGroup 3: food reward after 10 days

 

0

1

2

3

4

5

6

7

8

9

10

Day 1 Day 4 Day 8 Day 12 Day 16

Group 1

Group 2

Group 3

Behaviorism Falls Short: Language

• Chomsky: “Action in the past” as a property of stimuli is sneaking mental representations in the back door

• Association is insufficient to explain language learning: The evidence points to learning RULES

• Evidence: Over-regularization (“goed”)• Conclusion: Mere associations between words

can not explain language; any adequate theory of meaning must hypothesize internal representations of the rules of language (grammar)

So What was Behaviorism Lacking?

• Symbolic Representation – we have internal (mental) representations for things in the external world

• Structure – we learn sets of rules for combining symbols (e. g., grammar), not just associations between pairs of symbols

Associative Learning Rises Again?

• LSA – Latent Semantic Analysis– A theory of meaning, and a method for computer

analysis of the meanings of texts

– The meaning of a word = all of the words that co-occur with it in a sample of written text (roughly)

– Meaning is just a function of associations of words, not structure (syntax)

– How much of language meaning can LSA account for? A surprisingly large amount.