behavior models

Instructor: David A. Townsend

Email:

Class Web Page:

RESCORLA-WAGNER THEORY: BACKGROUNDResearch by Rescorla and Kamin requires a

change in our conception of conditioning. Animals do not evaluate CS-US pairings in

isolation.Evaluation occurs against a background that

includes: unpaired presentations of CS and US, or presentations of other CSs, or general experimental context.

Conditioning is most likely to occur when evaluation of entire situation reveals that a CS is best available predictor of US.

RESCORLA-WAGNER THEORY: BACKGROUNDThis view of Pavlovian conditioning makes

conditioning process appear much more complex than it had seemed.

How do animals do it? By what means do animals keep track of CSs

and USs, estimate probabilities, compute probability differences, and make CRs to CS?

If such calculations do occur, then animals are likely to make them automatically.

We need an account of the mechanism by which such automatic calculation might occur.

RESCORLA-WAGNER THEORY: BACKGROUNDIn last 30 years, many different theories have

been advanced to explain rich details of Pavlovian conditioning.

Most influential account was that of Robert A. Rescorla and Allan R. Wagner in 1972.

All later theories have been responses to shortcomings of Rescorla-Wagner account.

So, we will focus on Rescorla-Wagner theory.

RESCORLA-WAGNER THEORY: PRELUDEKamin’s blocking effect set stage for Rescorla-

Wagner model.Blocking effect suggested to Kamin that USs

were only effective when they were SURPRISING or unpredicted by CSs.

However, USs were not effective when they were unsurprising or predicted by CSs.

Added CS was not associated with any change in US.

RESCORLA-WAGNER THEORY: OVERVIEWRescorla-Wagner theory explains complex

contingency analysis in terms of simple associations of the sort Pavlov envisioned.

It can account for most standard conditioning phenomena in Chapter 3 as well as many newer phenomena in Chapter 4.

Theory is also precise; specified in such clear detail that one can derive predictions about behavior in untested experimental situations.

Mathematically

RESCORLA-WAGNER THEORY: TUTORIALSIf you need help, then please consult:http://psy.uq.oz.au/~landcp/PY269/r-wmod

el/index.htmlhttp://www.biols.susx.ac.uk/home/Martin_

Yeomans/Learning/Lecture6.htmlhttp://www.psych.ualberta.ca/~msnyder/A

cademic/Psych_281/C5/Ch5page.html

http://psy.uq.oz.au/~landcp/PY269/r-wmodel/index.html

http://psy.uq.oz.au/~landcp/PY269/r-wmodel/index.html

http://www.biols.susx.ac.uk/home/Martin_Yeomans/Learning/Lecture6.html

http://www.biols.susx.ac.uk/home/Martin_Yeomans/Learning/Lecture6.html

http://www.psych.ualberta.ca/~msnyder/Academic/Psych_281/C5/Ch5page.html

http://www.psych.ualberta.ca/~msnyder/Academic/Psych_281/C5/Ch5page.html

RESCORLA-WAGNER THEORY: ACQUISITION

Standard conditioning curves are negatively accelerated.

Changes in conditioning strength are very substantial early in training.

But, as training proceeds, a leveling-off point, or asymptote, is approached.

Generally speaking, changes in strength of conditioning get smaller with each trial.

RESCORLA-WAGNER THEORY: ACQUISITIONNegatively accelerated learning curve suggests

that organism does not profit equally from each training trial.

How much one profits depends on how much one already knows: When one knows nothing, profits are substantial (US

surprise is high).When one knows a great deal, profits from further trials

are small (US surprise is low).

RESCORLA-WAGNER THEORY: ACQUISITIONRather than learning a fixed amount with each

trial, one learns a fixed proportion of difference between one’s present level of learning and maximum possible.

As difference gets smaller (as one learns more), amount of new learning produced by further trials gets smaller.

Rescorla-Wagner model simply builds in a mathematical expression that conforms to negatively accelerated learning function.

RESCORLA-WAGNER THEORY: ACQUISITIONHere is the equation:

∆Vn = K(— Vn-1)V, associative strength, is measure of learning.

It is a theoretical quantity.It is not equivalent to magnitude or probability of

any particular CR.But, it is assumed to be closely related to such

measures of conditioned responding.

RESCORLA-WAGNER THEORY: ACQUISITION∆Vn = K(— Vn-1)

K reflects salience of CS.K can vary between 0 and 1 (0 ≤ K ≤ 1).Bigger K, bigger change in V on any given trial.Thus, salient stimuli mean large Ks, which mean

large ∆Vs, which mean large changes in association from trial to trial.

Intensity

Sensory modality

Organism

Types of US’s employed ( belongingness)

RESCORLA-WAGNER THEORY: ACQUISITION∆Vn = K(— Vn-1) indicates that different USs support different

maximum levels of conditioning. Asymptote of conditioning will vary with US;

different asymptotes are reflected by different s. More intense US, higher asymptote of

conditioning, and higher . is always equal to or greater than 0 ( ≥ 0).

RESCORLA-WAGNER THEORY: ACQUISITION∆Vn = K(— Vn-1)Change in strength on Trial n (∆Vn) is

proportional to difference between and prior associative strength Vn-1.

Because V grows from trial to trial, quantity ( - Vn-1) gets smaller and smaller, so ∆Vn also gets smaller and smaller, generating a negatively accelerated learning curve.

Eventually, V will equal , so that ( - Vn-1) will be 0, and conditioning will be complete

(asymptote will be reached).

∆Vn = K(— Vn-1)∆ (change in) DeltaV, associative strength, is measure of learningK reflects salience of CS. ( Lambda) indicates that different USs support

different maximum levels of conditioning.Change in strength on Trial n (∆Vn) is

proportional to difference between and Vn-1 prior associative strength.

Acquisition Trials

∆Vn= light (CS)=0 k = .20 = associated

strength of shock = 100

First conditioning trial:

Light (CS) is paired with shock (US)

First Trial∆Vn=k( -Vn-1)∆Vn= .20(100-0)∆Vn = .20 units

∆Vn = K(— Vn-1)∆ (change in) DeltaV, associative strength, is measure of learningK reflects salience of CS and US. ( Lambda) indicates that different USs support different maximum levels of conditioning.Change in strength on Trial n (∆Vn) is proportional to difference between and Vn-1 prior associative strength.

Acquisition Trials

∆Vtotal= light (CS)=0

k = .20 = associated




Second Trial∆Vn=k( -Vn-1)∆Vn= .20(100-20)∆Vn = .16 units


Acquisition Trials






Third Trial∆Vn=k( -Vn-1)∆Vn= .20(100-36)∆Vn = .12.8 units


Acquisition Trials






Forth Trial∆Vn=k( -Vn-1)∆Vn= .20(100-48.8)∆Vn = .10.2 units


N= trial4, N-1= trial 3

Rescorla-Wagner Model

Calculations from the Rescorla-Wagner model show a mathematical relationship to the process of conditioning 0

10

20

30

40

50

60

trial0

trial2

trail4

Vtotal

Rescorla-Wagner Theory (1972)

Organisms only learn when events violate their expectations (like Kamin’s surprise hypothesis)

Expectations are built up when ‘significant’ events follow a stimulus complex

These expectations are only modified when consequent events disagree with the composite expectation

Surprise

First Conditioning Trial

Trial K ( - Vn-1 ) = ∆Vn

1 .5 * 100 - 0 = 50

0

50

0

20

40

60

80

100

0 1 2 3 4 5 6 7 8

Trials

Ass

oci

ati

ve S

trength

(V

)


Second Conditioning Trial


2 .5 * 100 -50 = 25

0

50

75

0

20

40

60

80

100

0 1 2 3 4 5 6 7 8

Trials

Ass

oci

ati

ve S

tren

gth

(V

)

Third Conditioning Trial


3 .5 * 100 -75 = 12.5

0

50

75

87.5

0

20

40

60

80

100

0 1 2 3 4 5 6 7 8

Trials

Ass

oci

ati

ve S

tren

gth

(V

)

4th Conditioning Trial


Trial c (Vmax - Vall) = ∆Vcs

4 .5 * 100 - 87.5 = 6.25

0

50

75

87.593.75

0

20

40

60

80

100

0 1 2 3 4 5 6 7 8

Trials

Ass

oci

ati

ve S

trength

(V

)

Vall

∆Vcs = c (Vmax – Vall)



5 .5 * 10 - 93.75 = 3.125

0

50

75

87.593.7596.88

0

20

40

60

80

100

0 1 2 3 4 5 6 7 8

Trials

Ass

oci

ati

ve S

trength

(V

)



6 .5 * 100 - 96.88 = 1.56

0

50

75

87.593.7596.8898.44

0

20

40

60

80

100

0 1 2 3 4 5 6 7 8

Trials

Ass

oci

ati

ve S

trength

(V

)



7 .5 * 100 - 98.44 = .78

0

50

75

87.593.7596.8898.4499.22

0

20

40

60

80

100

0 1 2 3 4 5 6 7 8

Trials

Ass

oci

ati

ve S

trength

(V

)




8 .5 * 1 - 99.22 = .39

0

50

75

87.593.7596.8898.4499.2299.61

0

20

40

60

80

100

0 1 2 3 4 5 6 7 8

Trials

Ass

oci

ati

ve S

trength

(V

)

1st Extinction Trial


1 .5 * 0 -99.61 = -49.8

Acquisition

0

20

40

60

80

100

0 1 2 3 4 5 6 7 8

Trials

Ass

oci

ati

ve S

trength

(V

)

Vall

Extinction

99.61

49.8

0

20

40

60

80

100

0 1 2 3 4 5 6

Trials

Ass

oci

ati

ve S

tren

gth

(V

)

2nd Extinction Trial


2 .5 * 0 -49.8 = -24.9

0

50

75

87.593.75 96.88 98.44 99.22 99.61

0

20

40

60

80

100

0 1 2 3 4 5 6 7 8

Trials

Asso

ciativ

e St

reng

th (V

)

Vall

Acquisition

0

20

40

60

80

100

0 1 2 3 4 5 6 7 8

Trials

Asso

ciativ

e St

reng

th (V

)

Extinction

99.61

49.8

24.9

0

20

40

60

80

100

0 1 2 3 4 5 6

Trials

Ass

oci

ati

ve S

tren

gth

(V

)

Extinction Trials


3 .5 * 0 - 12.45 = -12.46

4 .5 * 0 - 6.23 = -6.23

5 .5 * 0 - 3.11 = -3.11

6 .5 * 0 - 1.56 = -1.56

Less and Less surprising

Hypothetical Acquisition & Extinction Curves with K=.5 and = 100

Extinction

99.61

49.8

24.9

12.456.23 3.11 1.560

20

40

60

80

100

0 1 2 3 4 5 6

Trials

Associ

ati

ve S

trength

(V

)

Acquisition

0

20

40

60

80

100

0 1 2 3 4 5 6 7 8

Trials

Asso

ciativ

e St

reng

th (V

)

Acquisition & Extinction Curves with c=.5 vs. c=.2 ( = 100)

Acquisition

0

20

40

60

80

100

120

0 1 2 3 4 5 6 7 8

Trials

Ass

oci

ati

ve S

trength

(V

)

c=.5c=.2

Extinction

0

20

40

60

80

100

120

0 1 2 3 4 5 6

Trials

Ass

oci

ati

ve S

trength

(V

)

c=.5c=.2

RESCORLA-WAGNER THEORY: COMPETITIONKey feature of Rescorla-Wagner model is how it

explains conditioning with compound stimuli comprising two or more elements.

Associative strength of a compound stimulus is assumed to equal the sum of associative strengths of elements.

VAX = VA + VX

Here, A and X may have different saliences, K and M, respectively.

∆Vn = K(— Vn-1)∆ (change in) Delta V, associative strength, is measure of learning K reflects salience of CS. ( Lambda) indicates that different USs support different maximum levels of

conditioning. Change in strength on Trial n (∆Vn) is proportional to difference

between and Vn-1 prior associative strength.

A,X symbols used for multiple CS’sSalience with multiple CS’s: A=K,

X=M

RESCORLA-WAGNER THEORY: OVERSHADOWINGVAX = VA + VX

How can theory account for overshadowing?With equally salient stimuli, VX would attain

only .50 rather than 1.00 if X alone were trained--mutual overshadowing.

Increases in salience of A would further reduce VX from .50 toward .00.

If salience of A (K) is very high and salience of X (M) is very low, then overshadowing should be complete.

Eyeblink Conditioning: OVERSHADOWING

Training: Tone/light + ShockTone = Eyeblink CRLight = ?

No CR to light

Corneal Air PuffElicits Eyeblink Response

Corneal Air PuffGiven with Tone

Tone Given AloneElicits Eyeblink Response

Overshadowing:

Overshadowing Whenever there are multiple stimuli or a compound stimulus, then ∆Vn = Vcs1

(K) + Vcs2 (M)

Trial 1:∆Vnoise = .2 (100 – 0) = (.2)(100) = 20∆Vlight = .3 (100 – 0) = (.3)(100) = 30Total ∆Vn = ∆ (K)Vnoise + ∆Vlight = 0 +20 +30 =50

Trial 2:∆Vnoise = .2 (100 – 50) = (.2)(50) = 10∆Vlight = .3 (100 – 50) = (.3)(50) = 15Total ∆Vn = Vn-1 + ∆Vnoise + ∆Vlight = 50+10+15=75

∆Vn = K(— Vn-1)

Noise= 30

Light= 45

Overshadowing

Trial 1: DVA = .40(100 – 0) = 40

DVx = .10(100 – 0) = 10

Trial 2: DVA = .40(100 – 50) = 20

DVx = .10(100 – 50) = 5

T2:

A=60

X=15

RESCORLA-WAGNER THEORY: BLOCKINGVAX = VA + VX

How would theory account for blocking?With equally salient stimuli, VX would be only .50

rather than 1.00 if AX only were trained.Prior training with A would further reduce VX,

because VA would already be substantial before AX trials were introduced.

Extensive training with A should lead to complete blocking of X.

Blocking

Group Phase 1 Phase 2 Phase 3

ExperimentalGroup (blocking)

ControlGroup

A US

Nothing

AB US

AB US

Test B

Test B

Same # trialsContiguityContingency

RESCORLA-WAGNER THEORY: BLOCKING

Acquisition

0

20

40

60

80

100

0 1 2 3 4 5 6 7 8

Trials

Ass

oci

ati

ve S

trength

(V

)

A

X

?

The Rescorla-Wagner associative model of conditioning is based upon four assumptions that refer to the process by which the CS and UC gain

associative strength (1) a particular US can only support a specific

level of conditioning, (2) associative strength increases with each

reinforced trial, but depends upon prior conditioning,

(3) particular CSs and US can support different rates of conditioning and

(4) when two or more stimuli are paired with the UC, the stimuli compete for the associative strength available for conditioning.

RESCORLA-WAGNER THEORY: CONTINGENCYTheory can also explain animals’ ability to detect

different degrees of contingency between CS and US.

Recall that fear of a CS for shock is a direct function of contingency between CS and US.

When contingency between events is zero, no learning of fear to CS occurs.

But, does a rat really compute probabilities to form a judgment of contingency?

Not according to Rescorla-Wagner model.

RESCORLA-WAGNER THEORY: CONTINGENCYTo explain contingency

sensitivity, Rescorla-Wagner theory makes use of background or contextual stimuli as Pavlovian predictors.

(Context = A discrete CS)Such contextual stimuli

themselves can compete with CSs for association with USs.

Case of random presentations of CS and US provides a useful illustration.

TRAINING

CONTEXTUALTEST

CUED TEST

Cue-plus-contextual Fear Conditioning

A.

TRAINING CONTEXTUALTEST

Context Alone Conditioning

Context Discrimination

TRAINING

TEST

Context 1

Context 2

Context 1

B.

C.

Variations of Fear Conditioning

RESCORLA-WAGNER THEORY: CONTINGENCY Random training can be seen to represent blocking with two kinds of

trials: A (context)-US [relatively frequent] AX (context plus CS)-US [relatively infrequent]

As animal receives frequent A-US pairings, VA (and hence VAX) approaches asymptote.

As VAX approaches asymptote from frequent A-US pairings, VX can receive no further increments and little responding to X will be observed despite occasional AX-US pairings.

US

CS

unpaired

time0.5 s

RESCORLA-WAGNER THEORY: CONTINGENCYSo, blocking is basic to effect of random CS and

US presentations. Contextual cues are present whenever US

occurs in absence of CS; contextual cues thus acquire excitatory strength.

On trials when CS is paired with US by chance, contextual cues are present as well.

So, context replaces Stimulus A in blocking example and randomly presented CS replaces Stimulus X.

RESCORLA-WAGNER THEORY: INHIBITIONIn Chapter 4, we saw that conditioning can be

either excitatory or inhibitory. At first glance, it is not obvious that Rescorla-

Wagner theory can explain inhibition. Inhibition requires a V that is less than zero; but,

none of the variables in the equation can ever be less than zero.

How can V become negative when none of the terms contributing to V

can be negative?

Time to Leave:

Zero

X Axis

Y A

xis

0

1

behavior models

Education