28.04.2013

1

Backward Chaining and Forward Backward Chaining and Forward Chaining Procedures Compared Chaining Procedures Compared

in Animal Modelsin Animal Models

Iver IversenUniversity of North Florida

Presentation at NAFO, 2013

� Basic Term: Three-Term-Contingency

� Stimulus

� Response

� Reinforcement

� Stimulus sets the occasion for the response to be reinforced

Stimulus Response

Reinforcer

Stimulus ControlStimulus Control

►►When a stimulus controls emission of the When a stimulus controls emission of the response in a reliable manner and the response in a reliable manner and the response is absent when the stimulus is response is absent when the stimulus is absent:absent:

�� We call it: Stimulus controlWe call it: Stimulus control

�� Each instance of the stimulus controls emission Each instance of the stimulus controls emission of the behaviorof the behavior

�� Absence of the stimulus controls absence of the Absence of the stimulus controls absence of the behaviorbehavior

28.04.2013

2

Discriminative StimulusDiscriminative Stimulus

SStimulus in the presence of which a response is timulus in the presence of which a response is reinforcedreinforced

The presence of the stimulus is called SThe presence of the stimulus is called S--deedee or Sor SDD ––controls the occurrence of the responsecontrols the occurrence of the response

The absence of the stimulus is called SThe absence of the stimulus is called S--delta or Sdelta or S ∆∆ ––controls absence of the responsecontrols absence of the response

Basic InformationBasic Information

►►Experiments with ratsExperiments with rats

►►Food deprivationFood deprivation

►►Lights as stimuliLights as stimuli

►►Lever presses or pole movements as Lever presses or pole movements as responsesresponses

►►Food pellets as reinforcementFood pellets as reinforcement

How latency to the discriminative stimulus (SD) changes over trials, for a typical Rat learning experiment

Response Rate Changes in S-Dee and S-Delta

Notice Response Rate on Y-axis (resp/min)

Schematic of the ContingenciesSchematic of the ContingenciesEarly in Discrimination Learning

Discrimination Learned

S∆ S∆SD

Moment to Moment Control: Moment to Moment Control: Display of Individual TrialsDisplay of Individual Trials

Iversen Data

28.04.2013

3

After Discrimination is learned, the response occurs in SD and not in S∆

The SD then becomes a reinforcer – called a conditioned reinforcer

The SD can then reinforce a novel response

Onset of SD becomes a reinforcing event

Demonstration of Conditioned Demonstration of Conditioned ReinforcementReinforcement

Response 1 � Stimulus � Response 2 � Reinforcer

Pull � Light � Press � Pellet

►►When an When an SS--deedee becomes a conditioned becomes a conditioned reinforcer it can strengthen new behavior reinforcer it can strengthen new behavior without the direct use of the primary without the direct use of the primary reinforcerreinforcer

►►SS--deedee controls response “A”controls response “A”

►►If response “B” turns on the If response “B” turns on the SS--deedeethen then response “B” is strengthened response “B” is strengthened by by the the onset of onset of SS--deedee

►►Response B occurs in SResponse B occurs in S--deltadelta

Chains of Stimuli and ResponsesChains of Stimuli and Responses SS--deedee and Sand S--delta are Differentdelta are Different

Lever is response “A” – occurs in SD

Pole is response “B” – occurs in S∆

Notice: Evidence that SD is a reinforcer is assessed in S∆

What are Chaining Procedures?What are Chaining Procedures?

►►Sequences of Discriminative Stimuli:Sequences of Discriminative Stimuli:

�� Stimuli (S) and Responses (R) followed by a Stimuli (S) and Responses (R) followed by a ReinforcerReinforcer

�� SaSa--Ra Ra �� SbSb--RbRb �� ScSc--RcRc �� ReinforcerReinforcer

�� Each stimulusEach stimulus--response pair is a unit of response pair is a unit of

stimulus control or a stimulus control or a linklink in a chain of in a chain of stimuli and responsesstimuli and responses

Two Types of ProceduresTwo Types of Procedures

►►Backward Chaining:Backward Chaining:

�� Each stimulus serves two roles: Each stimulus serves two roles:

�� 1) Discriminative stimulus1) Discriminative stimulus

�� 2) Conditioned reinforcer2) Conditioned reinforcer

►►Forward Chaining:Forward Chaining:

�� Each stimulus serves one role:Each stimulus serves one role:

�� 1) Discriminative Stimulus1) Discriminative Stimulus

�� 2) During training each stimulus is followed by 2) During training each stimulus is followed by primary reinforcementprimary reinforcement

28.04.2013

4

Step 1

Step 2

Step 3

Backward Backward ChainingChaining Forward ChainingForward Chaining

Step 1

Step 2

Step 3

Forward ChainingForward Chaining

►►Each stimulus must be acquired as a Each stimulus must be acquired as a discriminative stimulusdiscriminative stimulus

►►At each new link, the new response is At each new link, the new response is reinforced by the primary reinforcerreinforced by the primary reinforcer

►►The response previously reinforced by the The response previously reinforced by the primary reinforcer is primary reinforcer is maybe now maybe now reinforced reinforced by the new discriminative stimulusby the new discriminative stimulus

Forward ChainingForward Chaining Chaining Procedures ComparedChaining Procedures Compared

►► Backward Training has been a very successful Backward Training has been a very successful method to establish behavior chains in animalsmethod to establish behavior chains in animals

►►Research on Forward Training in animals has been Research on Forward Training in animals has been limitedlimited

►► Both types of training have been very successful Both types of training have been very successful with humanswith humans

►► Advantages of one over the other and chain Advantages of one over the other and chain reversals have not been subject for much reversals have not been subject for much investigation in neither animals nor humansinvestigation in neither animals nor humans

Purpose: Assess Stimulus ControlPurpose: Assess Stimulus Controlin Chainingin Chaining

►►Backward and Forward Chaining Backward and Forward Chaining were were comparedcompared

►►Each chain had 3 linksEach chain had 3 links

►►Rats were used as subjectsRats were used as subjects

►►DiscreteDiscrete--trial procedure with food trial procedure with food reinforcementreinforcement

►►Maximal stimulus duration = 60sMaximal stimulus duration = 60s

►►InterInter--trial Interval variable with 30 s trial Interval variable with 30 s averageaverage

Outline

Backward Chaining Forward Chaining

StepAa �$ 1 Aa �$

Bb � Aa �$ 2 Aa � Bb �$

Cc � Bb � Aa �$ 3 Aa � Bb � Cc �$

Reversal

Aa � Bb � Cc �$ 4 Cc � Bb � Aa �$

28.04.2013

5

ChainBack Light

Left Light

Left Lever

Right Light

Right Lever

Feeding Tray

Photo Sensor

� Magazine training (all 6 rats)

� Shape Left Lever (all 6 rats)

� Left Light – Left Lever Discrimination (all 6 rats)

� Formed Matched Pairs based on performance

� Chaining Procedure:

� Backward (3 rats)

� Two Links (Bb � Aa � $)

� Three Links (Cc � Bb � Aa � $)

� Reversal (Aa � Bb � Cc � $)

� Forward (3 rats)

� Two Links (Aa � Bb � $)

� Three Links (Aa � Bb � Cc � $)

� Reversal (Cc � Bb � Aa � $)

A a Left Light – Left Lever � $

B bRight Light – Right Lever �A a Left Light – Left Lever � $

C cBack Light – Chain Pull �B bRight Light – Right Lever �A a Left Light – Left Lever � $

A a Left Light – Left Lever � $

A a Left Light – Left Lever �B bRight Light – Right Lever � $

A a Left Light – Left Lever �B bRight Light – Right Lever �C c Back Light – Chain Pull � $

Step

1

Reversal

2

3

Backward Forward

Acquisition: Backward ChainAcquisition: Backward Chain

►►Backward Chaining went smoothly for the Backward Chaining went smoothly for the first two links involving lever pressingfirst two links involving lever pressing

►►When back light and chain pull was added:When back light and chain pull was added:

�� Performance Broke Down Performance Broke Down the first the first few sessionsfew sessions

�� Rat 13 and 17: Three sessions no chain pullRat 13 and 17: Three sessions no chain pull

�� Shaping Shaping of chain pull was necessary for two of chain pull was necessary for two sessionssessions

�� Rat 14: Three sessions Rat 14: Three sessions some some chain pulling (no chain pulling (no shaping required)shaping required)

Acquisition: Forward ChainAcquisition: Forward Chain

►►First two links went smoothly as for First two links went smoothly as for Backward chaining ratsBackward chaining rats

►►When the back light and chain was added, When the back light and chain was added, all all three rats pulled the chain within the first three rats pulled the chain within the first two sessions and quickly acquired the chain two sessions and quickly acquired the chain pull response and formed the chain rapidly pull response and formed the chain rapidly

►►Shaping was not necessaryShaping was not necessary

28.04.2013

6

How Event Records WorkHow Event Records Work

C = Back LightR= Right LightL = Left Lightch= Chain pullri= Right leverle= Left LeverTr = Feeder EntryRf= Reinforcement

Video

Backward Training: Before Reversal Backward Training: Before Reversal (after 10 sessions)(after 10 sessions)

Forward Training:Forward Training:Before Reversal (after 10 sessions)Before Reversal (after 10 sessions)

CBA$

ABC$

Comparison of Backward and Forward Chaining

Why is the pattern of latencies different for Backward and Forward Training Rats?Travel Distance to Chain Pull is greater than travel distance between Left and Right lever

28.04.2013

7

Reversal of the ChainReversal of the Chain

Does training history affect performance with a novel chain sequence?

Backward Training:Backward Training:Reversal Reversal –– 11stst Session, Rat 13Session, Rat 13

Event Record MagnifiedEvent Record MagnifiedSession Start of ReversalSession Start of Reversal

Backward Training:Backward Training:Reversal Reversal –– 11stst Session, Rat Session, Rat 1414

Backward Reversal Final Session, Rat 13

Before Reversal: After Reversal:

28.04.2013

8

Forward Reversal Forward Reversal –– 11stst Session, Rat 15Session, Rat 15

Forward Reversal Forward Reversal –– 11stst Session, Rat 16Session, Rat 16 Forward Reversal Forward Reversal –– 11stst Session, Rat 18Session, Rat 18

Forward Reversal Final SessionForward Reversal Final Session

Before Reversal: After Reversal:

28.04.2013

9

Before Reversal: After Reversal:

Back

ward

Tra

inin

gForw

ard

Tra

inin

g

� Faster acquisition with Forward Chaining

� In reversal of chain, Forward Training gave an advantage:

� Quicker adjustment to reversal

� Prompt reinforcer retrieval� Each stimulus had been followed by a reinforced response

in training

� Stimulus-response units functioned as

independent units

Overall Results: Forward

� In reversal of chain: Backward trained rats had to reacquire stimulus control – reinforcer delivery didn’t control reinforcer retrieval for the first session

� Last response before the reinforcer continued to after pellet delivery (its prior reinforcer was onset of another stimulus not pellet delivery), so that response showed mini-extinction effects

� Established stimulus control broke down for two rats on the first session of reversal

Overall Results: Backward

►►Acquisition Acquisition led to similar chain performance led to similar chain performance prior to prior to reversalreversal

►►At least for rats, Forward Training seems to At least for rats, Forward Training seems to have an advantage over Backward Training have an advantage over Backward Training in adapting to new situations (chain in adapting to new situations (chain reversal)reversal)

►►Reversal Reversal showed that Forward Chaining showed that Forward Chaining generates a more “flexible chaingenerates a more “flexible chain” than ” than Backward TrainingBackward Training

►►BiBi--directional chains:directional chains:

�� Teaching dressing and undressingTeaching dressing and undressing

�� Teaching games (i.e., Monopoly)Teaching games (i.e., Monopoly)

�� Teaching packing and unpacking a bagTeaching packing and unpacking a bag

�� Teaching directional skills Teaching directional skills

�� Teaching musical scalesTeaching musical scales

Experiment 2