instrumental conditioning: motivational mechanisms
TRANSCRIPT
![Page 1: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/1.jpg)
Instrumental Conditioning: Motivational Mechanisms
![Page 2: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/2.jpg)
Contingency-Shaped Behaviour
• Uses three-term contingency
• Reinforcement schedule (e.g., FR10) imposes contingency
• Seen in non-humans and humans
![Page 3: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/3.jpg)
Rule Governed Behaviour
• Particularly in humans
• Behaviour can be varied and unpredictable
• Invent rules or use (in)appropriate rules across conditions (e.g., language)
• Age-dependent, primary vs. secondary reinforcers, experience
![Page 4: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/4.jpg)
Role of Response in Operant Conditioning
• Thorndike– Performance of response necessary
• Tolman– Formation of expectation
• McNamara, Long & Wike (1956)– Maze– Running rats or riding rats (cart)– Association what is needed
![Page 5: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/5.jpg)
Role of the Reinforcer
• Is reinforcement necessary for operant conditioning?
• Tolman & Honzik (1930)
• Latent learning– Not necessary for learning– Necessary for performance
![Page 6: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/6.jpg)
Results
Day 11
Ave
rag
e E
rro
rs
Days
food
no food
no food until day 11
![Page 7: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/7.jpg)
Associative Structure in Instrumental Conditioning
• Basic forms of association– S = stimulus, R = response, O = outcome
• S-R• Thorndike, Law of Effect• Role of reinforcer: stamps in S-R
association• No R-O association acquired
![Page 8: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/8.jpg)
Hull and Spence
• Law of Effect, plus a classical conditioning process
• Stimulus evokes response via Thorndike’s S-R association
• Also, S-O association creates expectancy of reward
• Two-process approach– Classical and instrumental are different
![Page 9: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/9.jpg)
One-Process or Two-Processes?
• Are instrumental and classical the same (one process) or different (two processes)?
• Omission control procedure– US presentation depends on non-
occurrence of CR– No CR, then CS ---> US– CR, then CS ---> no US
![Page 10: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/10.jpg)
Omission Control
CS
US
CR
Trial with a CR
CS
US
CR
Trial without a CR
![Page 11: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/11.jpg)
Gormenzano & Coleman (1973)
• Eyeblink with rabbits
• US=shock, CS=tone
• Classical group: 5mA shock each trial, regardless of response
• Omission group: making eyeblink CR to CS prevents delivery of US
![Page 12: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/12.jpg)
• One-process prediction:– CR acquisition faster and stronger for
Omission group– Reinforcement for CR is shock avoidance– In Classical group CR will be present
because it somehow reduces shock aversiveness
• BUT…– CR acquisition slower in Omission group– Classical conditioning extinction (not all
CSs followed by US)
• Supports Two-process theory
![Page 13: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/13.jpg)
Classical in Instrumental
• Classical conditioning process provides motivation
• Stimulus substitution• S acquires properties of O
– rg = fractional anticipatory goal response
• Response leads to feedback– sg = sensory feedback
• rg-sg constitutes expectancy of reward
![Page 14: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/14.jpg)
Timecourse
S
R O
rg - sg
Through stimulus substitution S elicits rg-sg, giving motivational expectation of reward
![Page 15: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/15.jpg)
Prediction
• According to rg-sg CR should occur before operant response; but doesn’t always
• Dog lever pressing on FR33 ---> PRP
• Low lever presses early, then higher; but salivation only later
Lever pressing
salivation
Time from start of trialM
agni
tude
![Page 16: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/16.jpg)
Modern Two-Process Theory
• Classical conditioning in instrumental
• Neutral stimulus ---> elicits motivation
• Central Emotional State (CES)
• CES is a characteristic of the nervous system (“mood”)
• CES won’t produce only one response– Bit annoying re: prediction of effect
![Page 17: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/17.jpg)
Prediction
• Rate of operant response modified by presentation of CS
• CES develops to motivate operant response• CS from classical conditioning also elicits
CES• Therefore, giving CS during instrumental
conditioning should alter CES that motivates instrumental response
![Page 18: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/18.jpg)
“Explicit” Predictions
• Emotional states
USCS Appetitive Aversive
(e.g., food) (e.g., shock)CS+ Hope FearCS- Disappointment Relief
![Page 19: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/19.jpg)
• Behavioural predictions
Aversive US
Instrumental schedule CS+(fear) CS-(relief)
Positive reinforcement decrease increaseNegative reinforcement increase decrease
![Page 20: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/20.jpg)
R-O and S(R-O)
• Earlier interpretations had no response-reinforcement associations
• Intuitive explanation, though
• Perform response to get reinforcer
![Page 21: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/21.jpg)
Colwill & Rescorla (1986)
• R-O association• Devalue reinforcer
post-conditioning• Does operant
response decrease?• Bar push right or left
for different reinforcers– Food or sucrose
devalued reinforcer
normal reinforcer
Mea
n re
spon
ses/
min
.Blocks of Ext. Trials
Testing of Reinforcers
![Page 22: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/22.jpg)
Interpretation
• Can’t be S-R– No reinforcer in this model
• Can’t be S-O– Two responses, same stimuli (the bar), but
only one response affected
• Conclusion– Each response associated with its own
reinforcer– R-O association
![Page 23: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/23.jpg)
Hierarchical S-(R-O)
• R-O model lacks stimulus component
• Stimulus required to activate association
• Really, Skinner’s (1938) three term contingency
• Old idea; recent empirical testing
![Page 24: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/24.jpg)
Colwill & Delameter (1995)
• Rats trained on pairs of S+• Biconditional discrimination problem
– Two stimuli– Two responses– One reinforcer
• Match the correct response to the stimuli to be reinforced
• Training, reinforcer devaluation, testing
![Page 25: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/25.jpg)
• Training– Tone: lever --> food; chain --> nothing– Noise: chain --> food; lever --> nothing– Light: poke --> sucrose; handle --> nothing– Flash: handle --> sucrose; poke --> nothing
• Aversion conditioning• Testing: marked reduction in previously
reinforced response– Tone: lever press vs. chain– Noise: chain vs. lever– Light: poke vs. handle– Flash: handle vs. poke
![Page 26: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/26.jpg)
Analysis
• Can’t be S-O– Each stimulus associated with same reinforcer
• Can’t be R-O– Each response reinforced with same outcome
• Can’t be S-R– Due to devaluation of outcome
• Each S activates a corresponding R-O association
![Page 27: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/27.jpg)
Reinforcer Prediction, A Priori
• Simple definition– A stimulus that increases the future
probability of a behaviour
– Circular explanation
• Would be nice if we could predict beforehand
![Page 28: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/28.jpg)
Need Reduction Approach
• Primary reinforcers reduce biological needs
• Biological needs: e.g., food, water• Not biological needs: e.g., sex,
saccharin• Undetectable biological needs: e.g.,
trace elements, vitamins
![Page 29: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/29.jpg)
Drive Reduction
• Clark Hull• Homeostasis
– Drive systems
• Strong stimuli aversive• Reduction in stimulation is reinforcer
– Drive is reduced
• Problems– Objective measurement of stimulus intensity– Where stimulation doesn’t change or increases!
![Page 30: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/30.jpg)
Trans-situationality
• A stimulus that is a reinforcer in one situation will be a reinforcer in others
• Subsets of behaviour– Reinforcing behaviours– Reinforcable behaviours
• Often works with primary reinforcers
• Problems with other stimuli
![Page 31: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/31.jpg)
Primary and Incentive Motivation
• Where does motivation to respond come from?
• Primary: biological drive state
• Incentive: from reinforcer itself
![Page 32: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/32.jpg)
But… Consider:
• What if we treat a reinforcer not as a stimulus or an event, but as a behaviour in and of itself
• Fred Sheffield (1950s)• Consummatory-response theory
– E.g., not the food, but the eating of food that is the reinforcer
– E.g., saccharin has no nutritional value, can’t reduce drive, but is reinforcing due to its consumability
![Page 33: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/33.jpg)
Premack’s Principle
• Reinforcing responses occur more than the responses they reinforce
• H = high probability behaviour• L = low probability behaviour• If L ---> H, then H reinforces L• But, if H ---> L, H does not reinforce L• “Differential probability principle”• No fundamental distinction between
reinforcers and operant responses
![Page 34: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/34.jpg)
Premack (1965)• Two alternatives
– Eat candy, play pinball– Phase I: determine individual behaviour
probability (baseline)• Gr1: pinball (operant) to eat (reinforcer)• Gr2: eating candy (operant) to play pinball
(reinforcer)
– Phase II (testing)• T1: play pinball (operant) to eat (reinforcer)
– Only Gr1 kids increased operant
• T2: eat (operant) to play pinball (reinforcer)– Only Gr2 kids increased operant
![Page 35: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/35.jpg)
Premack in Brief
Any activity…
…could be a reinforcer
… if it is more likely to be “preferred” than the operant response.
![Page 36: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/36.jpg)
Response Deprivation Hypothesis
• Restriction to reinforcer response• Theory:
– Impose response deprivation– Now, low probability responses can reinforce high
probability responses
• Instrumental procedures withhold reinforcer until response made; in essence, deprived of access to reinforcer
• Reinforcer produced by operant contingency itself
![Page 37: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/37.jpg)
Behavioural Regulation
• Physiological homeostasis• Analogous process in behavioural
regulation• Preferred/optimal distribution of
activities• Stressors move organism away from
optimum behavioural state• Respond in ways to return to ideal state
![Page 38: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/38.jpg)
Behavioural Bliss Point
• Unconstrained condition: distribute activities in a way that is preferred
• Behavioural bliss point (BBP)• Relative frequency of all behaviours in
unconstrained condition• Across conditions
– BBP shifts
• Within condition– BBP stable across time
![Page 39: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/39.jpg)
Imposing a Contingency
• Puts pressure on BBP• Act to defend challenges to BBP• But requirements of contingency (may)
make achieving BBP impossible• Compromise required• Redistribute responses so as to get as
close to BBP as possible
![Page 40: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/40.jpg)
Minimum Deviation Model
• Behavioural regulation
• Due to imposed contingency:
• Redistribute behaviour
• Minimize deviation of responses from BBP– Get as close as you can
![Page 41: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/41.jpg)
Time running
Tim
e dr
inki
ng
10 20 30 40
40
30
20
10
restricted running
restricted drinking
![Page 42: Instrumental Conditioning: Motivational Mechanisms](https://reader031.vdocument.in/reader031/viewer/2022032806/56649f005503460f94c16b58/html5/thumbnails/42.jpg)
Strengths of BBP Theory
• Reinforcers: not special stimuli or responses
• No difference between operant and reinforcer
• Explains new allocation of behaviour
• Fits with findings on cognition for cost:benefit optimization