psy105 neural networks 4/5 4. “traces in time” assignment note: you don't need to read the...
TRANSCRIPT
PSY105 Neural Networks 4/5
4. “Traces in time”
Assignment note: you don't need to read the full book to answer the first half of the question. You should be able to answer it based on chapter 1 and the lecture notes.
Lecture 1 recap
• We can describe patterns at one level of description that emerge due to rules followed at a lower level of description.
• Neural network modellers hope that we can understand behaviour by creating models of networks of artificial neurons.
Lecture 2 recap
• Simple model neurons– Transmit a signal of (or between) 0 and 1– Receive information from other neurons– Weight this information
• Can be used to perform any computation
Lecture 3 recap
• Classical conditioning is a simple form of learning which can be understood as an increase in the weight (‘associative strength’) between two stimuli (one of which is associated with an ‘unconditioned response’)
Nota Bene
• Our discussion of classical conditioning has involved– A behaviour: learning to associate a response with
a stimulus– A mechanism: neurons which transmit signals– These are related by…a rule or algorithm
Learning Rules
“When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased.”
Hebb, D.O. (1949), The organization of behavior, New York: Wiley
Operationalising the Hebb Rule
• Turn ….“When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased.”
• ….Into a simple equation which is a rule for changing weights according to inputs and outputs
A Hebb Rule
• Δ weight = activity A x activity B x learning rate constant
• In words: increase the weight in proportion to the activity of neuron A multiplied by the activity of neuron B
Stimulus On
Stimulus Off
Time
Implications of this rule
Stimulus 1
Stimulus 2
CS1
UCS
?
Implications of this rule
Stimulus 1
Stimulus 2
?
CS1
UCS
Implications of this rule
Stimulus 1
Stimulus 2
?
CS1
UCS
weight
Implications of this rule
Stimulus 1
Stimulus 2
?
CS1
UCS
Activity A
Activity B
weight
Implications of this rule
Stimulus 1
Stimulus 2
?
CS1
UCS
Activity A
Activity B
weight
= x x 0.1
The most successful model of Classical Conditioning is the Rescorla Wagnar model
Accounts for the effects of combinations of stimuli in learning S-S links
Based on the discrepancy between what is expected to happen and what happens
But…Deals with discrete trials…ie has no model of time
Rescorla RA, Wagner AR. A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In: Classical Conditioning II: Current Research and Theory (Eds Black AH, Prokasy WF) New York: Appleton Century Crofts, pp. 64-99, 1972
Robert Rescorla (2008) Rescorla-Wagner model. Scholarpedia, 3(3):2237.
The problem of continuous time
Stimulus 1
Stimulus 2
The problem of continuous time
Stimulus 1
Stimulus 2
The problem of continuous time
Stimulus 1
Stimulus 2
The problem of continuous time
Stimulus 1
Stimulus 2
Activity A
Activity B = 0
The problem of continuous time
Stimulus 1
Stimulus 2
Activity A
Activity B = 0
The problem of continuous time
Stimulus 1
Stimulus 2
Activity A = 0
Activity B
• We need to add something to our model to deal with a learning mechanism that is always “on”
Traces
Stimulus 1
Traces
Stimulus 1
Traces
Stimulus 1
Stimulus 2
Traces
Stimulus 1
Stimulus 2
Activity B
Activity A
Traces
Stimulus 1
Stimulus 2
Consequences of this implementation
• Size of CS stimulus• Duration of CS stimulus• Size of UCS stimulus• Duration of UCS stimulus• Separation in time of CS and UCS• The order in which the CS and UCS occur – (cf. Rescola-Wagner discrete time model)