from neurons to brains to neural network models 1 + + + - -- - - - + + + i1i1 i2i2 i3i3 32 x1x1 x2x2...
TRANSCRIPT
FROM NEURONS TO BRAINS TO NEURAL NETWORK MODELS
1
+ + +- --- --
+ + +
I1 I2 I3
3
2
x1 x2 x3
(c) CELEST 20072
THE UNIFYING THEME OF CELEST CURRICULUM: METACOGNITION
LEARNING ABOUT LEARNING
THINKING ABOUT THINKING
A focus on neuroscience is a novel and compelling approach to learning because it explicitly focuses on human perception and learning
Teaches students various study strategies while instructing students in a variety of critical math and science skills
(c) CELEST 20073
FROM NEURONS…
Anatomy, morphology, physiology, specialization…
Neurons and the Synapse
How Neurons transmit an action potential and how the synapse
works
(c) CELEST 20075
The Beginning
Any thought, experience, or action that you do can be known as a stimulus. Those stimuli generate nerve impulses.
For example, when you see something light is reflected off a surface and enters your eye. Then it stimulates the retina’s photoreceptors which begins stimulating the nerves to create an electric impulse. That goes to the neurons.
http
://reu.u
wo
sh
.ed
u/va
ug
ha
n.p
hp
REFLECTED LIGHT
Photoreceptors on the retina
(c) CELEST 20076
ANATOMY OF A NEURONNerve impulse travels along Nerve cells otherwise known as neurons. These neurons have many parts which are involved in the transmission of the cell.
Here are the parts to a neuron that are present in the transmission of this impulse
Dendrites- act to conduct the electrical stimulation received from other neural cells to the cell body
Cell Body- also known as the “soma” can vary in size depending upon the type of neuron. It also contains the nucleus
Nucleus- Is responsible for producing most of the RNA in the Neurons and most proteins used by neurons are created by mRNA, which can create structures such as ion channels.
(c) CELEST 20077
ANATOMY OF A NEURON 2
Axon- Conducts the electric impulse from the cell body to the axon terminals
Myelin sheath- Is an insulating material which prevents the electric impulse from leaking allowing the impulse to travel rapidly. Some types of neurons don’t have this.
Schwann Cell- also aids in the insulation allowing the electric impulse to travel rapidly down the axon
Nodes of Ranvier- is the place where the electric impulse jumps to in each cell to carry the impulse down the axon acting like an electric amplifier.
Axon Terminal- Is the end of the axon which is part of the chemical synapse.
(c) CELEST 20078
Types of NeuronsThere are many types of Neurons here are four examples
of 4 more common ones.
Bipolar neurons are usually part of sensory path such as smell, sight, taste, hearing and vestibular functions. Unipolar neurons are also sensory neurons.Multipolar neurons are the majority of the brains neuronsPyramidal neuron are found in the hippocampus and cerebral cortex.
(c) CELEST 20079
ANATOMY OF AN ACTION POTENTIAL
The electrical impulse if large enough becomes known as an action potentials which is used to communicate with other neurons
An action potential occurs when an electrical charge travels down the axon from the cell body to the axon terminals through the Nodes of Ranvier
Axon
Axon Terminals
DendritesCell #1
Cell #2
(c) CELEST 200710
Action Potentials The impulse causes sodium channels to
open which allows sodium ions to start flowing into the neuron changing the charge gradient causing cell depolarization ( which means the potential difference is rising).
The deplolarization eventually reaches a threshold for starting
Sodium ion flow
the action potential, which means that the neuron will fire and more sodium channels open. The sodium ions continue to flow in and the depolariztion continues. As it reaches its maximum potential the sodium channels begin closing,
(c) CELEST 200711
Action PotentialsPotassium ion flow And the potassium channels
begin opening, which allows potassium ions to flow into the cell.
This flow begins repolarization and starts returning the potential to the rest potential.
However, channels stay open too long and the cell becomes hyperpolarized. At cell cannot fire until the cells restpotential is restored. This restoration is when the sodium potassium pump along with the outflow of potassium restores the ion concentrations to the beginning and the cell is ready to fire again
(c) CELEST 200712
Sodium/Potassium PumpThe pump acts to restore the
original Sodium ion and Potassium ion concentration because now The concentration of the Sodium ions inside the cell is to high and the Potassium ion concentration is to low. So, the pump works by pumping 3 sodium ions in while 2 potassium ions are pumped out.
The pump works by having ATP and 3 sodium ions bind to the pump. Then the ATP is hydrolyzed, which releases ADP that causes a comformational change in the pump and the sodium ions are exposed outside of the cell.
(c) CELEST 200713
Sodium/Potassium Pump Two potassium ions then bind
with the pump
ATP binds to the pump again causing it to reorient and the potassium ions are released into the cell.
The process then starts all over again. So the whole process continues to cycle until the rest potential is restored.
(c) CELEST 200714
Action Potential 2
The graph shows the action potential process in terms of the electric impuse. It is important to know that the cell can depolarize in the positive or negative direction.
Getting a visual of this is very important, so check out these animations
(c) CELEST 200715
HOW NEURONS COMMUNICATEWhen the electric impulse
reaches the axon terminals the electrical signal is converted to a chemical signal
These chemical signal are called neurotransmitters, which can be either excitatory or inhibitory
Neurotransmitters are released from the axon terminal through the synapse to the dendrite terminals of one or many other cells
Axon Terminal
Synapse
Neurotransmitter
Dendrite Terminal
(c) CELEST 200716
Types of Neurotransmitters The many different types of neurotransmitters are
contained within the vesicles. Each vesicle contains a specific type of neurotransmitter. On the next slide is a sample list of some of the more common neurotransmitters and their functions. Some of them can be excitatory which means that when they hit the receptors it causes a depolarization on the post synaptic neuron. (causing the peak on the graph to go up)
The other neurotransmitters are inhibitory which means they hyperpolarize postsynaptic neuron. (causing the peak on the graph to go down)
(c) CELEST 200717
Neurotranmitters 2Acetylcholine - voluntary movement of the muscles mostly excitatory Norepinephine- wakefulness or arousal, excitatory Dopamine - voluntary movement and motivation,
"wanting" , excitatory or inhibitory Serotonin - memory, emotions, wakefulness, sleep and
temperature regulation, excitatoryGABA (gamma aminobutyric acid) - inhibition of motor
neurons, inhibitorGlycine- spinal reflexes and motor behavior,mostly inhibitory
(c) CELEST 200718
SYNAPSE
Most synapses are unidirectional: one neuron sends a neurotransmitter to the other at that synapse across the synaptic cleft, but not the other way
around. The neuron who sends the neurotransmitter is called the presynaptic neuron.The neuron who receives the chemical messenger is called the postsynaptic neuron.
More on Synapse
Synaptic cleft
(c) CELEST 200719
Neurotransmitters When the action potential reaches the synapse the
depolarization causes calcium ion channels to allow calcium ions in which allows the vesicles to fuse with the membrane, and the vesicles to release the neurotransmitters from the presynaptic neuron axon terminal. The transmitter then can bind with the postsynaptic dendrite arm receptors. This binding then can begin the whole transmission process. The receptors will then either release the neurotransmitter to be recycled in a process called uptake or it will be broken down by enzymes. Lets watch several clips.
(c) CELEST 200720
Synapse Clip 1
(c) CELEST 200721
Synapse Clip 2
(c) CELEST 200722
Synapse Clip 3
(c) CELEST 200723
EXCITING A POST-SYNAPTIC NEURON
The level of excitation the postsynaptic neuron can receive is a function of how many synaptic connections a neuron’s dendrite has, as well as how many receptor sites there are per synapse
(c) CELEST 200724
SIGNAL PROPAGATION
The whole circuit can be broken down into a number of neurons and synapses.
Each neuron is in a certain state of activation.
This state of activation can be transferred to another cell via synapses.
synapse
(c) CELEST 200725
TO BRAINS…
Anatomy, morphology, physiology, specialization…
THE BASIC PARTS OF THE BRAIN
(c) CELEST 200727
MAP OF THE CORTEXES
(c) CELEST 200728
INTERNAL STRUCTURES OF THE BRAIN
(c) CELEST 200729
YOUR “3-BRAINS IN ONE”
The Triune Brain
(c) CELEST 200730
“BRAIN 3”
(c) CELEST 200731
(c) CELEST 200732
“BRAIN 2”
• THE “OLD MAMMALIAN” BRAIN”
(c) CELEST 200733
(c) CELEST 200734
“BRAIN 1”
REFERENCES
BOOKS, WEBSITES
(c) CELEST 200736
References• Diamond, Marian. MAGIC TREES OF THE MIND.• Jensen, Eric. TEACHING WITH THE BRAIN IN MIND• TEACHING WITH THE ARTS IN MIND • LeDoux, Joseph. THE SYNAPTIC SELF.• Ratey, John. A USER’S GUIDE TO THE BRAIN.• Wolfe, Patricia MIND MATTERS • Wolfe, Patricia BUILDING THE READING BRAIN• Brain website #1• Brain website #2• Brain website #3• Brain website#4• Brain website #5• Brain website #6
(c) CELEST 200737
TO NEURAL NETWORK MODELS…
Goal: represent, explain and predict reality (neuron, neural mass, electronic properties, chemical reactions, brain function, animal and human behavior)
Methods: directed graph, mathematical equation
Analysis technique: equilibrium analysis, simulations with systematic parameter variation
Key concepts: constant and phased input, bottom up activation, top-down priming, matching…
(c) CELEST 200738
A mature science of learning requires that we understand how
BRAIN MECHANISMS
give rise to BEHAVIORAL FUNCTIONS
MODELING HOW THE BRAIN LEARNS
(c) CELEST 200739
Mind-Body Problem
Many groups study BRAIN OR BEHAVIOR
BRAIN provides MECHANISMS BEHAVIOR provides FUNCTIONS
Without a link between them
BRAIN MECHANISMS have no FUNCTIONBEHAVIORAL FUNCTIONS have no MECHANISM
WHY IS IT IMPORTANT TOLINK BRAIN TO BEHAVIOR?
CELEST provides this link!
(c) CELEST 200740
What level of brain organizationcontrols behavior?
What is the functional unit of behavior?
BRAIN evolution needs to achieveBEHAVIORAL success
What level of BRAIN processing governs BEHAVIORAL success?
HOW DOES THE BRAIN CONTROL BEHAVIOR?
40 years of modeling show:The NETWORK and SYSTEM levels!
(c) CELEST 200741
BEHAVIOR IS AN EMERGENT PROPERTY OF NEURAL NETWORKS
Need to simultaneously describe 3 levels (at least): BEHAVIOR
NETWORK NEURON
and a MODELING language to link them
How are individual NEURONS designed and connected so that the NETWORKS they comprise generate emergent properties that govern successful BEHAVIORS?
Does this mean that individual neurons are unimportant?Not at all!
CELEST studies all of these levels simultaneously
(c) CELEST 200742
HOW MODELS LINK BRAIN TO BEHAVIOR
A successful MODELING APPROACH has unified these levels during 40 years of modeling led by CELEST scientists. In this approach, you analyse:
REAL-TIME AUTONOMOUS LEARNING SYSTEMS!
This theme makes possible a MODELING CYCLE that can link brain to behavior
How an individual adapts on its own in real time to a complex and changing environment
(c) CELEST 200743
Design Principles
Mathematicaland Computer
Analysis
Technological Applications
MODELING CYCLE
BehavioralData
Predictions
MIND
Neural Data
Predictions
BRAIN
(c) CELEST 200744
TWO KEY CONCLUSIONS
1. Advanced brains look like they do to enable
Lesson: The Architecture is the Algorithm
Lesson: You cannot fully understand adult neural information processing without studying how the brain LEARNS
2. Recent models show how the brain’s ability to DEVELOP and LEARN greatly constrain the laws of
REAL-TIME AUTONOMOUS LEARNING
ADULT INFORMATION PROCESSING
(c) CELEST 200745
BEHAVIORAL AND BRAIN MODELING
of normal and abnormal LEARNING during
Perception
Cognition
Emotion
Action
Discovers MECHANISMS that control learning
MODELS SERVE AS CELEST UNIFYING THEMES
(c) CELEST 200746
THINKING OUTSIDE THE BOX
Models are not only tied to data
How can we begin to know how the brain works? Think about it!
Thought experiments are often used to consider what must be true for particular situations to exist
(c) CELEST 200747
EXAMPLE 1: STABILITY-PLASTICITY DILEMMA
(c) CELEST 200748
EXAMPLE 2: MASS ACTION
(c) CELEST 200749
EXAMPLE 3: THE GATED DIPOLE
(c) CELEST 200750
DESIGN OF LEARNING ENVIRONMENTS:HOW PEOPLE LEARN
Learner-centered Focus on student’s previous knowledge and misconceptions
Knowledge-centered Structured towards progressive formalization of knowledge
Promote deep understanding and subsequent transfer
Assessment-centered Constant and interactive feedback
Community-centered Universally relevant and applicable topic, easily applied to everyday experience and problem solving
(c) CELEST 200751
LEARNER-CENTERED
CELEST curriculum focuses on the learner creating a deep understanding about how their brain works.
Example: BrightnessLab corrects misconceptions about how vision works
(c) CELEST 200752
KNOWLEDGE-CENTERED
CELEST curriculum is knowledge centered because it provides for progressive formalization using a system of models that range from those similar to everyday experience to increasingly abstract conceptual designs and mathematical formalizations and analysis.
(c) CELEST 200753
KNOWLEDGE-CENTERED MODELING
Petrosino, 2003
EXAMPLE: BrightnessLab Models
(c) CELEST 200754
ASSESSMENT-CENTERED
CELEST curriculum promotes interaction between students and their peers, students and their teacher, and students and the computer
Student activities are designed to provide formative feedback
Summative feedback activities are designed to test students’ content-knowledge and provide an arena to help students develop strategies to expand and transfer their knowledge to solve a wider variety of problems
(c) CELEST 200755
COMMUNITY-CENTERED
CELEST curriculum is universally relevant and applicable to all people and readily transferred to everyday experience
Everybody has a brain!
(c) CELEST 200756
WHAT WEB-BASED CURRICULUM EXISTS?
http://cns.bu.edu/CELEST/http://cns.bu.edu/CELEST/http://cns.bu.edu/CELEST/privatehttp://cns.bu.edu/CELEST/private
BrightnessLab: Seeing is Believing / Brightness Contrast
Sequence Learning: Make Your Memory Stronger!
Associative Learning: Learning in the blink of an eye
Obstacle Avoidance Navigation: Watch Where You’re Going!
Recognition: How do we know what we know?
(c) CELEST 200757
WHY WE BEGIN WITH THESE MODULES
Perception: the basis for knowledge about the world. Half of the brain is dedicated to visual processing. BrightnessLab begins the systematic study of visual processing
Action: reflex and planned movements. Given a goal, how do we decide to move as a reaction to sensory (visual) input? Obstacle Avoidance Navigation explores the question of reactive movement as opposed to memory guided movement
Cognition: how we know that we know. We begin the exploration of memory and learning by studying Sequence Learning of numerical lists and continue through an examination of Recognition and metacognition
Emotion: spontaneous physical and mental states. Since perception, action & cognition are all mediated by emotions, we begin to address them by studying adaptive timing, a basic form of Associative Learning
(c) CELEST 200758
COMMON PRINCIPLES
Laminar or layered organization
Parallel and interaction processing streams
Activation (excitatory and inhibitory) has a limit
Activation will naturally (passively) decay
(c) CELEST 200759
BREAKTHROUGHS IN BRAIN COMPUTINGModels that link detailed brain CIRCUITS to the ADAPTIVE BEHAVIORS that they control
INDEPENDENT MODULES Computer Metaphor
COMPLEMENTARY COMPUTING Brain as part of the physical world
Describe NEW PARADIGMS for brain computing
LAMINAR COMPUTINGWhy are all neocortical circuits laminar?
How do laminar circuits give rise to biological intelligence?
Mind/Body Problem
(c) CELEST 200760
Principles ofUNCERTAINTY and COMPLEMENTARITY
Multiple Parallel Processing Streams Exist
UNCERTAINTY PRINCIPLES operate at individual levelsHierarchical interactions resolve uncertainty
Each stream computes COMPLEMENTARY propertiesParallel interactions overcome complementary weaknesses
HIERARCHICAL INTRASTREAM INTERACTIONS
PARALLEL INTERSTREAM INTERACTIONS
ADAPTIVE BEHAVIOR = EMERGENT PROPERTIES
(c) CELEST 200761
VISUAL BOUNDARY AND SURFACE COMPUTATIONS ARE COMPLEMENTARY
orientedinwardinsensitive to direction-of-contrast
unorientedoutwardsensitive to direction-of-contrast
BOUNDARYCOMPLETION
SURFACE FILLING-IN
Neon color spreading
(c) CELEST 200762
Object plans and working memory
Spatial plans and working memory
Spatially invariant object recognition and attention
Spatialattention and tracking
3-D filling-in of binocular surfaces and figure-ground perception
Predictive target tracking and background suppression
Optic flow navigation and image stabilization
Depth-selective capture and filling-in of monocular surfaces
3-D boundarycompletion and separation of occludingand occluded boundaries
Enhancement ofmotion directionand featuretracking signals
Monoculardouble-opponentprocessing
Stereopsis Motion detection
Photodetection and discount illuminant
IT PPC
V4 MST
V2V2
MT
V1
Boundary-surface consistency
Formotionbinding
Retina and LGN
PFC PFC
WHAT STREAM WHERE STREAM
Object plans and working memory
Spatial plans and working memory
Spatially invariant object recognition and attention
Spatialattention and tracking
3-D filling-in of binocular surfaces and figure-ground perception
Predictive target tracking and background suppression
Optic flow navigation and image stabilization
Depth-selective capture and filling-in of monocular surfaces
3-D boundarycompletion and separation of occludingand occluded boundaries
Enhancement ofmotion directionand featuretracking signals
Monoculardouble-opponentprocessing
Stereopsis Motion detection
Photodetection and discount illuminant
IT PPC
V4 MST
V2V2
MT
V1
Boundary-surface consistency
Formotionbinding
Retina and LGN
PFC PFC
WHAT STREAM WHERE STREAM
CELEST PROJECTS TO DEVELOP UNIFIED MODEL OF HOW VISUAL CORTEX SEES
BOTTOM-UPTOP-DOWNHORIZONTALinteractions everywhere toovercomeCOMPLEMENTARYWEAKNESSES
Not independentmodules
(c) CELEST 200763
BIOLOGICAL TAKE HOME LESSONS
1. Need to model PAIRS OF COMPLEMENTARY CORTICAL STREAMS
to computeCOMPLETE INFORMATION
about a changing world
2. Need INTERACTING TEAMS OF SCIENTISTS
A CENTER! to characterize the large
FUNCTIONAL BRAIN SYSTEMS that control adaptive behavior
(c) CELEST 200764
COMPLEMENTARY STREAMS COOPERATE TO COMPUTE COMPLETE INFORMATION
Perception-cognition-emotion-action systems use several types of
MULTI-DIMENSIONAL LEARNED INFORMATION FUSION
Multiple sources of partial information are combined during learning
Complementary types of learning work together to solve environmental problemse.g., What-Where learned information fusion
CELEST thrusts are designed to model how this works
(c) CELEST 200765
CELEST MODELS COMPLETE BRAIN SYSTEMS
Perception-cognition-emotion-action systems use several types of
Multiple sources of partial information are combined during learning
Complementary types of learning work together to solve environmental problems
e.g., What-Where learned information fusion
Not a future wish; a present coordinated research program
MULTI-DIMENSIONAL LEARNED INFORMATION FUSION
(c) CELEST 200766
WHY THESE PARTICULAR THRUSTS?ORDINARY BEHAVIORS USE LARGE FUNCTIONAL
BRAIN SYSTEMS
Child’s task: Visually find and pick up a stationary cup of milk to drink
Spatially orient to the cup Where stream 3See cup What stream 1Recognize cup What stream 1 Want to pick cup up What stream 3Plan to pick cup up What-Where stream 3,5Pick cup up What-Where stream 1,3,5
THRUST
This perception-cognition-emotion-action cycleuses What-Where learned information fusion
Need visual, temporal, parietal, prefrontal cortices...
(c) CELEST 200767
WHY THESE PARTICULAR THRUSTS?ORDINARY BEHAVIORS USE LARGE FUNCTIONAL
BRAIN SYSTEMS
Child’s task: Orient to mother’s voice and say: “Mommy, give me milk”
Hear mother’s voice What stream 2Recognize voice What stream 2Spatially orient to voice Where stream 3Want to talk to mother What stream 3Plan to talk to mother What-Where stream 3,5 Talk to mother What-Where stream 2,3,5
THRUST
This perception-cognition-emotion-action cyclealso uses What-Where learned information fusion
Need auditory, temporal, parietal, prefrontal cortices...
(c) CELEST 200768
CELEST THRUSTS ENABLE MODELING OF COMPLETE BRAIN SYSTEMS
Perception-cognition-emotion-action systems enable the brain to learn adaptive behaviors in real time within a changing world
Just as important for
developing new engineering systems that intelligently process huge amounts of data in unpredictably changing environments
providing insights into how to improve
learning in the classroom
(c) CELEST 200769
WHY IS THIS POSSIBLE NOW?Recent models and modeling PARADIGMSdeveloped by CELEST scientists:
COMPLEMENTARY COMPUTINGand
LAMINAR COMPUTINGhave begun to clarify how these large functional brain systemscompute the sort of
complete informationthat controls successful
adaptive behaviors
CELEST brings together personnel and resources needed to take the next steps
(c) CELEST 200770
A NEURON-INSPIRED MODEL
xi zij xj
vi eij vj
Source: http://webspace.ship.edu/cgboer/neuron.gif© Copyright 2003 C. George Boeree
xi Short-term memory traces
vi Cell populations
eij Axons
zij Long-term memory traces
xj Short-term memory traces
for the next neuron
vj Cell populations
Source: S. Grossberg (1988). Nonlinear neural networks: Principles, mechanisms, and architectures. Neural Networks, 1, 17-61.
Key:
(c) CELEST 200771
GRAPHING CONVENTIONS
Modulators Learned weights
Excitation
Inhibition
--
++
(c) CELEST 200772
TYPES OF CONNECTIONS
Convergent Divergent
“In-star” “Out-star”
(c) CELEST 200773
TYPES OF CONNECTIONS
Feedforward Feedback
VARIETIES OF LEARNING MUST BE MODELED
RecognitionReinforcement TimingSpatialMotor Control
IdentifyEvaluateSynchronizeLocateAct
WhatWhy WhenWhereHow
(c) CELEST 200775
A model that clarifies how animals learn to attend to external events that predict satisfaction of internal drives in real time
Autonomous Adaptive Mobile Robots
e.g., MAVIN RobotWaxman et al., MIT Lincoln Lab
CogEM MODEL
(c) CELEST 200776
+
SCS1
SCS2
CS1 CS2
D
SENSORY
MOTOR
DRIVE
Competitionfor STM
ConditionedReinforcerLearning Incentive
MotivationalLearning
Internal Drive Input
MotorLearning
+ +
CogEM MODEL:3 Types of Representations and Learning
Grossberg, 1971+
(c) CELEST 200777
DRIVE REPRESENTATIONS
Sites where reinforcement and homeostatic inputs interact to generate emotional and motivational output signals
Emotion nodesBower et al., 1981
Adaptive Critic ElementsBarto, Sutton, and Anderson, 1983
Facilitator Neuron (Aplysia)Walters and Byrne, 1983Hawkins, Abrams, Carew, and Kandel, 1983
(c) CELEST 200778
NEURAL DRIVE REPRESENTATIONS
Facilitator Neuron (Aplysia)
Walters and Byrne, 1983
Hawkins, Abrams, Carew, and Kandel, 1983
Amygdala
Aggleston et al., 1995
LeDoux et al., 1988
(c) CELEST 200779
INTERPRETATION OF CogEM ANATOMY
SENSORYCORTEX
PREFRONTALCORTEX
AMYGDALA
DRIVE
(c) CELEST 200780
AMYGDALA AND NEARBY AREAS
(c) CELEST 200781
Adapted from Barbas (1995)
Visual Cortex
AuditoryCortex
GustatoryCortex
OlfactoryCortex
Lateral PrefrontalCortex
OrbitalPrefrontalCortex
AMYGDALA
SomatosensoryCortex
(c) CELEST 200782
Buonomano, Baxter, & Byrne, Neural Networks, 1990
Grossberg, Behavioral and Brain Sciences, 1983
FACILITATOR NEURON ~ DRIVE REPRESENTATION
+
+
+
+
-
-
APLYSIA
Why similar circuit in MAMMALS and INVERTEBRATES?
Both solve similar environmental/behavioral problems!
SYNCHRONIZATION PROBLEM
Variable CS-US Delays
PERSISTENCE PROBLEMS
Multiple emotional meanings
CS1 CS2
CR1 CR2
Food Sex
Grossberg (1975)
(c) CELEST 200784
Why is ONSET of a shockNEGATIVELY REWARDING?
Why is OFFSET of a shock POSITIVELY REWARDING?
OPPONENT EMOTIONS IN DRIVE REPRESENTATIONS
FEAR vs RELIEF
FEAR
RELIEF
(c) CELEST 200785
REINFORCEMENT
VISUAL PERCEPTION
MacKay Illusion
ON
OFF
Shock on Fear (Estes & Skinner, 1941)Shock off Relief (Denny, 1971)
Picture off Negative Aftereffect
Picture on Percept
OPPONENT REBOUND IS UBIQUITOUS
(c) CELEST 200786
CS
US
Fear
Relief
OPPONENT PROCESSINGCognitive-Drive Associations
Primary excitatory associations Primary inhibitory associations
CS
US
Fear
CS
Fear
CS
Fear Relief
ON OFF
CS
Relief
Rebound
(c) CELEST 200787
BEHAVIORAL CONTRAST: REBOUNDS!
1. A sudden DECREASE in frequency or amount of FOOD can act as a NEGATIVE reinforcer: Frustration
2. A sudden DECREASE in frequency or amount of SHOCK can act as POSITIVE reinforcer: Relief
ShockLevel
Trials
(c) CELEST 200788
Responses per minute (VI schedule)
Daily sessions
TRIAL SHOCK LEVEL
1-56-10
11-1516-2021-25
0 Moderate 0 Intense 0
BEHAVIORAL CONTRAST
Reynolds (1968)
(c) CELEST 200789
MULTIPLE FUNCTIONAL ROLES OF SHOCK
1. Reinforcement sign reversalAn ISOLATED shock is a negative reinforcerIn certain CONTEXTS, a shock can be a positive reinforcer
2. STM-LTM interactionPrior shock levels need to be remembered (LTM) and used to calibrate the effect of the present shock (STM)
3. DISCRIMINATIVE AND SITUATIONAL CUESThe present shock level is UNEXPECTED (NOVEL) with respect to the shock levels that have previously been contingent upon experimental cues
1. Shock as a reinforcer
3. Shock as an expectancy2. Shock as a sensory cue
(c) CELEST 200790
How are ON and OFF reactions generated at the drive representations?
Through a
GATED DIPOLE
OPPONENT PROCESS
Grossberg (1972)
OPPONENT PROCESSING
(c) CELEST 200791
UNBIASED TRANSDUCER
S = inputT = outputT = SB
Suppose T is due to release of chemical transmitter y at a synapse:
S T y
RELEASE RATE: T = S y (mass action)
ACCUMULATION: y = B~
B is the gain
Grossberg (1968)
(c) CELEST 200792
TRANSMITTER ACCUMULATION AND RELEASE
T = S yy B
Differential Equation:
Transmitter y tries to recover to ensure unbiased transduction
y = A (B – y) – S ydt
d
Accumulate Release
What if it falls behind?
Transmitter y cannot be restored at an infinite rate:
Evolution has exploited the good properties that happen then
(c) CELEST 200793
HABITUATIVE TRANSMITTER GATE
T = S y
y = A (B – y) – S ydt
d
Recent experiments support this prediction:
Visual Cortex: Abbott et al. (1997): depressing synapses
Somatosensory Cortex: Markram et al. (1998)
(c) CELEST 200794
MINOR MATHEMATICAL MIRACLE
At equilibrium:
0 dy
dtA(B y) Sy
y AB
A S
y decreases when input S increases:
However, output Sy increases with S!
Sy ABS
A S(gate, mass action)
Transmitter
(c) CELEST 200795
HABITUATIVE TRANSMITTER GATE
ABS1
A S0
Weber Law
ABS0
A S 1
(c) CELEST 200796
NONRECURRENT GATED DIPOLE
(c) CELEST 200797
y1
ON-RESPONSE TO PHASIC ON-INPUT
S1=f(I+J) S2=f(I)
11
SA
BA y
22
SA
BA y
1
1111
SA
ABSyST
2
2222
SA
ABSyST
+ +- -
y2
s2s1
IJ
OFFON
T1T2
J))f(If(I))(A(A
f(I))-J)B(f(IAT-TON
2
21
Note Weber Law
When f has a threshold, small I requires larger J to fire due to numerator, but makes suprathreshold ON bigger due to denominator
When I is large, quadratic in denominator and upper bound of f make ON small
(c) CELEST 200798
OFF-REBOUND DUE TO PHASIC INPUT OFFSET
Shut off J (Not I!). Then:
A J)f(I
AB
1y f(I)A
ABy
2
y1 and y2 are SLOW
T1 = S1y1 T2 = S2y2
T1 < T2
J))f(If(I))(A(A
f(I))J)ABf(I)(f(ITTOFF
12
=
Why is the rebound transient? Note equal f(l) inputs
A
f(I)
ON
OFF=Arousal sets sensitivity of rebound:
<
S1 = f(l) and S2 = f(l)
Note Weber Law due to remembered previous input
(c) CELEST 200799
NOVELTY RESET: REBOUND TO AROUSAL ONSET
Equilibrate to I and J: S1=f(I+J) S2=f(I)
11 SA
BA y
2 S2A
BA y
Keep phasic input J fixed; increase arousal I to I* = I + ∆ I:
J))f(If(I))(A(A
J)*f(I)f(I-J)B(f(I*)f(I-J))*f(I-AB(f(I*)
How to interpret this complicated equation?
OFF = T2 - T1 = f(I*+J) y2 - f(I*) y1
OFF reaction if T1 < T2
(c) CELEST 2007100
NOVELTY RESET: REBOUND TO AROUSAL ONSET
f(w) f(w)= Cw: Linear signal
A)-IABJ(OFF
J)II)(A(A
OFF > 0 only if there is enough novelty: ∆I > A
∆I = I*- I
OFF response increases with J: If a given cell has a greater effect on a mismatchedexpectation, then it is reset more vigorously
Selective reset of dipole field by unexpected event
(c) CELEST 2007101
GOLDEN MEAN
Behavior
Arousal
Underaroused Depression Overaroused Depression
“UP” brings excitability “DOWN”
INVERTED U AS A FUNCTION OF AROUSAL
Elevated thresholdHyperexcitable above threshold
Low thresholdHypoexcitable above threshold
J))f(If(I))(A(Af(I))-J)B(f(IA2
ON
(c) CELEST 2007102
Consider the simplest type of
COGNITIVE-EMOTIONAL LEARNING
(c) CELEST 2007103
CLASSICAL CONDITIONING(Nonstationary prediction)Bell (CS) Bell (CS)
Fear (UR)
(CR)
ASSOCIATIVE LEARNING
A BCS US
ABCS US
ABCS US
A BCS US
CR
(c) CELEST 2007104
INTERSTIMULUS INTERVAL (ISI) EFFECT
ISI
CS
US
CR
0
0 ISILarge ISI obvious: No CS-US correlation
Why poor learning at 0 ISI, with good correlation?
(c) CELEST 2007105
INTERSTIMULUS INTERVAL (ISI) EFFECT
(c) CELEST 2007106
SECONDARY CONDITIONING(Advertising!)
CS1 becomes a CONDITIONED REINFORCER
CS2 becomes a CONDITIONED REINFORCER
CS1
CS2
CS1
FEAR
FEAR
US
(c) CELEST 2007107
How are
CLASSICAL CONDITIONING
and
ATTENTION
related?
(c) CELEST 2007108
PARALLEL PROCESSING OF EQUALLY SALIENT CUES
CS2
Light
FEAR
US
CS1
Bell t
t
t
t
vs. OVERSHADOWING (Pavlov)
CS1
CS2
FEAR
FEAR
(c) CELEST 2007109
BLOCKINGMINIMAL ADAPTIVE PREDICTION
CS1
CS2
FEAR
FEAR
CS2 IS IRRELEVANT
Phase I
Phase II
US
CS1
CS2
US
CS1
(c) CELEST 2007110
BLOCKING = ISI + SECONDARY CONDITIONING
Blocking Zero ISI
1)
2)
CS1
Fear
US
CS2
Fear
CS1
CS
Fear
US
No CS2 conditioning No CS conditioning
(c) CELEST 2007111
CS1 becomes a conditioned reinforcer by learning to activate a strong reinforcer-motivational (emotional) feedback pathway
Sensory Representation
Incentive Motivation
Drive Representation
Conditioned Reinforcer
CS1
US
+
CONDITIONED REINFORCER
(c) CELEST 2007112
CogEM EXPLANATION OF ATTENTIONAL BLOCKING
1. Sensory representations compete for LIMITED CAPACITY STM2. Previously reinforced cues amplify their STM via
POSITIVE FEEDBACK3. Other cues lose STM via COMPETITION
Competitionfor STM
SENSORY
MOTORDRIVE
SCS1
SCS2
CS1 CS2
+
+
D
ConditionedReinforcerLearning Incentive
MotivationalLearning
Internal Drive Input
MotorLearning
+
(c) CELEST 2007113
CS
SensoryInput
STMactivitywithoutmotivationalfeedback
STMactivitywith motivationalfeedback
+
time
(c) CELEST 2007114
BLOCKING
STM suppressedBy competition
STM amplifiedBy (+) feedback
X2
X1
t
t
X2
X1
CS1
CS2
+
(c) CELEST 2007115
SCS SUS
Sampling interval
POSITIVE ISI
CS input
US input
SCS activity
SUS activity
(c) CELEST 2007116
ISI EFFECT
Grossberg and Levine, 1987
(c) CELEST 2007117
CS1 D LTM Trace
EMOTIONAL CONDITIONING
CS1
+
D
Anticipatory CRa c
b d
CS1’S STM trace
D’s STM Trace
US’s STM Trace
(c) CELEST 2007118
CS2 D LTM Trace
CS1’S STM trace
CS2’s STM Trace
a c
b d
CS1 D LTM Trace
CS1 - US CS1 + CS2 - CS2
US
CS1
+
CS2
BLOCKING
(c) CELEST 2007119
UNIFIED EXPLANATION OF BLOCKINGISI EFFECT
SECONDARY CONDITIONING ANTICIPATORY CR
COOPERATION
between
COGNITIVE and EMOTIONAL
representations
COMPETITION
between
COGNITIVE
representations
(c) CELEST 2007120
MINIMAL ADAPTIVE PREDICTION
BLOCKING UNBLOCKING
CS1 US
CS1Fear
CS1 US1
CS1 Fear
CS1 + CS2 US
CS2 Fear
CS2
Fear if US2 > US1
CS2 is irrelevant CS2 predicts US change
Learn if CS2 predicts a different (novel) outcome than CS1
CS2 not redundant (“wallpaper”)
CS1 + CS2US2
xRelief if US2 < US1
1) 1)
2) 2)
(c) CELEST 2007121
MINIMAL ADAPTIVE PREDICTOR
CS2 US2t
(US1 US2) ><
HOW ART WAS DISCOVERED IN 1973!
1. Pay attention to (code, learn) RELEVANT cues
CS1 predicts US1
2. Unexpected CONSEQUENCES redefine the set of relevant cues
Changing US1 to US2 makes CS2 relevant
3. Unexpected consequence (NOVELTY) feeds back in time via a NONSPECIFIC event to redefine relevant cues
4. Distinguish NOVELTY from EMOTIONAL SIGN