a primer to computational modeling in psychology and … · 2020-01-31 · • case study:...
TRANSCRIPT
A Primer to Computational Modeling in Psychology and Neuroscience
My goals for today
• Not to teach you computational modeling
• Demystifying computational models
• Central message:
Computational models are not as complicated (nor as fancy) as they sound, and with a little bit of work,
everyone can incorporate it into their research
Outline
• What is a computational model?
• Case Study: Reinforcement learning models of decision-making
• Why computational modeling? • Picking up computational modeling
What is a computational model?
Building intuition from simple models
y=ax+b
How do we determine a and b? How do we quantify the error of the model? (i.e. how we measure how good the model is)?
Building intuition from simple models
Basic framework
• Fit the model(s) to dependent measure (i.e behavior or neural activity) to find optimal value for the free parameters
• Compare different models to see which model best explains dependent measure
Extending to more complicated models
• Algorithmic description of cognitive processes
• System of equations rather than one equation
• Multiple free, non-independent parameters, so difficult to systematically predict model behavior
Case Study:
Reinforcement learning models of decision-making
Classical Conditioning
Learning rule
Observedvalue
Temporal-difference learning algorithm
Expectedvalue
Update
ImmediateReward
DiscountedFutureReward
Intuition: You care about future rewards as well as current reward When do you need to update your values? When you are surprised! How much should you update? Depends on how surprised you were!
Dopamine Response
Schulz,Dayan&Montague,1997
Before training
After training
Dopamine Response
Dopamine Response
Pessiglione,etal.2006 O’Doherty,etal.2004
Wickens et al., 1996
Three-factor learning rule
Cortex = State Information Striatum = Value of States Dopamine = prediction error
(trains values)
Operant Conditioning
Why computational modeling?
Verbally expressed statements are sometimes flawed by internal inconsistencies, logical contradictions, theoretical weaknesses and gaps. A running computational model, on the other hand, can be considered as a sufficiency proof of the internal coherence and completeness of the ideas it is based upon...” (Fum, Del Misser, Stocco, 2007)
TakenfrompersonalpageofConstanRnA.Rothkopf,PhDhTps://fias.uni-frankfurt.de/~rothkopf/
Modeling Techniques
• Reinforcement Learning
• Symbolic systems
• Neural Networks
• Bayesian Networks • Agent-based models
• Many more …
All models are wrong, but some are useful
George Box
Learning how to model
• Learn how to code
• Basics of Modeling Computational Modeling in Cognition: Principles and Practice http://www.amazon.com/Computational-Modeling-Cognition-Principles-Practice/dp/1412970768
• Reinforcement Learning https://webdocs.cs.ualberta.ca/~sutton/book/ebook/the-book.html
• Neural Network Models https://grey.colorado.edu/CompCogNeuro/index.php/CCNBook/Main
• Probabilistic Models (i.e. Bayesian Models)
https://probmods.org/
Some additional pieces of information
Some additional pieces of information
Computers playing Atari games
AlphaGo vs. Lee Seedol (4 -1)