modeling transfer of learning in games of strategic interaction
DESCRIPTION
Modeling transfer of learning in games of strategic interaction. Ion Juvina & Christian Lebiere Department of Psychology Carnegie Mellon University. Background | Experiment | Model | In progress | Discussion. Outline. Background Experiment Cognitive model - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/1.jpg)
ACT-R Workshop July 2012 1
Modeling transfer of learning in games of strategic interaction
Ion Juvina & Christian Lebiere
Department of Psychology
Carnegie Mellon University
![Page 2: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/2.jpg)
2
Outline Background
Experiment
Cognitive model
Work in progress
Discussion
Background | Experiment | Model | In progress | Discussion
![Page 3: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/3.jpg)
3
Transfer of learning Alfred Binet (1899):
Formal discipline: Exercise of mental faculties -> generalization
Thorndike (1903): Identical element theory:
transfer of learning occurs only when identical elements of behavior are carried over from one task to another
Singley & Anderson (1989): Surface vs. deep similarities
Common “cognitive units”
Background | Experiment | Model | In progress | Discussion
![Page 4: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/4.jpg)
4
Transfer in strategic interaction
Bipartisan cooperation in Congress Golf -> bipartisanship?
Similarity? What is transferred?
Background | Experiment | Model | In progress | Discussion
![Page 5: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/5.jpg)
5
Prisoner’s Dilemma (PD)
Background | Experiment | Model | In progress | Discussion
![Page 6: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/6.jpg)
6
Chicken game (CG)
Background | Experiment | Model | In progress | Discussion
![Page 7: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/7.jpg)
PD & CG payoff matrices
PD A B
A -1, -1 10, -10
B -10, 10 1, 1
CG A B
A -10, -10 10, -1
B -1, 10 1, 1
Background | Experiment | Model | In progress | Discussion
![Page 8: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/8.jpg)
Similarities between PD & CG Surface (near transfer)
2X2 games 2 symmetric and 2 asymmetric outcomes [1,1] outcome is identical
Deep (far transfer) Mixed motive Non-zero sum Mutual cooperation is superior to competition
in long term Though unstable (risky)
Background | Experiment | Model | In progress | Discussion
![Page 9: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/9.jpg)
9
Differences between PD & CG Different equilibria:
Symmetric in PD: [-1,-1]
Asymmetric in CG: [-1, 10] and [10,-1]
Different strategies to maximize joint payoff (Pareto-efficient outcome): [1,1] in PD
Alternation of [-1,10] and [10,-1] in CG
Background | Experiment | Model | In progress | Discussion
![Page 10: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/10.jpg)
10
Questions / hypotheses Similarities
Identical element? Common cognitive units? Transfer of learning
Is there any transfer? Only in one direction?
Low – high entropy? (Bednar, Chen, Xiao Liu, & Page, in press) Identical element -> both ways
Mechanism of transfer Reciprocal trust mitigates the risk associated with the
long term solution (Hardin, 2002)
Background | Experiment | Model | In progress | Discussion
![Page 11: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/11.jpg)
Participants and design 480 participants (CMU students)
240 pairs 2 within-subjects games: PD & CG 4 between-subjects information conditions
No-info: 60 pairs Min-info: 60 pairs Mid-info: 60 pairs Max-info: 60 pairs
2 between-subjects order conditions in each information condition
PD-CG: 30 pairs CG-PD: 30 pairs
200 unnumbered rounds for each game
Background | Experiment | Model | In progress | Discussion
![Page 12: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/12.jpg)
Typical outcomes
Background | Experiment | Model | In progress | Discussion
![Page 13: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/13.jpg)
Pareto-optimal equilibria
Background | Experiment | Model | In progress | Discussion
![Page 14: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/14.jpg)
[1,1] increases with info
Background | Experiment | Model | In progress | Discussion
![Page 15: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/15.jpg)
Alternation increases with info
Background | Experiment | Model | In progress | Discussion
![Page 16: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/16.jpg)
PD – CG sequence
16
Background | Experiment | Model | In progress | Discussion
![Page 17: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/17.jpg)
CG – PD sequence
17
Background | Experiment | Model | In progress | Discussion
![Page 18: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/18.jpg)
18
PD before and after
Background | Experiment | Model | In progress | Discussion
![Page 19: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/19.jpg)
19
CG before and after
Background | Experiment | Model | In progress | Discussion
![Page 20: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/20.jpg)
20
Transfer from PD to CG Increased [1,1] (surface transfer)
Increased alternation (deep transfer)
Background | Experiment | Model | In progress | Discussion
![Page 21: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/21.jpg)
21
Transfer from CG to PD Increased [1,1] (surface + deep transf.)
Background | Experiment | Model | In progress | Discussion
![Page 22: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/22.jpg)
22
Divergent effects
[1,1]SurfaceSurface
DeepDeep
[1,1]
[10,-1] / [-1,10]
PD CG
Background | Experiment | Model | In progress | Discussion
![Page 23: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/23.jpg)
23
Convergent effects
[1,1]
SurfaceSurface
DeepDeep
[1,1]
[10,-1] / [-1,10]
CG PD
Background | Experiment | Model | In progress | Discussion
![Page 24: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/24.jpg)
Reciprocation by info
Background | Experiment | Model | In progress | Discussion
![Page 25: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/25.jpg)
Payoff by info in PD and CG
Background | Experiment | Model | In progress | Discussion
![Page 26: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/26.jpg)
26
Summary results Mutual cooperation increases with awareness of
interdependence (info) Transfer of learning
Better performance “after” than “before” Combined effects of surface and deep similarities
CG -> PD surface similarity facilitates transfer PD -> CG surface similarity interferes with transfer
Transfer occurs in both directions Mechanism of generalization
Reciprocal trust?
Background | Experiment | Model | In progress | Discussion
![Page 27: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/27.jpg)
27
Cognitive model Awareness of interdependence
Opponent modeling
Generality Utility learning (reinforcement learning)
Transfer of learning Surface transfer Deep transfer
Background | Experiment | Model | In progress | Discussion
![Page 28: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/28.jpg)
28
Opponent modeling Instance-based learning
Dynamic representation of the opponent
Sequence learning Prediction of opponent’s next move
Instance (snapshot of the current situation) Previous moves and opponent’s current move
Contextualized expectations
Background | Experiment | Model | In progress | Discussion
![Page 29: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/29.jpg)
29
Utility learning Reinforcement learning
Strategy: what move to make given Expected move of opponent Context (previous moves)
Reward functions Own payoff – Opponent’s payoff Opponent’s payoff Joint payoff – Opponent’s previous payoff
Background | Experiment | Model | In progress | Discussion
![Page 30: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/30.jpg)
30
Surface transfer Declarative sub-symbolic learning
Retrieval of instances guided by recency and frequency
Strategy learning A learned strategy continues to be used for
a while until it is unlearned
Background | Experiment | Model | In progress | Discussion
![Page 31: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/31.jpg)
31
Deep transfer Trust learning / Trust dynamics
Trust accumulator Increases when opponent makes cooperative (risky)
moves
Decreases when opponent makes competitive moves
Trust invest accumulator Increases with mutually destructive outcome
Decreases with unreciprocated cooperation (risk taking)
Background | Experiment | Model | In progress | Discussion
![Page 32: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/32.jpg)
32
Meta strategy Determines which reward function to use
Trust accumulator <= 0 Reward = own payoff – opponent’s payoff
Trust invest accumulator > 0 Reward = opponent’s payoff
Trust accumulator > 0 Reward = joint payoff – opp’s previous payoff
Background | Experiment | Model | In progress | Discussion
![Page 33: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/33.jpg)
Model diagram
HSCB 2011 33
InstanceCurrent moves: A BPrevious moves: A A
Declarative Memory
Inst2Inst1
Inst4
Inst3
PredictionPrevious moves: A BOpponent move: A
Procedural Memory
Rule2
Rule1 Rule3
MoveBest response: A Predicted move: A
Trust Trust accumulatorTrust invest
Opponent MoveA
Reward
ACT-R
Environment
ACT-R extension
Background | Experiment | Model | In progress | Discussion
![Page 34: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/34.jpg)
PD-CG
Background | Experiment | Model | In progress | Discussion
![Page 35: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/35.jpg)
CG-PD
Background | Experiment | Model | In progress | Discussion
![Page 36: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/36.jpg)
PD-CG surface transfer
Background | Experiment | Model | In progress | Discussion
![Page 37: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/37.jpg)
PD-CG deep transfer
Background | Experiment | Model | In progress | Discussion
![Page 38: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/38.jpg)
CG – PD surf+deep transfer
Background | Experiment | Model | In progress | Discussion
![Page 39: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/39.jpg)
Trust simulation
Background | Experiment | Model | In progress | Discussion
![Page 40: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/40.jpg)
40
Summary model results Awareness of interdependence
Opponent modeling
Generality Utility learning
Transfer of learning Surface level transfer: cognitive units
Deep level transfer: Trust
Background | Experiment | Model | In progress | Discussion
![Page 41: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/41.jpg)
41
In progress Expand model to account for all
information conditions
Develop more ecologically valid paradigm (IPD^3)
Model “affective” processes in ACT-R
Background | Experiment | Model | In progress | Discussion
![Page 42: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/42.jpg)
42
IPD^3
Background | Experiment | Model | In progress | Discussion
![Page 43: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/43.jpg)
43
General discussion Transfer of learning is possible
Deep similarities: interpersonal level IPD^3
To be used in behavioral experiments Tool for learning strategic interaction skills
Background | Experiment | Model | In progress | Discussion
![Page 44: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/44.jpg)
44
Acknowledgments
Coty Gonzalez Jolie Martin Hau-Yu Wong Muniba Saleem This research is supported by the Defense
Threat Reduction Agency (DTRA) grant number: HDTRA1-09-1-0053 to Cleotilde Gonzalez and Christian Lebiere
![Page 45: Modeling transfer of learning in games of strategic interaction](https://reader035.vdocument.in/reader035/viewer/2022062800/56814043550346895dabb1ff/html5/thumbnails/45.jpg)
45
Thank you for your attention! Questions?