oppa european social fund prague & eu: we invest in your ... › w › _media › misc ›...
TRANSCRIPT
![Page 1: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/1.jpg)
OPPA European Social FundPrague & EU: We invest in your future.
![Page 2: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/2.jpg)
A4M33MAS - Multiagent SystemsGame Theory: Extensive Form Games
• Michal Pechoucek & Michal Jakob• Department of Computer Science
Czech Technical University in Prague
In parts based on Kevin Leyton-Brown: Foundations of Multiagent Systems an introduction to algorithmic game theory, mechanism design and auctions
![Page 3: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/3.jpg)
Introduction• The normal form game representation does not incorporate any
notion of sequence, or time, of the actions of the players• The extensive form is an alternative representation that makes the
temporal structure explicit.• Two variants:
– perfect information extensive-form games– imperfect-information extensive-form games
2
![Page 4: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/4.jpg)
Definition
3
![Page 5: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/5.jpg)
Definition
4
![Page 6: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/6.jpg)
Definition
5
![Page 7: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/7.jpg)
Definition
6
![Page 8: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/8.jpg)
Definition
7
![Page 9: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/9.jpg)
Definition
8
![Page 10: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/10.jpg)
Definition
9
![Page 11: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/11.jpg)
Definition
10
![Page 12: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/12.jpg)
Example: Sharing game
Play as a fun game, dividing 100 dollar coins. (Play each partner only once.)
![Page 13: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/13.jpg)
Pure StrategyOverall, a pure strategy for a player in a perfect-information game is a complete speciffication of which deterministic action to take at every node belonging to that player.
![Page 14: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/14.jpg)
Pure StrategyOverall, a pure strategy for a player in a perfect-information game is a complete speciffication of which deterministic action to take at every node belonging to that player.
![Page 15: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/15.jpg)
Example: Sharing game
Play as a fun game, dividing 100 dollar coins. (Play each partner only once.)
![Page 16: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/16.jpg)
Pure StrategyOverall, a pure strategy for a player in a perfect-information game is a complete speciffication of which deterministic action to take at every node belonging to that player.
15
![Page 17: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/17.jpg)
Example: Sharing game
![Page 18: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/18.jpg)
Example: Sharing game
![Page 19: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/19.jpg)
Example
![Page 20: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/20.jpg)
Example
![Page 21: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/21.jpg)
Nash Equilibrium
• Why?
20
Pure Strategy Nash Equilibrium
![Page 22: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/22.jpg)
Induced normal
![Page 23: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/23.jpg)
Induced normal
22
![Page 24: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/24.jpg)
Induced normal
23
![Page 25: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/25.jpg)
Induced normal
24
![Page 26: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/26.jpg)
Induced normal
25
![Page 27: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/27.jpg)
Induced normal
26
![Page 28: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/28.jpg)
Nash Equilibrium?
27
![Page 29: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/29.jpg)
Subgame perfect NE?
28
![Page 30: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/30.jpg)
Subgame perfect NE?• The subgame of G rooted at h is the restriction of G to the
descendents of H. The set of subgames of G is defined by the subgames of G rooted at each of the nodes in G.
Definition (Subgame perfect Nash Equilibrium):s is a subgame perfect equilibrium of G i for any subgame G’ of G, the restriction of s to G’ is a Nash equilibrium of G’.
Notes:• since G is its own subgame, every SPE is a NE.• this definition rules out non-credible threats
29
![Page 31: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/31.jpg)
(A,G) (C,F)(B,H) (C,E)(A,H) (C,F)
30
Subgame Perfect Nash Equilibrium
![Page 32: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/32.jpg)
(A,G) (C,F)(B,H) (C,E)(A,H) (C,F)
31
Subgame Perfect Nash Equilibrium
![Page 33: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/33.jpg)
(A,G) (C,F)(B,H) (C,E)(A,H) (C,F)
32
Subgame Perfect Nash Equilibrium
![Page 34: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/34.jpg)
(A,G) (C,F)(B,H) (C,E)(A,H) (C,F)
33
Subgame Perfect Nash Equilibrium
![Page 35: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/35.jpg)
Computing SPE: Backward Induction• Idea: Identify the equilibria in the bottom-most trees, and adopt these
as one moves up the tree
• The procedure doesn't return an equilibrium strategy, but rather labels each node with a vector of real numbers. This labeling can be seen as an extension of the game's utility function to the non-terminal nodes
• The equilibrium strategies: take the best action at each node
![Page 36: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/36.jpg)
Computing SPE: Minimax Algorithm• Idea: For zero-sum games, Backward Induction is called the Minimax
algorithm.• It is enough to store one number per nodeand speed things up by
pruning nodes that will never be reached: alpha-beta pruning
![Page 37: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/37.jpg)
Computing SPE: Minimax Algorithm• It is enough to store one number per nodeand speed things up by
pruning nodes that will never be reached: alpha-beta pruning
![Page 38: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/38.jpg)
37
Imperfect Information EF Games
![Page 39: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/39.jpg)
38
Cooperate Defect
Cooperatex﹣1, ﹣1 ﹣4, 0
Defectx 0, ﹣4 ﹣3, ﹣3
Imperfect Information EF Games
![Page 40: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/40.jpg)
39
Imperfect Information EF GamesCooperate Defect
Cooperatex﹣1, ﹣1 ﹣4, 0
Defectx 0, ﹣4 ﹣3, ﹣3
![Page 41: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/41.jpg)
40
Imperfect Information EF GamesCooperate Defect
Cooperatex﹣1, ﹣1 ﹣4, 0
Defectx 0, ﹣4 ﹣3, ﹣3
![Page 42: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/42.jpg)
Imperfect Information EF Games
41
![Page 43: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/43.jpg)
Imperfect Information EF Games
42
![Page 44: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/44.jpg)
Imperfect Information EF Games
43
h0h
![Page 45: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/45.jpg)
Imperfect Information EF Games
44
h0h
I1,1
I2,1
I1,2
![Page 46: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/46.jpg)
Imperfect Information EF Games
45
![Page 47: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/47.jpg)
Imperfect Information EF Games
• Pure strategy: Cartesian product over action functions for each equivalence class
• Mixed strategy: distribution over vector denoting pure strategies (mixed strategy randomly chooses a deterministic path through the game tree)
• Behavioural strategy: deterministically selected stochastic paths across the game tree.
46
![Page 48: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/48.jpg)
47
![Page 49: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/49.jpg)
Example• mixed strategy: R is sitrictly dominant for player 1, D is best
response to player 2. (R,D) is in Nash.
48
![Page 50: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/50.jpg)
Example• mixed strategy: • behavioural strategy: 1 choosing L with probability p each time in
the information set. Utility obtained is than:
maximizing at =
49
1 ⇤ p2 + 100 ⇤ p(1� p) + 2 ⇤ (1� p)
�99p2 + 98p+ 2 98
198p
![Page 51: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/51.jpg)
Perfect recallThere is a broad class of imperfect-information games in which the expressive power of mixed and behavioral strategies coincides. This is the class of games of perfect recall. Intuitively speaking, in these games no player forgets any information he knew about moves made so far; in particular, he remembers precisely all his own moves.
50
![Page 52: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/52.jpg)
Perfect recall
51
![Page 53: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/53.jpg)
Perfect recall
52
![Page 54: OPPA European Social Fund Prague & EU: We invest in your ... › w › _media › misc › projects › oppa_oi_english … · Prague & EU: We invest in your future. A4M33MAS - Multiagent](https://reader033.vdocument.in/reader033/viewer/2022042403/5f14d111ddce2f5d2c390f4d/html5/thumbnails/54.jpg)
OPPA European Social FundPrague & EU: We invest in your future.