Lecture 3

Ongoing interactions: Dynamic games of

complete information - Part 1

Outline • Extensive-form games

– Sub-game perfect equilibrium, one-step deviation principle – Stackelberg equilibrium

• Interventions: Policing, treats, punishments • Repeated games – formalism, equilibria, automata • Direct vs. Indirect Reciprocity

– Tit-for-Tat etc. – Ratings, Reputations – Tokens/fiat-money

• Bayesian Games • Illustrative examples and comparisons

Questions/comments/observations are always encouraged, at any point during the lecture!!

Dynamic games

• players • who moves at each turn • knowledge at each turn • choices at each turn • outcomes • appropriate notion of solution

Include • Hidden actions (normal form games) • Hidden information (Bayesian games)

Framework + solution notion(s) – complicated Begin with • Complete information: no hidden actions • Perfect information: no hidden information

Extensive form games

• The normal form game representation does not incorporate any notion of sequence, or time, of the actions of the players

• The extensive form is an alternative representation that makes this temporal structure explicit.

• Two variants: – perfect information extensive-form games (= a game

where every player knows all the moves played by all the players before)

– imperfect-information extensive-form games

Extensive form games - definition

Extensive form games - definition

Extensive form games - definition

Game-tree representation

Example

Example

Game-tree representation

Example: the sharing game

Pure-strategies in perfect-information games

Pure-strategies example

Nash equilibria

Idea: the game – finite => has a set of penultimate nodes => players moving at these nodes choose strategies leading to the terminal node(s) with max payoff. The players That have successors the penultimate nodes, choose actions that max their payoffs, Given the choice of penultimate nodes etc. ……. resulting strategy is PSNE

Induced normal form

Induced normal form

Induced normal form

New (stronger) equilibrium concepts needed?

MAD game

On the board

Chain-store game

We now define an equilibrium refinement concept that does not suffer from this threat credibility problem

Subgame

Simplified: A subgame is determined by assuming that a certain history has already happened and considering the game from that point onward

Subgame example

Subgame perfect equilibrium

Backwards induction algorithm • Identify all terminal subgames • Determine the Nash eq. for these subgames • Modify the original game tree by replacing the terminal subgames with the Nash equilibrium payoffs • Repeat until the three is reduced to one stage game, and then determine the Nash equilibrium.

Example backwards induction algorithm

What about the MAD and CS games?

MAD Game

Chain Store Game

Subgame perfect equilibrium - example

Bargaining Game of Alternating Offers (Extensive Form Game)

Player 1

(1), (1)x M x−

Player 2 (1), (1)x M x−

accept reject

Player 2

(2), (2)M y y−

Round 1

Player 1

accept reject

(2), (2)M y y−Player 1 Round 2

Round 3

Another example: Alternative offers (Bargaining)

Another example: Alternative offers (Bargaining)

Another example: Alternative offers (Bargaining)

Another example: Stackelberg competition

Theorem (Zermelo) • Every finite extensive form game of complete

and perfect information has a sub-game perfect equilibrium (SGPE). If there are no ties, SGPE is unique.

• Proof: Backwards induction “Pruning the tree”

Remarks

Imperfect information

Example

Recall:

Example

Information sets for player 1: { } and {( , ),( , )}

4 pure strategies for player 1, corresponding to the

information sets { } and {( , ),( , )}:

and , and , and and and

L A L B

L A L B

L l L r R l R r

Normal form game

Induced normal form

Mixed and behavioral strategies

• It turns out there are two meaningfully different kinds of randomized strategies in imperfect information extensive form games

– mixed strategies – behavioral strategies

• Mixed strategy: randomize over pure strategies • Behavioral strategy: independent coin toss every time an

information set is encountered • A mixed strategy is a distribution of vectors (each vector

describing a pure strategy) • A behavioral strategy is a vector of distributions

Mixed and behavioral strategies

• Two types of randomization: • Mixed strategy: user randomizes over pure strategies • Behavioral strategy: users plans a collection of

randomizations, one for each point at which it needs to take an action

• Mixed strategies and behavioral strategies are

equivalent for games of perfect recall – No player ever forgets any information it knew

• Nash eq. in behavioral strategies = profile such that no player can increase its expected payoff by using a different behavioral strategy.

• Give an example of a behavioral strategy: – A with probability .5 and G with probability .3

• Give an example of a mixed strategy that is not a behavioral strategy:

– (.6(A,G), .4(B,H)) (Why not? Because the choices made by him at the two nodes are not independent)

• In this game every behavioral strategy corresponds to a mixed strategy..

• What about imperfect-recall games? Examples next

Games of imperfect recall

Games of imperfect recall