Lecture Notes on Game Theory1

Tilman Borgers, Department of Economics, University of Michigan

August 31, 2021

1. Definitions

Definition 1. A strategic game is a list (N, (Ai)i∈N , (ui)i∈N ) where N isa finite set of the form N = {1, 2, . . . , n} (with n ≥ 2), for every i ∈ N Aiis a non-empty, finite set, and for every i ∈ N ui is a real-valued functionwith domain A = ×i∈NAi.

N is the set of players. Ai is the set of player i’s actions, and ui is playeri’s von Neumann Morgenstern utility function. We write ai for elements ofAi. We refer to elements of Ai also as pure (as opposed to mixed) actions ofplayer i. We write a for elements of A. We write A−i for ×j∈N,j 6=iAj . Wewrite a−i for elements of A−i.

Definition 2. A mixed action αi for player i is a probability distributionover Ai.

We identify the mixed action αi that place probability 1 on action ai withthis action itself. For any finite setX we write ∆(X) for the set of probabilitydistributions over X, and thus ∆(Ai) is the set of mixed actions of playeri. We write α for elements of ×i∈N∆(Ai). We write α−i for elements of×j∈N,j 6=i∆(Aj). The expected utility of player i when players choose mixedaction profile α is:

(1) Ui(α) =∑

(a1,a2,...,an)∈A

ui(a1, a2, . . . , an)∏j∈N

αj(aj)

Note that this definition implicitly assumes that different players’ random-izations when implementing their mixed action are independent. We shallmake this assumption throughout.

2. Best Replies and Actions that are not Strictly Dominated

Two basic ideas about rationality in games are (i) rational players onlychoose actions that maximize expected utility for some belief about theother players’ actions, and (ii) rational players only choose actions that arenot strictly dominated by other actions. Here, we formalize these two ideas,

1Definitions and notation in these notes are very similar, although not completelyidentical, to the definitions and notation in Osborne and Rubinstein [1].

1

2

and then demonstrate that they imply exactly the same predictions aboutthe behavior of rational players.

Definition 3. A belief µi of player i is an element of ∆(A−i). An actionai ∈ Ai is a best reply to a belief µi if

(2) ai ∈ arg maxa′i∈Ai

∑a−i∈A−i

(ui(a

′i, a−i)µi(a−i)

).

Note that we do not require beliefs to be the products of their marginals,that is, we don’t require player i to believe that the other players’ actionsare stochastically independent. If we did, Proposition 1 below would not betrue.

Definition 4. An action ai ∈ Ai is strictly dominated by a mixed actionαi ∈ ∆(Ai) if

(3) Ui(αi, a−i) > ui(ai, a−i) ∀a−i ∈ A−i.

Note that we allow the strictly dominating action to be a mixed action.If we did not, then Proposition 1 below would not be true.

Proposition 1. An action a∗i ∈ Ai is a best reply to some belief µi ∈ ∆(A−i)if and only if a∗i is not strictly dominated.

Proof. Step 1: We prove the “only if” part, that is, we assume that a∗iis a best reply to a belief µi ∈ ∆(A−i), and infer that a∗i is not strictlydominated. The proof is indirect. Suppose a∗i were strictly dominated byαi ∈ ∆(Ai). Then, obviously, αi yields strictly higher expected utility giventhe belief µi than a∗i :

(4)∑

a−i∈A−i

Ui(αi, a−i)µi(a−i) >∑

a−i∈A−i

ui(a∗i , a−i)µi(a−i).

We thus have that αi is a better response to µi than a∗i , which is almostwhat we want to obtain, but not quite. To obtain the desired contradiction,we want to find a pure action that is a better response to µi than a∗i . Thiscan be done as follows. We re-write the left hand side of (4) as follows:∑a−i∈A−i

Ui(αi, a−i)µi(a−i) =∑

a−i∈A−i

∑ai∈Ai

αi(ai)ui(αi, a−i)µi(a−i)

=∑ai∈Ai

∑a−i∈A−i

αi(ai)ui(αi, a−i)µi(a−i)

=∑ai∈Ai

αi(ai)

∑a−i∈A−i

ui(ai, a−i)µi(a−i)

.(5)

3

Combining (4) and (5) we have:

(6)∑ai∈Ai

αi(ai)

∑a−i∈A−i

ui(ai, a−i)µi(a−i)

>∑

a−i∈A−i

ui(a∗i , a−i)µi(a−i).

The left hand side of (6) is a convex combination of the expressions in largebrackets in that term. This convex combination can be larger than the righthand side of (6) only if one of the expressions in large brackets is larger thanthe right hand side of (6), i.e., for some ai ∈ Ai:

(7)∑

a−i∈A−i

ui(ai, a−i)µi(a−i) >∑

a−i∈A−i

ui(a∗i , a−i)µi(a−i).

and thus ai is a better response to µi than a∗i , which contradicts the as-sumption a∗i is a best response to µi among all pure actions.

Step 2:2 We prove the “if” part, that is, we assume that a∗i is not strictlydominated, and we show that there is a belief µi ∈ ∆(A−i) to which a∗i isa best response. The proof is constructive. We define two subsets, X andY , of the set R|A−i|, that is, the Euclidean space with dimension equal tothe number of elements of A−i. We shall think of the elements of these setsas payoff vectors. Each component indicates a payoff that player i receiveswhen the other players choose some particular a−i ∈ A−i.

Now pick any one-to-one mapping f : A−i −→ {1, 2, . . . , |A−i|}. For any

action ai of player i, we write ui(ai, 〈a−i〉) ∈ R|A−i| for the vector of payoffsthat player i receives when playing ai, and when the other players play theirvarious action combinations a−i. Specifically, the k-th entry of ui(ai, 〈a−i〉)is the payoff ui(ai, f

−1(k)) where f−1 is the inverse of f . Intuitively, f definesan order in which we enumerate the elements of A−i, and ui(ai, 〈a−i〉) liststhe payoffs of player i when he plays ai and the other players play a−i inthe order defined by f .

The set X is:

(8) X = {x ∈ R|A−i||x > ui(a∗i , 〈a−i〉)}.

Here, “>” is to be interpreted as: “strictly greater in every component.”Therefore, the set X is the set of payoff vectors that are strictly greaterin every component than ui(a

∗i , 〈a−i〉), that is the payoff vector that corre-

sponds to the undominated action a∗i .

The set Y is:

(9) Y = co{y ∈ R|A−i||∃ai ∈ Ai : y = ui(ai, 〈a−i〉)}.

Here, “co” stands for “convex hull.” The payoff vectors in Y are the payoffvectors that player i can achieve by choosing some mixed action. The weightthat the convex combination that defines an element of y places on each

2An example and a graph that illustrate Step 2 follow after the end of the proof.

4

element of {x ∈ R|A−i||∃ai ∈ Ai : x = ui(ai, 〈a−i〉)} corresponds to theprobability which the mixed action assigns to each pure action ai ∈ Ai.

It is obvious that both sets are nonempty and convex sets. Moreover, theirintersection is empty. If X and Y overlapped, then every common elementwould correspond to the payoffs arising from a mixed action of player i thatstrictly dominates a∗i . Because by assumption no such mixed action exists,X and Y cannot have any elements in common.

In the previous paragraph we have checked all the assumptions of theseparating hyperplane theorem: we have two nonempty and convex sets thathave no elements in common. The separating hyperplane theorem (Theorem

1.68 in Sundaram [4]) then says that there exist some row vector π ∈ R|A−i|

which is not equal to zero in every component, and some δ ∈ R, such that:

(10) π · x ≥ δ ∀x ∈ X

and

(11) π · y ≤ δ ∀y ∈ Y.

Here “·” stands for the scalar product of two vectors. We treat all vectorsin X and Y as column vectors. Therefore, the above scalar products arewell-defined.

We now make two observations. The first is:

(12) δ = π · ui(a∗i , 〈a−i〉).

To show this note that by definition ui(a∗i , 〈a−i〉) ∈ Y , and therefore, by (11),

π · ui(a∗i , 〈a−i〉) ≤ δ. Next, for every n ∈ N define xn = ui(a∗i , 〈a−i〉) + εn · ι,

where ε ∈ (0, 1) is some constant and ι is the column vector in R|A−i| inwhich all entries are “1”. Observe that for every n ∈ N we have xn ∈ X,so that by (10) we have: π · xn ≥ δ for every n. On the other hand, wehave: limn→∞ xn = ui(a

∗i , 〈a−i〉). By the continuity of the scalar product

of vectors therefore: π · ui(a∗i , 〈a−i〉) ≥ δ. This, combined with our earlierobservation π · ui(a∗i , 〈a−i〉) ≤ δ implies (12).

Our second observation is:

(13) π ≥ 0

where we interpret “≥” to mean “greater or equal in every component,” butnot identical, and 0 stands for the vector consisting of zeros only. That π isnot equal to 0 is part of the assertion of the separating hyperplane theorem.We shall prove indirectly that no component of π can be negative. Assumethat π has a negative component. Without loss of generality, assume thatit is the first. Now we consider the vector x = ui(a

∗i , 〈a−i〉) + (1, ε, ε, . . . , ε)

where ε > 0 is some number. Clearly, the vector x that we define like thisis contained in X. Moreover,

(14) π · x = π · ui(a∗i , 〈a−i〉) + π · (1, ε, ε, . . . , ε) = δ + π · (1, ε, ε, . . . , ε)

5

where for the second equality we have used (12). We now want to evaluateπ · (1, ε, ε, . . . , ε). Unfortunately, we need some additional notation: weshall use for the k -th component of the vector π the symbol: πk. Here,k ∈ {1, 2, . . . , |A−i|}. We then obtain:

(15) π · (1, ε, ε, . . . , ε) = π1 + ε

|A−i|∑k=2

πk

Now observe that by the contrapositive assumption of our indirect proofπ1 < 0. Therefore, for small enough ε the right hand side of (15) is negative.Using this fact, we obtain from (14):

(16) π · x < δ

which contradicts x ∈ X and (10). This completes our indirect proof of(13).

Now we denote by ‖π‖ the Euclidean norm of π. Because π 6= 0, if wedefine µi by:

(17) µi =1

‖π‖π,

then the Euclidean norm of µi is 1, and thus µi ∈ ∆(A−i), that is, µi isa belief of player i. We complete the proof by showing that a∗i is a bestresponse to µi. (11) and (12) together imply:

(18) π · y ≤ π · ui(a∗i , 〈a−i〉) ∀y ∈ Y.Dividing this inequality by ‖π‖ we get:

(19) µi · y ≤ µi · ui(a∗i , 〈a−i〉) ∀y ∈ Y.Now by definition of Y for every ai: ui(ai, 〈a−i〉) ∈ Y . Therefore:

(20) µi · ui(ai, 〈a−i〉) ≤ µi · ui(a∗i , 〈a−i〉) ∀ai ∈ Ai.This is what we wanted to prove: a∗i is a best response in Ai to µi. �

We illustrate Step 2 of the proof of Proposition 1 with the following gamein which player 1 chooses rows and player 2 chooses columns. Only thepayoffs of player 1 are shown.

L R

T 2 2

MA 3 -2

MB 0 -1

MC -1 0

B -2 3

Example 1

6

Player 1’s action T is not strictly dominated. We illustrate in Figure 1the construction of beliefs to which action T is a best reply. The sets Xand Y to which Step 2 of the proof of Proposition 1 refers are shown in thefigure.

Y

X

-3 -2 -1 1 2 3

-3-2

-11

23

ui(ai, L)

ui(ai, R)

Figure 1

The hyperplane (straight line) separating X and Y is the dashed line inFigure 1. Figure 1 also shows the orthogonal vector for this hyperplane.

3. Cautious Best Replies and Actions that are not Weakly Dominated

A cautious player might hold a belief that attaches some strictly positive,although possibly arbitrarily small, probability to every action profile of theother players. Here we show that the set of best responses to such beliefs isexactly the set of actions that are not weakly dominated.

Definition 5. An action ai ∈ Ai is weakly dominated by a mixed actionαi ∈ ∆(Ai) if

(21) Ui(αi, a−i) ≥ ui(ai, a−i) ∀a−i ∈ A−iand

(22) Ui(αi, a−i) > ui(ai, a−i) ∃a−i ∈ A−i.

Proposition 2. In a finite strategic game an action a∗i is a best reply amongall pure actions of player i to a belief µi ∈ ∆(A−i) with full support, i.e.µi(a−i) > 0 for every a−i ∈ A−i, if and only if a∗i is not weakly dominated.

7

Before proving Proposition 2 we shall illustrate by means of an examplethat the construction of Step 2 of the proof of Proposition 1 does notnecessarily yield a belief µi with full support and therefore can not be usedto prove Proposition 1. Indeed, that construction can be done even if anaction is weakly dominated. In this case the construction yields a beliefthat assigns zero probability to some of the action combinations of the otherplayers. Our example is the same as example 1 except that we assume thatplayer 1 has an additional action that yields payoffs 3 and 2, dependingwhether player 2 chooses L or R. This action thus weakly dominates T . Wealso assume that action B yields payoffs -2 and 2 rather than -2 and 3, aswas the case before. This is to avoid that a mixed action strictly dominatesT . The sets X and Y from Step 2 of the proof of Proposition 1 for thismodified example are shown in Figure 2.

Y

X

-3 -2 -1 1 2 3

-3-2

-11

23

ui(ai, L)

ui(ai, R)

Figure 2

The hyperplane separating X and Y is the dashed line in Figure 2. Theorthogonal vector that we have drawn has coordinates 0 and 1, which corre-spond to the belief of player 1 that puts probability 1 on player 2 playing R.The construction in the proof of Proposition 1 allows thus beliefs that putprobability zero on some actions. We demonstrate below how a modifiedconstruction shows that full support beliefs can be obtained whenever anaction is not weakly dominated.

Proof. The argument that proves that a best response to a full support beliefis not weakly dominated is essentially the same as Step 1 of the proof ofProposition 1, and we omit it.3

3All that changes is the reasoning that leads to inequality (4), and which we did noteven spell out in the proof of Proposition 1.

8

The proof of the converse is a less obvious modification of the argument inStep 2 of the proof of Proposition 1.4 As in that proof, we shall construct athe belief to which a not weakly dominated action is a best reply, and thatbelief will be the orthogonal vector of a hyperplane separating two sets Xand Y . To ensure that this belief has full support, we shall use a strongerseparating hyperplane theorem than we used in the proof of Proposition 1.Specifically, it will be useful to appeal to a separating hyperplane theoremthat allows us to replace the weak inequality in (10) above by a strict in-equality. Such a theorem is the “strong separating hyperplane theorem ”which is Theorem 6 in Border [2]. To apply this theorem we need, in par-ticular, that the set X is compact. To achieve this, we have to change thedefinition of X substantially. This will be done below. This change in thedefinition of X will then also necessitate a small change in the definition ofY . Hopefully, the remainder of the proof will clarify how these changes inthe definitions of X and Y are used in the proof.

We define X and Y as follows:

(23) X = {x ∈ R|A−i||x = ui(a∗i , 〈a−i〉) + y for some y ∈ ∆}.

(where ∆ is the unit simplex in R|A−i|) and

Y ={y ∈ R|A−i|| y = ui(a

∗i , 〈a−i〉)(24)

+∑ai∈Ai

λ(ai) (ui(ai, 〈a−i〉)− ui(a∗i , 〈a−i〉))

for some function λ : Ai → R+

}.

Thus, X is now the set of all payoff vectors that result if we add an elementof the unit simplex to ui(a

∗i , 〈a−i〉), whereas before it was the set of all payoff

vectors that are strictly greater in every component than ui(a∗i , 〈a−i〉). Note

that the modified definition of X leaves X a convex set. Obviously, Xis also compact. The set Y is the smallest cone with vertex ui(a

∗i , 〈a−i〉)

that includes the set Y as previously defined. Observe that with this newdefinition Y is convex. By Lemma 6 in Border [3] it is also closed becauseit is a finitely generated convex cone. It is generated by the finite collectionof vectors ui(ai, 〈a−i〉)− ui(a∗i , 〈a−i〉) (where ai ∈ Ai).

The properties of X and Y listed in the previous paragraph are sufficientto allow us to apply the strong separating hyperplane theorem provided thatX and Y are disjoint. We prove this indirectly. Suppose z ∈ X∩Y . Observethat z ∈ X implies:

(25) y ≥ ui(a∗i , 〈a−i〉).

4An example and a graph that illustrate the modification of Step 2 follow after theend of the proof.

9

Because z ∈ Y there is therefore some function λ : Ai → R+ such that:

ui(a∗i , 〈a−i〉) +

∑ai∈Ai

λ(ai) (ui(ai, 〈a−i〉)− ui(a∗i , 〈a−i〉)) ≥ ui(a∗i , 〈a−i〉)(26)

which is equivalent to:∑ai∈Ai

λ(ai)ui(ai, 〈a−i〉) ≥∑ai∈Ai

λ(ai)ui(a∗i , 〈a−i〉).(27)

Because the two sides of (27) are not equal, we have to have λ(ai) 6= 0for at least some ai ∈ Ai. We can therefore divide both sides of (27) by∑

ai∈Aiλ(ai) and obtain:

(28)∑ai∈Ai

λ(ai)∑a′i∈Ai

λ(a′i)ui(ai, 〈a−i〉) ≥ ui(a∗i , 〈a−i〉)

Inequality (28) says that a∗i is weakly dominated by the mixed action whichgives each action in Ai the probability λ(ai)/

∑a′i∈Ai

λ(a′i). This contradicts

our assumption that a∗i is not weakly dominated.

We can now conclude using the strong separating hyperplane theorem(Theorem 6 in Border [2] that X and Y can be strongly separated, that is,

there are some π ∈ R|A−i| which is not equal to zero in every component,and some δ ∈ R, such that:

(29) π · x > δ ∀x ∈ Xand

(30) π · y ≤ δ ∀y ∈ Y.

We have two observations about π and δ. The first is:

(31) δ ≥ π · ui(a∗i , 〈a−i〉).This follows from (30) and ui(a

∗i , 〈a−i〉) ∈ Y . The second observation is:

(32) π � 0.

where “�” means “strictly greater in every component.” Suppose (32) werenot true, that is, one component of π, say the k-th component, were less thanor equal to zero. Denote by ιk the unit vector in ∆ that equals 1 in the k-thcomponent and zero in all other components. Thus ui(a

∗i , 〈a−i〉) + ιk ∈ X.

Moreover, by our assumption about π, we would have:

(33) π · (ui(a∗i , 〈a−i〉) + ιk) ≤ π · ui(a∗i , 〈a−i〉) ≤ δwhere the last inequality is (31). But (33) contradicts (29).

Inequality (32) allows us to normalize π so that it becomes a probabilityvector. We denote this probability vector by µi.

(34) µi =π

‖π‖

10

Note that µi � 0, and therefore we can complete the proof by showing thata∗i maximizes expected utility in Ai when beliefs are µi.

Obviously, inequality (30) holds if we replace π by µi and divide the righthand side by ‖π‖.

(35) µi · y ≤δ

‖π‖∀y ∈ Y.

Now suppose that a∗i did not maximize expected utility in Ai:

(36) µi · (ui(ai, 〈a−i〉)− ui(a∗i , 〈a−i〉)) > 0

for some ai ∈ Ai. We can find some large enough λ > 0 such that:

(37) µi · λ (ui(ai, 〈a−i〉)− ui(a∗i , 〈a−i〉)) > µi · ui(a∗i , 〈a−i〉) +δ

‖π‖This is equivalent to:

(38) µi ·(λ (ui(ai, 〈a−i〉)− ui(a∗i , 〈a−i〉))− ui(a∗i , 〈a−i〉)

)>

δ

‖π‖

Now note that(λ (ui(ai, 〈a−i〉)− ui(a∗i , 〈a−i〉))− ui(a∗i , 〈a−i〉)

)∈ Y (set

λ(ai) = λ and λ(a′i) = 0 for all a′i 6= ai). Thus, we have found an ele-ment y of Y such that µi · y > δ/‖π‖, which contradicts (35). �

We illustrate Step 2 of the proof of Proposition 2 with the same gamethat we used above to illustrate the proof of Proposition 1. In that game,player 1’s action A is not only not strictly dominated, but also not weaklydominated. We illustrate in Figure 3 the construction of the sets X and Yto which the proof of Proposition 2 refers.

Y

-3 -2 -1 1 2 3

-3-2

-11

23

ui(ai, L)

ui(ai, R)

X

Figure 3

11

The set Y is now the cone generated by the set that we had called Yin Figure 1. In Figure 3 this cone is indicated by parallel lines from thetop left to the bottom right. The set X is the unbroken line in the bottomleft corner of the shaded rectangle that represented X in Figure 1. Thehyperplane separating X and Y is the dashed line in Figure 3. Figure 3 alsoshows the orthogonal vector for this hyperplane.

4. Iterated Elimination of Strictly Dominated Actions

If players know each others’ utility functions, and know that other play-ers are rational, and know that other players know that players are rational,etc.,5 it seems plausible to iterate the elimination of strictly dominated ac-tions.

Definition 6. A finite sequence ((Xti )i∈N )Tt=0 (where T ∈ N) is a process of

iterated elimination of strictly dominated actions (IESDA) if for all i ∈ N :

(i) X0i = Ai

c and for all i ∈ N and t = 0, 1, . . . , T − 1:

(ii) Xt+1i ⊆ Xt

i

(iii) ai ∈ Xti \X

t+1i only if ai is strictly dominated in the strategic game

with player set N , action sets Xtj for every j ∈ N , and utility func-

tions equal the restrictions of uk to ×j∈NXtj.

(iv) if ai ∈ XTi then ai is not strictly dominated in the strategic game with

player set N , action sets XTj for every j ∈ N , and utility functions

equal to the restrictions of uk to ×j∈NXTj .

Note that there are multiple processes of IESDA. However, as we showbelow, they all terminate with the same sets of actions, and therefore thephrase “actions that survive IESDA” is unambiguously defined. We firstneed a preliminary lemma.

Lemma 1. Let ((Xtj)j∈N )Tt=0 be a process of IESDA. Suppose ai ∈ XT

i is a

best reply to some belief µi ∈ ∆(XT−i) among all actions in XT

i . Then ai isalso a best response to µi among all actions in Ai.

Proof. Let ai be any best response to µi ∈ ∆(XT−i) among all actions in

Ai. Then ai can not be eliminated by IESDA in any step t, because ofthe equivalence of not being strictly dominated and being a best responseto some belief, because µi having support in ∆(XT

−i) implies that µi has

support in ∆(Xt−i), and because ai is a best response to µi among all actions

in Xti . Therefore, ai ∈ XT

i . If ai is a best response to µi in XTi , it must

therefore yield at least as high expected utility, and hence equilibrium the

5I.e. utility functions and rationality are common knowledge.

12

same expected utility, as ai. Therefore, it is also a best response to µi amongall actions in Ai. �

Proposition 3. Let ((Xti )j∈N )Tt=0 and ((Xt

i )j∈N )Tt=0 both be processes of

IESDA. Then XTi = X T

i for all i ∈ N .

Proof. It suffices to prove:

(39) X Ti ⊆ Xt

i ∀t = 0, 1, . . . , T.

This is sufficient because it implies X Ti ⊆ XT

i , and, by the symmetric ar-

gument, we can also infer XTi ⊆ X T

i , so that the assertion follows. Weprove (39) by induction over t. The assertion is trivially true for t = 0. For

the induction step we have to prove that X Ti ⊆ Xt

i for all i ∈ N implies

X Ti ⊆ Xt+1

i for all i ∈ N . Consider some i ∈ N and some ai ∈ X Ti . We

have to show ai ∈ Xt+1i . Because ai is not strictly dominated in the game

left over at the end of the proessss ((Xti )i∈N )Tt=0, it is a best response among

all actions in Ai to a belief µi ∈ ∆(X T−i) (using the above Lemma). By

the inductive assumption, µi’s support is included in Xt−i. Because ai is a

best response to µi among all actions in Ai, it is also a best response to µiamong the actions in in Xt

i . Therefore, it cannot be eliminated in step t of

the process ((Xti )i∈N )Tt=0, and therefore ai ∈ Xt+1

i . �

Adapting Definition 6 appropriately, one might also define the iteratedelimination of weakly dominated actions. The analog of Proposition 3 is,however, not correct when iterated elimination of weakly dominated actionsis considered. For this and other reasons iterated elimination of weaklydominated actions is a problematic solution concept that we do not considerany further in these notes.

5. Other Elimination Processes: Iterated Elimination of WeaklyDominated Actions; Rationalizability

Instead of iterating the elimination of strictly dominated actions, we mightfeel tempted to iterate the elimination of weakly dominated actions. Iter-ated elimination of weakly dominated strategies is, however, a problematicprocedure. A first sign that there is a problem with this procedure is thefact that the order in which we eliminate weakly dominated strategies mightmatter for the result, i.e. Proposition j3 does not apply to iterated elimina-tion of weakly dominated strategies. The simplest example that illustratesthis is as follows.

13

L R

T 1,1 1,1

B 1,1 0,0

Example 2

In this example, if we first eliminate player 2’s weakly dominated actionR, but none of player 1’s actions, then we cannot eliminate any furtheractions in subsequent rounds, and hence both T and B are left over forplayer 1. But if we first eliminate player 1’s weakly dominated action B,and none of player 2’s actions, then we can again not eliminate any furtheractions in subsequent rounds, and both L and R are left over for player 2.Thus, our conclusion depends on the order of elimination.

The deeper reason for this problem is that unlike IESDA, the iteratedelimination of weakly dominated actions is not based on a common knowl-edge assumption. Weakly dominated strategies can only be eliminated ifa player attaches positive probability to all strategy combinations of theother players, but then the player can not know any statement that hasnon-trivial. implications for the other players’ action choice.

There is one other iterated elimination procedure that we should men-tion. Suppose we assume that every player believes that the other players’actions are “independent.” Formally, suppose we require that any player i’sbelief about two other players j’s and k’s actions is the product of the twomarginal measures on player j’s and player k’s action sets, then Proposition1 is no longer true in all games. There are games in which more strate-gies can be eliminated if we require that beliefs are the products of theirmarginals than if we allow beliefs to reflect arbitrary correlations. Obvi-ously, in such games there must be at least 3 players. There is no simplecharacterization in terms of some “dominance” notion of the property ofbeing a best response to a product belief. However, the procedure of iter-ated elimination of actions that are not best responses to product beliefs iswell-defined. Moreover, it can be shown (easily) that the outcome of thisprocedure is order-independent, unlike the outcome of iterated deletion ofweakly dominated strategies. The actions that are left at the end of thisprocedure are called “rationalizable.” We omit here a further discussion ofrationalizability.

6. Nash Equilibrium

We now come to the most classic solution concept for strategic games.

14

Definition 7. A list of mixed actions α ∈ ×i∈N∆(Ai) is a Nash equilibriumif for every player i ∈ N :

(40) αi ∈ arg maxα′i∈∆(ai)

Ui(α′i, α−i)

It is useful to re-write the definition of Nash equilibrium in the followingway. For every player i and for every α−i ∈ ×j∈N,j 6=i∆(aj) define the set ofbest replies of player i:

(41) Bi(α−i) = arg maxα′i∈∆(Ai)

Ui(α′i, α−i).

For every α ∈ ×j∈N∆(Aj) define:

(42) B(α) = ×i∈NBi(α−i).

A Nash equilibrium is then a fixed point of the correspondence B, that is,an α such that α ∈ B(α).

The following result gives a characterization of best responses that isuseful when determining Nash equilibria.

Lemma 2. For every player i and every α−i we have αi ∈ Bi(α−i) if andonly if αi(ai) > 0 implies ai ∈ arg maxa′i∈Ai

Ui(a′i, α−i).

Proof. The same argument that leads to (5) in Step 1 of the proof of Propo-sition 1 shows that for every αi ∈ ∆(Ai):

(43) Ui(αi, α−i) =∑ai∈Ai

αi(ai)

∑a−i∈A−i

ui(ai, a−i)∏

j∈N ;j 6=iαj(aj)

.

This implies:

(44) Ui(α) ≤ maxai∈Ai:αi(ai)>0

Ui(ai, α−i).

which in turn implies:

(45) Ui(α) ≤ maxai∈Ai

Ui(ai, α−i).

Therefore:

(46) maxαi∈∆i(Ai)

Ui(αi, α−i) ≤ maxai∈Ai

Ui(ai, α−i).

On the other hand, because ai ∈ ∆(Ai) for all ai ∈ Ai:

(47) maxαi∈∆(ai)

Ui(αi, α−i) ≥ maxai∈Ai

Ui(ai, α−i).

Therefore,

(48) maxαi∈∆(ai)

Ui(ai, α−i) = maxai∈Ai

Ui(ai, α−i).

15

which means that αi is a best response to α−i if and only if

(49) Ui(α) = maxai∈Ai

Ui(ai, α−i).

The proof is completed by observing that (43) shows that whenever ai ∈arg maxa′i∈Ai

Ui(a′i, α−i) for all αi(ai) > 0 equation (49) holds. By con-

trast, if there is some ai /∈ arg maxa′i∈AiUi(a

′i, α−i) with αi(ai) > 0 then

Ui(αi, α−i) < maxai∈Ai;αi(ai)>0 Ui(ai, α−i). �

We now prove a preliminary result that then allows us to prove the exis-tence theorem for Nash equilibria.

Lemma 3. For every player i and every α−i the set Bi(α−i) is non-emptyand convex.

Proof. Bi(α−i) is non-empty because among the finitely many elements ofAi at least one must be a best response among all elements of Ai to α−iin Ai, and by Lemma 2 this action is contained in Bi(α−i). To see thatBi(α−i) is convex, assume αi, α

′i ∈ Bi(α−i) and 0 < λ < 1. We have to

show: αλi = λαi + (1 − λ)α′i ∈ Bi(α−i). By Lemma 2 this means that wehave to show αλi (ai) > 0 implies that ai ∈ arg maxa′i∈Ai

Ui(a′i, α−i). But

αλi (ai) > 0 implies that either αi(ai) > 0, or α′i(ai) > 0, or both. Becauseαi and α−i are best responses to α−i, we can apply Lemma 2 to at least oneof these mixed actions, and obtain directly what we have to show. �

Proposition 4. Every strategic game has at least one Nash equilibrium.

Proof. It suffices to show that B satisfies the conditions of Kakutani’s fixedpoint theorem (Lemma 20.1 in Osborne and Rubinstein [1]), which guar-antees the existence of at least one fixed point. For this we have to verifythat the domain of B is compact and convex, that B(α) is non-empty andconvex for every α, and that B has a closed graph. That the conditions forthe domain are satisfied is obvious. Non-emptiness and convexity of B(α)for every α follow from Lemma 3. That B has a closed graph follows fromthe maximum theorem (Theorem 9.14 in Sundaram [4]). �

Finally, we relate Nash equilibrium actions to actions that survive IESDA.

Proposition 5. Suppose that α is a Nash equilibrium, and that for somei ∈ N and ai ∈ Ai we have: αi(ai) > 0. Then ai survives IESDA.

Proof. Let ((Xti )j∈N )Tt=0 be a process of IESDA. We shall show that ai ∈ Xt

ifor all t = 0, 1, . . . , T . We prove this by induction over t. The assertion isobvious for t = 0. Now suppose we had shown the assertion for t. We want

16

to prove it for t + 1. By Proposition 1 it is sufficient to show that ai is abest response to some belief µi over ×j∈N,j 6=iXt

j . Define this belief by:

(50) µi(a−i) =∏

j∈N,n6=iαj(aj).

This is a probability measure over ×j∈N,j 6=iXtj because, by the inductive

assumption, αj(aj) > 0 implies aj ∈ Xtj . Moreover, ai is a best response

because by assumption α is a Nash equilibrium, and therefore, by Lemma 2every pure action that is contained in the support of αi is a best responseto α−i. �

One can also easily show that every action that is played with positiveprobability in a Nash equilibrium is rationalizable.

7. Correlated Equilibrium

The following equilibrium concept, “correlated equilibrium” is interme-diate between Nash equilibrium and IESDA. It is less commonly used inapplications. Yet, when one considers the epistemic or evolutionary foun-dations of game theoretic solution concepts, we find that there seem strongarguments in favor of correlated equilibrium.

We begin with some notation. Let γ be some probability distribution overA. We shall denote by γi ∈ ∆(Ai) the marginal distribution of ai, that is,for every ai ∈ Ai:

(51) γi(ai) =∑

a−i∈A−i

γ(ai, a−i).

Consider any action ai ∈ Ai with γi(ai) > 0. We shall denote by γ(a−i|ai)the conditional distribution of a−i, conditional on ai. That is, for everya−i ∈ A−i we have:

(52) γi(a−i|ai) =γ(ai, a−i)

γi(ai).

Finally, for every arbitrary action a′i ∈ Ai we shall denote by Ui(a′i|ai) the

expected utility of player i if player i chooses action ai and the other players’actions are distributed according to γ(·|ai), that is:

(53) Ui(a′i|ai) =

∑a−i∈A−i

ui(a′i, a−i)γ(a−i|ai).

Equipped with this notation, we can now state the following definition.

Definition 8. A correlated equilibrium is a probability distribution γ overA such that for every player i, for every action ai ∈ Ai such that γi(ai) > 0we have:

(54) Ui(ai|ai) ≥ Ui(a′i|ai) for all a′i ∈ Ai.

17

To understand this definition intuitively, imagine a mediator who recom-mends to players which action to choose. The mediator proceeds as follows.First, she chooses an action profile a ∈ A according to some probabilitydistribution γ. She does not reveal a completely to all players. Rather, sheonly reveals for each player i which action ai she has drawn for that agent i.Think of this action ai as the recommendation that the mediator makes toplayer i. If player i knows the distribution γ, and trusts that the mediator istruthful, then the inequality in Definition 8 says that agent i, when receivingthe recommendation to play ai, maximizes her expected utility by followingthe mediator’s recommendation. Here, when calculating her expected util-ity, she infers whatever she can about the distribution of the other player’actions a−i from the fact that she has been recommended to play ai.

Let us illustrate the concept of correlated equilibrium with the followingexample:

S T

S 6,6 2,7

T 7,2 0,0

Example 3

Let γ be the distribution that assign probability 1/3 each to the strategycombinations (S, S), (S, T ), and (T, S), but probability 0 to the strategycombination (T, T ). Then γ is a correlated equilibrium. Consider player1. When he receives the recommendation to play S, the distribution γ(·|S)assigns probability 1/2 each to the actions S and T of player 2. Given thisdistribution, player 1’s expected utility from playing S is (6 + 2)/2 = 4, andher expected utility from playing T is: (7 + 0)/2 = 3.5. Thus, she finds itin her interest to follow the recommendation. When player 1 receives therecommendation to play T , the distribution γ(·|T ) assigns probability 1 tothe action S of player 2. Therefore, player 1’s expected utility from playing Sis 6, but her expected utility from playing T is 7. Therefore, again, she findsthat her expected utility is maximized if she follows the recommendation.

The game in Example 3 is known as the game of “Chicken,” where Sstands for acting “softly.” and T stands for acting “tough.” The game hasthree Nash equilibria: (S, T ), (T, S), and the mixed strategy equilibriumin which every player chooses S with probability 2/3 and T with probabil-ity 1/3. In the mixed strategy Nash equilibrium the undesirable outcomethat both players choose T occurs with probability 1/9. By contrast, thecorrelated equilibrium completely avoids this outcome.

We first note that every Nash equilibrium in pure or mixed strategies isa correlated equilibrium in the following sense:

18

Proposition 6. If α is a Nash equilibrium, then the distribution γ givenby:

(55) γ(a) = α1(a1) · α2(a1) · . . . · αn(an) for all a ∈ A

is a correlated equilibrium.

Proof. When γ is defined as described in Proposition 6, then obviously γi =αi for every player i, and therefore Ui(·|ai) = Ui(·, α−i) for every ai ∈ Ai.But it then follows directly from Lemma 2 that every player i maximizesconditional expected utility when recommended action ai, as long as γi(ai) >0. �

The next result describes easily derived properties of the set of all corre-lated equilibria.

Proposition 7. For every strategic game the set of correlated equilibria isa non-empty and convex set.

Proof. The set of correlated equilibria is non-empty because, by Proposition6, every Nash equilibrium is a correlated equilibrium, and by Proposition 4,every strategic game has a correlated equilibrium. It remains to show thatthe set of correlated equilibria is convex, that is: if γ and γ′ are correlatedequilibria, and if λ ∈ (0, 1), then also λγ + (1− λ)γ′ is a correlated equilib-rium. What we have to show is that for every player i, and for every ai ∈ Aifor which λγi(ai) + (1− λ)γ′i(ai) > 0 we have:∑

a−i∈A−i

ui(ai, a−i)λγ(ai, a−i) + (1− λ)γ′(ai, a−i)

λγi(ai) + (1− λ)γ′i(ai)≥

∑a−i∈A−i

ui(a′i, a−i)

λγ(ai, a−i) + (1− λ)γ′(ai, a−i)

λγi(ai) + (1− λ)γ′i(ai)(56)

for all a′i ∈ Ai. Multiplying both sides by λγi(ai) + (1 − λ)γ′i(ai), we canre-write what we have to show as:∑

a−i∈A−i

ui(ai, a−i)(λγ(ai, a−i) + (1− λ)γ′(ai, a−i)

)≥

∑a−i∈A−i

ui(a′i, a−i)

(λγ(ai, a−i) + (1− λ)γ′(ai, a−i)

).(57)

This can be re-written as:

λ∑

a−i∈A−i

ui(ai, a−i)γ(ai, a−i) + (1− λ)∑

a−i∈A−i

ui(ai, a−i)γ′(ai, a−i) ≥

λ∑

a−i∈A−i

ui(a′i, a−i)γ(ai, a−i) + (1− λ)

∑a−i∈A−i

ui(a′i, a−i)γ

′(ai, a−i).(58)

19

This follows from:∑a−i∈A−i

ui(ai, a−i)γ(ai, a−i) ≥∑

a−i∈A−i

ui(a′i, a−i)γ(ai, a−i) and

∑a−i∈A−i

ui(ai, a−i)γ′(ai, a−i) ≥

∑a−i∈A−i

ui(a′i, a−i)γ

′(ai, a−i).(59)

The first inequality is trivially true if γi(ai) = 0, because then both sidesof the inequality equal zero. But if γi(ai) > 0, then we can divide the firstinequality by γi(ai), and obtain:∑

a−i∈A−i

ui(ai, a−i)γ(ai, a−i)

γi(ai)≥

∑a−i∈A−i

ui(a′i, a−i)

γ(ai, a−i)

γi(ai),(60)

which is true because γ is a correlated equilibrium. The same argumentproves the second inequality. �

We conclude this section by proving that Proposition 5 extends to corre-lated equilibria.

Proposition 8. Suppose that γ is a correlated equilibrium, and that forsome i ∈ N and ai ∈ Ai we have: γi(ai) > 0. Then ai survives IESDA.

Proof. Let ((Xti )j∈N )Tt=0 be a process of IESDA. We shall show that ai ∈ Xt

ifor all t = 0, 1, . . . , T . We prove this by induction over t. The assertionis obvious for t = 0. Now suppose we had shown the assertion for t. Wewant to prove it for t + 1. By Proposition 1 it is sufficient to show thatai is a best response to some belief µi over ×j∈N,j 6=iXt

j . Define this belief

by: µi = γ(·|ai). This is a probability measure over ×j∈N,j 6=iXtj because

γ(aj |ai) > 0 implies γj(aj) > 0, and therefore by the inductive assumptionaj ∈ Xt

j . Moreover, ai is a best response to µi by the definition of correlatedequilibrium. �

Propositions 6 and 8 show that the concept of correlated equilibrium isintermediate between Nash equilibrium and IESDA.

We now calculate as an illustration for the game in Example 3 the set ofall correlated equilibria. We shall denote probability distributions over A inthat example by (p, q, r, s) ∈ R+ where p + q + r + s = 1, and where theinterpretation of the probabilities is indicated in the following table:

S T

S p q

T r s

20

When is a vector (p, q, r, s) a correlated equilibrium? Consider the conditionthat ensures that player 1, when the mediator recommends that he play S,finds it indeed in his interest to play S. Conditional on the recommendationS is expected utility maximizing if:

6p+ 2q ≥ 7p(61)

which is equivalent to p ≤ 2q. This is the first of four conditions that arenecessary and equivalent for (p, q, r, s) to be a correlated equilibrium. Welist all conditions below, and leave it to the reader to derive the other threeconditions.

p ≤ 2q(62)

p ≤ 2r(63)

s ≤ 0.5q(64)

s ≤ 0.5r(65)

The set of correlated equilibria is convex, as we saw above, and therefore,it is of interest to determine its extreme points. In fact, by the Krein-Milman theorem, the set of correlated equilibria is the convex hull of itsextreme points. The following result describes the extreme points.

Claim: The set of correlated equilibria of the game of Chicken has fiveextreme points:

(0, 1, 0, 0), (0, 0, 1, 0),

(4

9,2

9,2

9,1

9

),

(1

2,1

4,1

4, 0

), and

(0,

2

5,2

5,1

5

).

Observe that the first two extreme points are the two pure strategy Nashequilibrium, and that the third extreme point is the mixed strategy Nashequilibrium of Chicken. Because the set of correlated equilibria is the convexhull of the five correlated equilibria listed in the claim, one can implementany arbitrary correlated equilibrium by writing it as a convex combinationof the extreme points, picking one of the “extreme” correlated equilibria atrandom, with the probabilities of each equilibrium determined by its weightin the convex combination, then announcing which of the extreme equilibriahas been picked, and then giving each player an action recommendationwhere the recommendations going to both players are jointly determined bya random draw according to the chosen “extreme” correlated equilibrium.

Proof: We first show that no other points are extreme points of the setof correlated equilibrium, and then we show that these five points are allextreme points.

Suppose first q + r = 1. All such probability vectors are correlated equi-libria. We argue that in an extreme point either q or r must be zero. This

21

is because, if both q and r are positive, then one can find an ε > 0 such that(0, q+ε, r−ε, 0) and (0, q−ε, r+ε, 0) are also correlated equilibria, and be-cause the given correlated equilibrium is the convex combination with equalweights of the two new correlated equilibrium, we don’t have an extremepoint.

Consider next q + r < 1. We begin by noting that in any correlatedequilibrium we then must have that both q > 0 and r > 0. To see thissuppose w.l.o.g. q = 0. Then it cannot be that player 2 is recommended toplay T with positive probability, because she would then know that player1 has been recommended T as well, and she would have an incentive todeviate. So player 2 plays S with probability 1. But then it must be thatplayer 1 plays T with probability 1. She would otherwise always deviate toT . Therefore, it must be that the Nash equilibrium (T, S) is played withprobability 1, which contradicts q + r < 1. We conclude that we must haveq > 0 and r > 0.

Now suppose q + r < 1, q > 0, r > 0, p > 0 and s > 0. This canbe an extreme point only if q = r. To see this, suppose w.l.o.g. q < r.Then, for sufficiently small ε > 0, both (p− 2ε, q − ε, r+ 3.5ε, s− 0.5ε) and(p + 2ε, q + ε, r − 3.5ε, s + 0.5ε) are correlated equilibria, and the originalcorrelated equilibrium is the convex combination with equal weights of thesetwo new correlated equilibria, and therefore cannot be an extreme point. Wealso note that we must have p = 2q, because, otherwise, for sufficiently smallε, both (p−1.5ε, q+ε, r+ε, s−0.5ε) and (p+1.5ε, q−ε, r−ε, s+05ε) wouldbe correlated equilibria, and we could not have an extreme point. Similarly,we must have s = 0.5q. From q = r, p = 2q and s = 0.5q, we conclude thatthe only candidate for an extreme point of the set of correlated equilibriumthat satisfies q + r < 1, q > 0, r > 0, p > 0 and s > 0 is: (4

9 ,29 ,

29 ,

19).

Suppose next q + r < 1, q > 0, r > 0, p = 1 − q − r, and s = 0. Usingarguments as in the previous paragraph one shows that we must have q = r,and also p = 2q. But the system q = r, p = 2q, and q + r + p = 1 has justone solution: (1

2 ,14 ,

14 , 0).

Now suppose q+ r < 1, q > 0, r > 0, p = 0, and s = 1− q− r. By similararguments as in the previous paragraph we then have an extreme point onlyif q = r and s = 0.5q. This leads to the solution: (0, 2

5 ,25 ,

15).

We conclude that five points listed in the claim are the only candidatesfor extreme points of the set of correlated equilibria. It remains to showthat all of them are indeed extreme points. But if one of them were notan extreme point, then by the Krein-Milman theorem, it would have to bepossible to write one of them as a convex combination of the other points.That this is impossible follows from straightforward arguments that we omit.

Q.E.D.

22

We can now determine the set of payoff vectors that correspond to corre-lated equilibria. The five payoff vectors that correspond to the five extremepoints of the set of correlated equilibria are:

(2, 7), (7, 2),

(14

3,14

3

),

(21

4,21

4

), and

(18

5,18

5

).

Note that these are the only candidates for extreme points of the set of payoffvectors that correspond to correlated equilibria, but that not all of these needto be extreme points. Indeed, the mixed strategy Nash equilibrium payoffvector is clearly a convex combination of the last two payoff vectors, andtherefore not an extreme point. The remaining four are all extreme pointsof the set of correlated equilibrium payoff vectors. The set is shown in thefollowing figure. All payoff vectors corresponding to extreme points of theset of correlated equilibria are marked by small circles in this figure.

2 3.6 5.25 7

23.6

5.25

7

u1

u2

Figure 4

We ask next which of the correlated equilibria are on the boundary of theset of correlated equilibria in this example. We say that a vector (p, q, r, s) ison the boundary of the set of correlated equilibria if every open ball around(p, q, r, s) contains a probability vector that is not a correlated equilibrium.Recall that every correlated equilibrium can be represented as a convex com-bination of the five extreme correlated equilibria listed above. Thus, we canrepresent correlated equilibria by indicating their weights (w1, w2, . . . , w5)with wk ≥ 0 for all k and w1 +w2 + . . .+w5 = 1, where w1 is the weight on(0, 1, 0, 0), w2 is the weight on (0, 0, 1, 0),etc. We shall use this representa-tion in the statement of our result about the boundary points of the set ofcorrelated equilibria.

23

Claim: (w1, w2, . . . , w5) is on the boundary of the set of correlated equi-libria if and only if both of the following two conditions hold:

(i) w1w2 = 0 (i.e. at most one of w1 and w2 is non-zero);(ii) w4w5 = 0 (i.e. at most one of w4 and w5 is non-zero).

Proof: One can prove first that a correlated equilibrium is on the bound-ary if and only if there is an action that is recommended with probabilityzero, or that is recommended with positive probability and for which theincentive constraint binds. One can prove then, second, that there is anaction that is recommended with probability zero, or that is recommendedwith positive probability and for which the incentive constraint binds ifand only if (i) and (ii) of the Claim hold. (Details to be filled in later.)

Q.E.D.

References

[1] Martin J. Osborne and Ariel Rubinstein, A Course in Game Theory,Cambridge and London: MIT Press, 1994.

[2] Kim C. Border, Separating Hyperplane Theorems, course notes avail-able at: http://www.hss.caltech.edu/∼kcb/Notes/SeparatingHyper-plane.pdf.

[3] Kim C. Border, Alternative Linear Inequalities, course notes availableat: http://www.hss.caltech.edu/∼kcb/Notes/Alternative.pdf

[4] Rangarajan K. Sundaram, A First Course in Optimization Theory,Cambridge et. al.: Cambridge University Press, 1996.