2003-05-05 CS634 1
Beyond Nash Equilibrium
Correlated Equilibrium and Evolutionary Equilibrium
Jie Bao2003-05-05
Iowa State University
2003-05-05 CS634 2
Equilibrium in Games Pure strategy Nash equilibrium Mixed strategy Nash equilibrium Correlated equilibrium Evolutionary equilibrium Bayesian Nash equilibrium …
NE is too strict on what is “ration”…
2003-05-05 CS634 3
Correlated Equilibrium Aumann 1974 A generalization of “rational” solution In a CE
the action played by any player is a best response (in the expected payoff sense) to the conditional distribution over the other players given that action,
and thus no player has a unilateral incentive to deviate from playing their role in the CE.
Example: Traffic Signal- a single bit of shares information allows a fair split of waiting times.
“running a light” can’t bring greater expected payoff
The actions of players are “correlated”
2003-05-05 CS634 4
Example : BoS
Bach or Stravinsky Game (or Battle of Sex) NE: strategy profile -> payoff profile
(Bach, Stravinsky) ->(2 ,1) (Stravinsky, Bach) ->(1 ,2) (1/3 Bach, 1/3 Bach) -> (2/3, 2/3)
Another equilibrium: the player observe the outcome of a public coin toss, which determines which of the two pure strategy Nash equilibria they play.->(3/2,3/2)
Bach Stravinsky
Bach 2,1 0,1
Stravinsky 1,0 1,2
?
2003-05-05 CS634 5
BoS: Mixed Strategy
(b) Agent 2’s expected utility, blue line is it’s best response to given p (agent 1’s strategy)
(d) Look from the top, (1/3,1/3) is the only mixed NE. pure NEs include (0,1) and (1,0)
(a) Agent 1’s expected utility, red line is it’s best response to given q (agent 2’s strategy)
(c) Overlap of a and b
2003-05-05 CS634 6
BoS: CE - Toss Coin Equ. <N,(Ai),(ui)>= <{1,2} ,{Bach , Stravinsky},
payoff matrix U> Probability space (Ω,π)
Ω = {x,y} π(x) = π(y) = ½
information partition of each agent P1 = P2 = {{x},{y}}
For each player {1,2} action function σi : Ω->Ai, σ1 (x) = σ2 (x) = Bach σ1 (y) = σ2 (y) = Stravinsky
Payoff profile: ½(B,B), ½(S,S) -> (3/2, 3/2) Compared with (for agent 1):
½(S,B), ½(S,S) ->1/2
½(B,B), ½(B,S) ->1
½(S,B), ½(B,S) ->0
2003-05-05 CS634 7
CE: formal definition A strategic game <N,(Ai),(ui)> A finite probability space (Ω,π)
Ωis a set of states and π is a probability measure on π For each player i∈N, a partition Pi of Ω(player i’s information
partition) For each player i∈N, a function σi : Ω->Ai, with σi (w)= σi (w’)
whenever w∈Pi, and w’∈Pi for some P∈Pi, (σi is player i’s strategy) such that for every I∈N and every function τi : Ω->Ai for which
τi(w)=τi(w’) whenever w∈Pi, and w’∈Pi for some P∈Pi (i.e. for every strategy of player i) we have
Note that we assume the players share a common belief about the probabilities with which the states occur.
))(),(()())(),(()( iiiiii uu
2003-05-05 CS634 8
CE contains Mixed NE For every mixed strategy Nash equilibriu
mαof a finite game <N,(Ai),(ui)>, there is a correlated equilibrium <(Ω,π), Pi ,σi > in which for each player i∈N, the distribution on Ai induced by σi is αi.
Construct CE from Mixed NE Ω= A=(X j ∈NAj) – strategy profiles π(α)=Πj ∈N αj(aj) – prob. Of the strategy pr
ofiles Pi(bi)={a ∈A: ai= bi}, Pi consist of the |Ai| se
ts Pi(bi) σi(a)=ai
CE is a more general concept than Mixed NE and Pure NE
NE
CE
Mixed NE
2003-05-05 CS634 9
Convex Combination of CE->CE Let G=<N,(Ai),(ui)> be a strategic
game. Any convex combination of correlated equilibrium payoff of G is a correlated equilibrium payoff of G
Interpret: first a public random device determines which of the K correlated equilibria is to played, and then the random variable corresponding to the kth correlated equilibrium is realized.
•CE: <(Ωk,πk), Pik ,σi
k >
•CE payoff profiles: U1 ,.. Uk
•c1,…ck , all ci>=0, and Σ ci=1
•Construct a new CE• Ω = union of all Ωk
• π(w) = ck πk(w) , if w in Ωk
• Pi=union (on k) of Pik
•σi (w) = σi k
(w)
• Payoff profiles Σ ckUk
2003-05-05 CS634 10
Example
Pure NE payoff profile: (7,2) (2,7) Mixed NE payoff profile: (42/3 ,42/3) CE: Ω={x,y,z}, π(x)= π(y)=
π(z)=1/3, P1={{x},{y,z}}, P2={{x,y},{z}}, σ1(x)=B, σ1(y)= σ1(z)=T, σ2(x)= σ2(y)=L, σ2(z)=R,
-> (5,5) The CE is outside the convex hull of
Pure / Mixed NE payoff profiles
1 2 L R
T 6,6 2,7
B 7,2 0,0
1 2 L R
T y z
B x -
2003-05-05 CS634 11
States and outcomes in CE Let G=<N,(Ai),(ui)> be a strategic game. Every proba
bility distribution over outcomes that can be obtained in a CE of G can be obtained in a CE’ in which
the set of state Ω’ is A and for each i∈N, player i’s information partition Pi’ consis
ts of all sets of the form {a∈Ai: ai=bi} for some action bi∈Ai [π’(a) = πk({w ∈ Ω: σ(w)=a}) σi’ (a) = σi
k (a i)]
This theorem allows us to confine attention to equilibria in which the set states is the set of outcomes
2003-05-05 CS634 12
Notes about CE If players hold different beliefs, additional
equilibrium payoff profiles are possible. Nash equilibrium is a special case of CE in
which we demand that πbe a product distribution for some distribution πi, so every player acts independently of all others
It’s possible to compute CE via linear programming in polynomial time, while NE is exponentially complex!
2003-05-05 CS634 13
Evolutionary Equilibrium ESS: Maynard Smith &
Price, 1972 A steady state in which
all organism take this action and not mutant can invade the population.
Example: the sex ratio in bee population is 1(male):3(female)
Selfish
Selfish
Selfish
SelflessSelfless Selfless
Selfless
Selfless
Selfless
Selfish
2003-05-05 CS634 14
ESS: Definition:
Let G=<{1,2},(B,B),(ui)>be a symmetric strategic game, where u1(a,b)=u2(b,a)=u(a,b) for some function u.
An evolutionarily stable strategy(ESS) of G is an action b*∈B for which (b*,b*) is a NE of G and u(b,b)<u(b*,b) for every best
response b∈B to b* with b≠b*.
2003-05-05 CS634 15
Example: Hawk--Dove
Choose to be Hawk or dove? Pure NE: (D,H) & (H,D) Mixed NE: (0.5D/0.5H, 0.5D/05H) If the players have the freedom to
choose to be hawk or dove in a repeated game, and utility will be used to reproduce their offspring, what’ll be the optimal strategy?
D H
D ½, ½ 0,1
H 1,0 (1-c/)2, (1-c/)2
2003-05-05 CS634 16
HD Game – mixed strategy
(a) Agent 1’s expected utility, red line is it’s best response to given q (agent 2’s strategy)
(b) Agent 2’s expected utility, blue line is it’s best response to given p (agent 1’s strategy)
(c) Overlap of a and b
(d) Look from the top, (0.5,0.5) is the only mixed NE. pure NEs include (0,1) and (1,0)
With c = 2, Action = {'Hawk' , 'Dove'} ;
2003-05-05 CS634 17
HD Game – ESS invading
In all those games, ESS (half-dove-half-hawk mixed strategy (0.5,0.5)) starts from a percentage of 0.1 in the population
Population: 200, Game Round = 1000 Reproduce: proportional to total utility of each type Note that Dove is not completely eliminated
ESS vs. Dove ESS vs. Hawk ESS vs. 1/4 Hawk ESS vs. 1/4 Dove
2003-05-05 CS634 18
HD Game – ESS being invaded
In all those games, ESS (half-dove-half-hawk mixed strategy (0.5,0.5)) starts from a percentage of 0.9 in the population
Setting is same to the last slide ESS can successfully defend the invasion of mutants, although it may
not be a completely expelling The experiment shows that ESS can be taken to be the set of mixed
strategy over some finite set of actions
ESS vs. Dove ESS vs. Hawk ESS vs. 1/4 Hawk ESS vs. 1/4 Dove
2003-05-05 CS634 19
Not all NE are ESS A strict equilibrium b* is an ESS
(b*,b*) if a symmetric NE and no strategy other than b* is a best r
esponse to b* A nonstrict equilibrium may not be an
ESS Mixed NE (1/3, 1/3, 1/3) expected payof
f t/3Can be invaded by any pure strategy
Receives expected utility t/3 when it encounters MixedNE
Receives expected t when it encounters another pure strategy
t,t 1,-1 -1,1
-1,1 t,t 1,-1
1,-1 -1,1 t,t
Example:
ESS
Mixed NE
2003-05-05 CS634 20
More about ESS Widely used in sociobiology
See Dawkins<selfish gene>, chapter 6 Wilson, < sociobiology – New
Synthesis>, chapter 5 And in politic science and sociology
See <the evolution of cooperation>, where tit-for-tat is a ESS in Evolutionary Pensioner Dilemma Game