A Short Introduction to Game Theory

Heiko Hotz

CONTENTS 1

Contents

1 Introduction 21.1 Game Theory – What is it? . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Game Theory – Where is it applied? . . . . . . . . . . . . . . . . . . . . . . 2

2 Definitions 32.1 Normal Form Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.2 Extensive Form Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.3 Nash Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.3.1 Best Response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.3.2 Localizing a Nash Equilibrium in a Payoff-matrix . . . . . . . . . . . 4

2.4 Mixed Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3 Games 53.1 Prisoner’s Dilemma (PD) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3.1.1 Other Interesting Two-person Games . . . . . . . . . . . . . . . . . . 63.2 The Ultimatum Game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.3 Public Good Game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.4 Rock, Paper, Scissors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

4 Evolutionary Game Theory 84.1 Why EGT? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84.2 Evolutionary Stable Strategies . . . . . . . . . . . . . . . . . . . . . . . . . 9

4.2.1 ESS and Nash Equilibrium . . . . . . . . . . . . . . . . . . . . . . . 94.2.2 The Hawk-Dove Game . . . . . . . . . . . . . . . . . . . . . . . . . . 94.2.3 ESS of the Hawk-Dove Game . . . . . . . . . . . . . . . . . . . . . . 10

4.3 The Replicator Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114.4 ESS and Replicator Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . 11

5 Applications 125.1 Evolution of cooperation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125.2 Biodiversity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

A Mathematical Derivation 14A.1 Normal form games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14A.2 Nash equilibrium and best answer . . . . . . . . . . . . . . . . . . . . . . . 14A.3 Evolutionary stable strategies (ESS) . . . . . . . . . . . . . . . . . . . . . . 14A.4 Replicator equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15A.5 Evolutionary stable state of the Hawk-Dove game . . . . . . . . . . . . . . . 15

B Pogram Code 17B.1 Spatial PD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17B.2 Hawk-Dove game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

1 INTRODUCTION 2

1 Introduction

This paper gives a brief overview of gametheory. Therefore in the first section Iwant to outline what game theory gener-ally is and where it is applied.

In the next section, I introduce someof the most important terms used in gametheory, such as normal form games andNash equilibrium as well as some of themost popular games, e.g. the Prisoner’sDilemma or the Ultimatum Game.

I then present the basic concepts ofevolutionary game theory (EGT), a morespecialized branch of game theory. Wewill see how EGT uses new concepts suchas evolutionary stable strategies (ESS) andreplicator dynamics, and its importanceto sciences like biology and physics.

At last I present two examples, pro-grammed in the language NetLogo, whichwill demonstrate the applications of EGTand the similarities to condensed matterphysics.

1.1 Game Theory – What is it?

The concepts of game theory provide acommon language to formulate, structure,analyse and eventually understand differ-ent strategical scenarios. Generally, gametheory investigates conflict situations, theinteraction between the agents and theirdecisions.

A game in the sense of game theoryis given by a (mostly finite) number ofplayers, who interact according to givenrules. Those players might be individu-als, groups, companies, associations andso on. Their interactions will have an im-pact on each of the players and on thewhole group of players, i.e. they are in-terdependent.

To be more precise: A game is de-scribed by a set of players and their possi-bilities to play the game according to therules, i.e. their set of strategies.

This description leads to a widespreddefinition of game theory:

The subject of game theoryare situations, where the re-sult for a player does not onlydepend on his own decisions,but also on the behaviour ofthe other players.

Game theory has its historical originin 1928. By analysing parlour games, Johnvon Neumann realised very quickly thepracticability of his approaches for theanalysis of economic problems.

In his book Theory of Games and Eco-nomic Behavior, which he wrote togetherwith Oskar Morgenstern in 1944, he al-ready applied his mathematical theory toeconomic applications.

The publication of this book is gen-erally seen as the initial point of moderngame theory.

1.2 Game Theory – Where is itapplied?

As we have seen in the previous section,game theory is a branch of mathemat-ics. Mathematics provide a common lan-guage to describe these games. We havealso seen that game theory was alreadyapplied to economics by von Neumann.When there is competition for a resourceto be analysed, game theory can be usedeither to explain existing behaviour or toimprove strategies.

The first is especially applied by sci-ences which analyse long-term situations,like biology or sociology. In animality, forexample, one can find situations, wherecooperation has developed for the sake ofmutual benefits.

The latter is a crucial tool in scienceslike economics. Companies use game the-ory to improve their strategical situationin the market.

2 DEFINITIONS 3

Despite the deep insights he gained fromgame theory’s applications to economics,von Neumann was mostly interested inapplying his methods to politics and war-fare, perhaps descending from his favoritechildhood game, Kriegspiel, a chess-likemilitary simulation. He used his meth-ods to model the Cold War interaction be-tween the U.S. and the USSR, picturingthem as two players in a zero-sum game.

He sketched out a mathematical modelof the conflict from which he deduced thatthe Allies would win, applying some ofthe methods of game theory to his pre-dictions.

There are many more applications inthe sciences, which have already been men-tioned, and in many more sciences like so-ciology, philosophy, psychology and cul-tural anthropology. It is not possible tolist them all in this paper, more informa-tion can be obtained in the references atthe end of this paper.

2 Definitions

I now introduce some of the basic defini-tions of game theory. I use a non-mathematicaldescription as far as possible, since math-ematics is not really required to under-stand the basic concepts of game theory.

However, a mathematical derivation isgiven in appendix A.1 and A.2.

2.1 Normal Form Games

A game in normal form consists of:

1. A finite number of players.

2. A strategy set assigned to each player.(e.g. in the Prisoner’s Dilemma eachplayer has the possibility to cooper-ate (C) and to defect (D). Thus hisstrategy set consists of the elementsC and D.)

3. A payoff function, which assigns acertain payoff to each player depend-ing on his strategy and the strat-egy of the other players (e.g. in thePrisoner’s Dilemma the time each ofthe players has to spend in prison).

The payoff function assigns each playera certain payoff depending on his strat-egy and the strategy of the other play-ers. If the number of players is limitedto two and if their sets of strategies con-sist of only a few elements, the outcomeof the payoff function can be representedin a matrix, the so-called payoff matrix,which shows the two players, their strate-gies and their payoffs.

Example:

Player1\Player2 L RU 1, 3 2, 4D 1, 0 3, 3

In this example, player 1 (vertical) hastwo different strategies: Up (U) and Down(D). Player 2 (horizontal) also has twodifferent strategies, namely Left (L) andRight (R).

The elements of the matrix are theoutcomes for the two players for playingcertain strategies, i.e. supposing, player 1chooses strategy U and player 2 choosesstrategy R, the outcome is (2, 4), i.e. thepayoff for player 1 is 2 and for player 2 is4.

2.2 Extensive Form Games

Contrary to the normal form game, therules of an extensive form game are de-scribed such that the agents of the gameexecute their moves consecutively.

This game is represented by a gametree, where each node represents everypossible stage of the game as it is played.There is a unique node called the initial

2 DEFINITIONS 4

node that represents the start of the game.Any node that has only one edge con-nected to it is a terminal node and rep-resents the end of the game (and also astrategy profile). Every non-terminal nodebelongs to a player in the sense that itrepresents a stage in the game in which itis that player’s move. Every edge repre-sents a possible action that can be takenby a player. Every terminal node hasa payoff for every player associated withit. These are the payoffs for every playerif the combination of actions required toreach that terminal node are actually played.

Example:

Figure 1: A game in extensive form

In figure 1 the payoff for player 1 will be2 and for player 2 will be 1, provided thatplayer 1 plays strategy U and player 2plays strategy D’.

2.3 Nash Equilibrium

In game theory, the Nash equilibrium (namedafter John Nash, who first described it)is a kind of solution concepts of a gameinvolving two or more players, where noplayer has anything to gain by changingonly his own strategy.

If each player has chosen a strategyand no player can benefit by changinghis strategy while the other players keeptheirs unchanged, then the current set of

strategy choices and the corresponding pay-offs constitute a Nash equilibrium.

John Nash showed in 1950, that everygame with a finite number of players andfinite number of strategies has at least onemixed strategy Nash equilibrium.

2.3.1 Best Response

The best response is the strategy (or strate-gies) which produces the most favorableimmediate outcome for the current player,taking other players’ strategies as given.

With this definition , we can now de-termine the Nash equilibrium in a normalform game very easily by using the payoffmatrix.

The formal proof that this procedureleads to the desired result is given in ap-pendix A.2.

2.3.2 Localizing a Nash Equilibriumin a Payoff-matrix

Let us use the payoff matrix of the Pris-oner’s Dilemma, which will be introducedin 3.1, to determine the Nash equilibrium:

Player1\Player2 C DC 3, 3 0, 5D 5, 0 1, 1

The procedure is the following: First weconsider the options for player 1 by a givenstrategy of player 2, i.e. we look for thebest answer to a given strategy of player2.

If player 2 plays C, the payoff for player1 for coosing C will be 3, for choosing D itwill be 5, so we highlight his best answer,D:

C DC 3, 3 0, 5D 5 , 0 1, 1

Now we repeat this procedure for the case,

3 GAMES 5

that player 2 plays D (the best answerin this case is D, since C gains payoff 0whereas D gains payoff 1), then we do thesame for player 2 by a given strategy ofplayer 1 and we will get:

C DC 3, 3 0, 5

D 5 , 0 1 , 1

The Nash equilibrium is then determinedby the matrix element, in which both play-ers marked their best answers.

Thus, the strategies that constitutea Nash equilibrium is defection by bothplayers, because if any player changed hisstrategy to C whereas the other one stayswith D, he would get a less payoff.

2.4 Mixed Strategies

Consider the following payoff matrix, whichcorresponds to the game Matchin Pen-nies:

Player1\Player2 Head TailHead 1 , -1 -1 , 1Tail -1 , 1 1 , -1

The best responses are already marked,and it is obvious, that there is no matrixcell, in which both players marked theirbest response.

What do game theorists make of agame without a Nash equilibrium? Theanswer is that there are more ways to playthe game than are represented in the ma-trix. Instead of simply choosing Head orTail, a player can just flip the coin to de-cide what to do. This is an example ofa mixed strategy, which simply means aparticular way of choosing randomly amongthe different strategies.

The mixed strategy equilibrium of thematching pennies game is well known: eachplayer should randomize 50-50 betweenthe two alternatives.

Mixed strategy equilibrium points outan aspect of Nash equilibrium that is of-ten confusing for beginners. Nash equi-librium does not require a positive reasonfor playing the equilibrium strategy. Inmatching pennies, the two players are in-different: they have no positive reason torandomize 50-50 rather than doing some-thing else. However, it is only an equilib-rium if they both happen to randomize50-50. The central thing to keep in mindis that Nash equilibrium does not attemptto explain why players play the way theydo. It merely proposes a way of playingso that no player would have an incentiveto play differently.

If, for example, player 1 chooses toplay Head with a probability of 80 % andplay Tail with a probability of 20 % , thenplayer 2 will eventually anticipate the op-ponent’s strategy. Hence he will play Taileverytime. This will lead to a positivepayoff of 0.8 · 1 + 0.2 · (−1) = 0.6 pergame for player 2, respectively a payoffof −0.6 for player 1. The best payoff onecan do in such a fair zero-sum game is0, and this will be achieved by playing12 Head + 1

2 Tail.Mathematically spoken, a mixed strat-

egy is just a linear combination of thepure strategies.

3 Games

Now I want to present some of the moststudied games of game theory.

3.1 Prisoner’s Dilemma (PD)

We have already used the payoff matrixof the Prisoner’s Dilemma in 2.3.2 to lo-calize the Nash equilibrium of this game,and now I want to tell the story behindthis famous dilemma:

Two suspects are arrested by the police.The police have insufficient evidence for

3 GAMES 6

a conviction, and, having separated bothprisoners, an officer visits each of themto offer the same deal: if one testifies forthe prosecution against the other and theother remains silent, the betrayer goes freeand the silent accomplice receives the full10-year sentence. If both stay silent, thepolice can sentence both prisoners to onlysix months in jail for a minor charge. Ifeach betrays the other, each will receivea two-year sentence. Each prisoner mustmake the choice of whether to betray theother or to remain silent. However, nei-ther prisoner knows for sure what choicethe other prisoner will make. So the ques-tion this dilemma poses is: How will theprisoners act?

We will use the following abbreviations:To testify means to betray the other sus-pect and thus to defect (D), to remainsilent means to cooperate (C) with theother suspect. And for the sake of clarity,we want to use positive numbers in thepayoff matrix.

C DC R=3, R=3 S=0, T=5D T=5, S=0 P=1, P=1

• R is a Reward for mutual coopera-tion. Therefore, if both players co-operate then both receive a rewardof 3 points.

• If one player defects and the othercooperates then one player receivesthe Temptation to defect payoff (5in this case) and the other player(the cooperator) receives the Suckerpayoff (zero in this case).

• If both players defect then they bothreceive the Punishment for mutualdefection payoff (1 in this case).

As we have already seen, the logical movefor both players is defection (D). The dilemmalies herein, that the best result for player1 and player 2 as a group (R = 3 for both)can’t be achieved.

In defining a PD, certain conditions haveto hold. The values we used above, todemonstrate the game, are not the onlyvalues that could have been used, but theydo adhere to the conditions listed below.

Firstly, the order of the payoffs is im-portant. The best a player can do is T(temptation to defect). The worst a playercan do is to get the sucker payoff, S. Ifthe two players cooperate then the rewardfor that mutual cooperation, R, shouldbe better than the punishment for mu-tual defection, P. Therefore, the followingmust hold:

T > R > P > S.

In repeated interactions, another condi-tion it is additionally required:

Players should not be allowed to getout of the dilemma by taking it in turns toexploit each other. Or, to be a little morepedantic, the players should not play thegame so that they end up with half thetime being exploited and the other halfof the time exploiting their opponent. Inother words, an even chance of being ex-ploited or doing the exploiting is not asgood an outcome as both players mutu-ally cooperating. Therefore, the rewardfor mutual cooperation should be greaterthan the average of the payoff for the temp-tation and the sucker. That is, the follow-ing must hold:

R > (S + T )/2

3.1.1 Other Interesting Two-personGames

Depending on the order of R, T, S, andP, we can have different games. Most are

3 GAMES 7

trivial, but two games stand out:

• Chicken (T > R > S > P )

C DC R=2, R=2 S=1, T=3D T=3, S=1 P=0, P=0

Example: Two drivers with something toprove drive at each other on a narrowroad. The first to swerve loses faces amonghis peers (the chicken). If neither swerves,however, the obvious worst case will oc-cur.

• Stag Hunt (R > T > P > S)

C DC R=3, R=3 S=0, T=2D T=2, S=0 P=1, P=1

Example: Two hunters can either jointlyhunt a stag or individually hunt a rabbit.Hunting stags is quite challenging and re-quires mutual cooperation. Both need tostay in position and not be tempted bya running rabbit. Hunting stags is mostbeneficial for society but requires a lot oftrust among its members. The dilemmaexists because you are afraid of the oth-ers’ defection. Thus, it is also called trustdilemma.

3.2 The Ultimatum Game

Imagine you and a friend of yours arewalking down the street, when suddenlya stranger stops you and wants to play agame with you:

He offers you 100 $ and you have toagree on how to split this money. You,as the proposer, make an offer to yourfriend, the responder. If he accepts youroffer, the deal goes ahead. If your friendrejects, neither player gets anything. Thestranger will take back his money and thegame is over.

Obviously, rational responders shouldaccept even the smallest positive offer, sincethe alternative is getting nothing. Pro-posers, therefore, should be able to claimalmost the entire sum. In a large num-ber of human studies, however, conductedwith different incentives in different coun-tries, the majority of proposers offer 40 to50 % of the total sum, and about half ofall responders reject offers below 30 %

3.3 Public Good Game

A group of 4 people are given $ 200 eachto participate in a group investment project.They are told that they could keep anymoney they do not invest. The rules ofthe game are that every $1 invested willyield $ 2, but that these proceeds wouldbe distributed to all group members. Ifevery one invested, each would get $ 400.However, if only one person invested, that“sucker” would take home a mere $ 100.Thus, the assumed Nash equilibrium couldbe the combination of strategies, whereno one invests any money. And we canshow that this is indeed the Nash equilib-rium.

We will not display this game in a pay-off matrix, since each player has a too bigset of strategies (the strategy sn is givenby the amount of money that player nwants to contribute, e.g. s1 = 10 means,that player 1 invests 10 $). Neverthelessthis is a game in normal form and there-fore it has a payoff function for each player.The payoff function for, let’s say, player 1is given by

P =2 · (s1 + s2 + s3 + s4)

4− s1

=2 · (s2 + s3 + s4)

4− 0, 5 · s1

But this means, that every investments1 of player 1 will diminish his payoff.Therefore, a rational player will choose

4 EVOLUTIONARY GAME THEORY 8

the strategy sn = 0, i.e. he will invest nomoney at all. But if everyone plays thisstrategy, no one can benefit by changinghis strategy while the other players keeptheir strategy unchanged. And this is justthe definition of the Nash equilibrium.

The dilemma lies exactly herein, sincethe greatest benefit for the whole grouparises, if everyone contributed all of hismoney.

3.4 Rock, Paper, Scissors

The players simultaneously change theirfists into any of three ”objects”:

• Rock: a clenched fist.

• Paper : all fingers extended, palmfacing downwards, upwards, or side-ways.

• Scissors: forefinger and middle fin-ger extended and separated into a”V” shape.

The objective is to defeat the oppo-nent by selecting a weapon which defeatstheir choice under the following rules:

1. Rock crushes Scissors (rock wins)

2. Scissors cut Paper (scissors win)

3. Paper covers Rock and roughness iscovered (paper wins)

If players choose the same weapon, thegame is a tie and is played again.

This is a classic non-transitive systemwhich involves a community of three com-peting species satisfying a relationship.

Such relationships have been demon-strated in several natural systems.

4 Evolutionary Game The-ory

Since game theory was established as adiscipline for its own, it has been verysuccessful.

However, there have been situations,which could not be properly described bythe language of game theory.

Many people have attempted to usetraditional game theory to analyze eco-nomic or political problems, which typi-cally involve a large population of agentsinteracting. However, traditional gametheory is a “static” theory, which reducesits usefulness in analyzing these very sortsof situations. EGT improves upon tra-ditional game theory by providing a dy-namics describing how the population willchange over time. Therefore a new math-ematical extension has been developed (mainlyby John Maynard Smith in his book Evo-lution and the Theory of Games, 1982),which is called evolutionary game theory(EGT)

I will show in the next section, in whatkinds of situations EGT might be appli-cable and what are the most significantdifferences to game theory.

4.1 Why EGT?

Evolutionary game theory (EGT) stud-ies equilibria of games played by a pop-ulation of players, where the fitness ofthe players derives from the success eachplayer has in playing the game. It pro-vides tools for describing situations wherea number of agents interact and whereagents might change the strategy they fol-low at the end of any particular interac-tion.

So, the questions of EGT are: Whichpopulations are stable? When do the in-dividuals adopt other strategies? Is itpossible for mutants to invade a given pop-ulation?

4 EVOLUTIONARY GAME THEORY 9

Another very prominent application isthe quest for the origins and evolutionof cooperation. The effects of populationstructures on the performance of behav-ioral strategies became apparent only inrecent years and marks the advent of anintriguing link between apparently unre-lated disciplines. EGT in structured pop-ulations reveals critical phase transitionsthat fall into the universality class of di-rected percolation on square lattices.

Together with EGT as an extensionof game theory, new concepts were devel-oped to investigate and to describe thesevery problems. I will now introduce twoof them, which are crucial to describe EGT,evolutionary stable strategies and the repli-cator dynamics. The first one is appliedtu study the stability of populations, thelatter one investigates the adoption of strate-gies.

4.2 Evolutionary Stable Strate-gies

An evolutionary stable strategy (ESS) isa strategy which if adopted by a popula-tion cannot be invaded by any competingalternative strategy. The concept is anequilibrium refinement to the Nash equi-librium.

The definition of an ESS was intro-duced by John Maynard Smith and GeorgeR. Price in 1973 based on W.D. Hamil-ton’s (1967) concept of an unbeatable strat-egy in sex ratios. The idea can be tracedback to Ronald Fisher (1930) and CharlesDarwin (1859).

4.2.1 ESS and Nash Equilibrium

A Nash equilibrium is a strategy in a gamesuch that if all players adopt it, no playerwill benefit by switching to play any al-ternative strategy.

If a player, choosing strategy µ in apopulation where all other players play

strategy σ, receives a payoff of E(µ, σ),then strategy σ is a Nash equilibrium ifE(σ, σ) ≥ E(µ, σ), i.e. σ does just asgood or better playing against σ than anymutant with strategy µ does playing againstσ.

This equilibrium definition allows forthe possibility that strategy µ is a neutralalternative to σ (it scores equally, but notbetter). A Nash equilibrium is presumedto be stable even if µ scores equally, onthe assumption that players do not playµ.

Maynard Smith and Price specify (May-nard Smith & Price, 1973; Maynard Smith1982) two conditions for a strategy σ tobe an ESS. Either

1. E(σ, σ) > E(µ, σ), or

2. E(σ, σ) = E(µ, σ) andE(σ, µ) > E(µ, µ)

must be true for all σ 6= µ.In other words, what this means is

that a strategy σ is an ESS if one of twoconditions holds:

1. σ does better playing against σ thanany mutant does playing against σ,or

2. some mutant does just as well play-ing against σ as σ, but σ does betterplaying against the mutant than themutant does.

A derivation of ESS is given in ap-pendix A.3.

4.2.2 The Hawk-Dove Game

As an example for ESS, we consider theHawk-Dove Game. In this game, two in-dividuals compete for a resource of a fixedvalue V (The value V of the resource cor-responds to an increase in the Darwinianfitness of the individual who obtains theresource). Each individual follows exactlyone of two strategies described below:

4 EVOLUTIONARY GAME THEORY 10

• Hawk: Initiate aggressive behaviour,not stopping until injured or untilone’s opponent backs down.

• Dove: Retreat immediately if one’sopponent initiates aggressive behaviour.

If we assume that

1. whenever two individuals both ini-tiate aggressive behaviour, conflicteventually results and the two indi-viduals are equally likely to be in-jured,

2. the cost of the conflict reduces indi-vidual fitness by some constant valueC,

3. when a hawk meets a dove, the doveimmediately retreats and the hawkobtains the resource, and

4. when two doves meet the resourceis shared equally between them,

the fitness payoffs for the Hawk-Dove gamecan be summarized according to the fol-lowing matrix:

Hawk DoveHawk 1

2(V − C), 12(V − C) V, 0

Dove 0, V V/2, V/2

One can readily confirm that, for the Hawk-Dove game, the strategy Dove is not evo-lutionarily stable because a pure popula-tion of Doves can be invaded by a Hawkmutant. If the value V of the resourceis greater than the cost C of injury (sothat it is worth risking injury in orderto obtain the resource), then the strat-egy Hawk is evolutionarily stable. In thecase where the value of the resource is lessthan the cost of injury, there is no ESSif individuals are restricted to followingpure strategies, although there is an ESSif players may use mixed strategies.

4.2.3 ESS of the Hawk-Dove Game

Clearly, Dove is no stable strategy, sinceV2 = E(D,D) < E(H,D) = V , a popu-lation of doves can be invaded by hawks.Because of E(H,H) = 1

2(V−C) and E(D,H) =0, H is an ESS if V > C. But what ifV < C? Neither H nor D is an ESS.

But we could ask: What would hap-pen to a population of individuals whichare able to play mixed strategies? Maybethere exists a mixed strategy which is evo-lutianary stable.

Consider a population consisting of aspecies, which is able to play a mixedstrategy I, i.e. sometimes Hawk and some-times Dove with probabilities p and 1− prespectively.

For a mixed ESS I to exist the follow-ing must hold:

E(D, I) = E(H, I) = E(I, I)

Suppose that there exists an ESS inwhich H and D, which are played withpositive probability, have different pay-offs. Then it is worthwile for the playerto increase the weight given to the strat-egy with the higher payoff since this willincrease expected utility.

But this means that the original mixedstrategy was not a best response and hencenot part of an ESS, which is a contradic-tion. Therefore, it must be that in anESS all strategies with positive probabil-ity yield the same payoff.

Thus:

E(H, I) = E(D, I)

⇔ p E(H,H) + (1− p)E(H,D) =p E(D,H) + (1− p)E(D,D)

⇔ p2(V − C) + (1− p)V =(1− p)V

2

⇔ p = VC

4 EVOLUTIONARY GAME THEORY 11

Thus a mixed strategy with a proba-bility V/C of playing Hawk and a proba-bility 1 − V/C of playing Dove is evolu-tionary stable, i.e. that it can not be in-vaded by players playing one of the purestrategies Hawk or Dove.

4.3 The Replicator Dynamics

As mentioned before, the main differenceof EGT to game theory is the investiga-tion of dynamic processes. In EGT, weare interested in the dynamics of a pop-ulation, i.e. how the population evolvesover time.

Let us consider now a population consist-ing of n types, and let xi(t) be the fre-quency of type i. Then the state of thepopulation is given by the vector x(t) =x1(t), . . . , xn(t).

We want now to postulate a law ofmotion for x(t). If individuals meet ran-domly and then engage in a symmetricgame with payoff matrix A, then (Ax)i isthe expected payoff for an individual oftype i and xT Ax is the average payoff inthe population state x.

The evolution of x over time is de-scribed by the replicator equation:

xi = xi[(Ax)i − xT Ax] (1)

The replicator equation describes a se-lection process: more succesful strategiesspread in the population.

A derivation of the replicator equationis given in appendix A.4

4.4 ESS and Replicator Dynam-ics

We have seen, that D is no ESS at all andfor V > C, H is an ESS. We have alsoseen, that for V < C the ESS is a mixedESS with a probability of V/C for playingH and with a probability of 1 − V/C forplaying D.

By setting xi = 0, we obtain the evo-lutionary stable states of a population.

A population is said to be in an evo-lutionarily stable state if its genetic com-position is restored by selection after adisturbance, provided the disturbance isnot too large.

If this equation is applied to the Hawk-Dove game, the result will be the follow-ing:

For V > C, the only evolutionary sta-ble state is a population consisting of hawks.For V < C, a mixed population with afraction V/C of hawks and a fraction 1−V/C of doves is evolutionary stable.

This result will be derived in appendixA.5.

At this point, one may see little differ-ence between the two concepts of evolu-tionary game theory. We have confirmedthat, for the Hawk-Dove game and forV > C, the strategy Hawk is the onlyESS. Since this state is also the only sta-ble equilibrium under the replicator dy-namics, the two notions fit together quiteneatly: the only stable equilibrium underthe replicator dynamics occurs when ev-eryone in the population follows the onlyESS. In general, though, the relationshipbetween ESSs and stable states of the repli-cator dynamics is more complex than thisexample suggests.

If only two pure strategies exist, thengiven a (possibly mixed) evolutionarily sta-ble strategy, the corresponding state ofthe population is a stable state under thereplicator dynamics. (If the evolutionar-ily stable strategy is a mixed strategy S,the corresponding state of the populationis the state in which the proportion ofthe population following the first strategyequals the probability assigned to the firststrategy by S, and the remainder followthe second strategy.)

However, this can fail to be true ifmore than two pure strategies exist.

5 APPLICATIONS 12

The connection between ESSs and stablestates under an evolutionary dynamicalmodel is weakened further if we do notmodel the dynamics by the replicator dy-namics.

In 5.1 we use a local interaction modelin which each individual plays the Pris-oner’s Dilemma with his or her neighbors.Nowak and May, using a spatial model inwhich local interactions occur between in-dividuals occupying neighboring nodes ona square lattice, showed that stable pop-ulation states for the Prisoner’s Dilemmadepend upon the specific form of the pay-off matrix.

5 Applications

5.1 Evolution of cooperation

As mentioned before, the evolution of co-operation is a fundamental problem in bi-ology because unselfish, altruistic actionsapparently contradict Darwinian selection.

Nevertheless, cooperation is abundantin nature ranging from microbial interac-tions to human behavior. In particular,cooperation has given rise to major tran-sitions in the history of life. Game theorytogether with its extensions to an evolu-tionary context has become an invaluabletool to address the evolution of coopera-tion. The most prominent mechanisms ofcooperation are direct and indirect reci-procity and spatial structure.

The mechanisms of reciprocity can beinvestigated very well with the Ultima-tum game and also with the Public Goodgame.

But the prime example to investigatespatially structured populations is the Pris-oner’s Dilemma.

Investigations of spatially extended sys-tems have a long tradition in condensedmatter physics. Among the most impor-tant features of spatially extended sys-tems are the emergence of phase transi-

tions. Their analysis can be traced backto the Ising model. The application ofmethods developed in statistical mechan-ics to interactions in spatially structuredpopulations has turned out to be very fruit-ful. Interesting parallels between non equi-librium phase transitions and spatial evo-lutionary game theory have added anotherdimension to the concept of universalityclasses.

We have already seen, that the Nashequilibrium of PD is to defect. But toovercome this dilemma, we consider spa-tially structured populations where indi-viduals interact and compete only withina limited neighborhood. Such limited lo-cal interactions enable cooperators to formclusters and thus individuals along theboundary can outweigh their losses againstdefectors by gains from interactions withinthe cluster. Results for different popula-tion structures in the PD are discussedand related to condensed matter physics.

This problem has been investigated byMartin Nowak (Nature, 359, pp. 826-829, 1992)

I programmed this scenario based onthe investigations of Nowak. The pro-gram is written in NetLogo, the programcode is given in appendix B.

5.2 Biodiversity

One of the central aims of ecology is toidentify mechanisms that maintain bio-diversity. Numerous theoretical modelshave shown that competing species cancoexist if ecological processes such as dis-persal, movement, and interaction occurover small spatial scales. In particular,this may be the case for nontransitive com-munities, that is, those without strict com-petitive hierarchies. The classic non-transitivesystem involves a community of three com-peting species satisfying a relationship sim-ilar to the children’s game rock-paper-scissors,

5 APPLICATIONS 13

where rock crushes scissors, scissors cutspaper, and paper covers rock. Such rela-tionships have been demonstrated in sev-eral natural systems. Some models pre-dict that local interaction and dispersalare sufficient to ensure coexistence of allthree species in such a community, whereasdiversity is lost when ecological processesoccur over larger scales. Kerr et al., testedthese predictions empirically using a non-transitive model community containing threepopulations of Escherichia coli. They foundthat diversity is rapidly lost in our experi-mental community when dispersal and in-teraction occur over relatively large spa-tial scales, whereas all populations coexistwhen ecological processes are localized.

There exist three strains of Escherichiacoli bacteria:

• Type A releases toxic colicin andproduces, for its own protection, animmunity protein.

• Type B produces the immunity pro-tein only.

• Type C produces neither toxin norimmunity.

The production of the toxic colicin andthe immunity protein causes some highercosts. Thus the strain which producesthe toxin colicin is superior to the strainwhich has no immunity protein (A beatsC).

The one with no immunity protein issuperior to the strain with immunity pro-tein, since it has lower costs to reproduce(C beats B)

The same holds for the strain with im-munity protein but no production of thecolicin compared to the strain which pro-duces colicin (B beats A).

In figure 2, one can see, that on astatic plate, which is an environment inwhich dispersal and interaction are pri-marily local, the three strain coexist.

Figure 2: Escherichia coli on a static plate

green: resistant strainred: colicin producing strainblue: sensitive strain

Figure 3: Escherichia coli in a flask

In figure 3, where the strains are heldin a flask, a well-mixed environment inwhich dispersal and interaction are notexclusively local, only the strain, whichproduces the immunity protein only willsurvive.

A MATHEMATICAL DERIVATION 14

A Mathematical Derivation

A.1 Normal form games

A game in normal form consists of:

1. A (finite) number of players M ={a1, . . . , an}

2. A strategy set Si assigned to eachplayer i ∈ M .The combination of all sets of strate-gies S =

∏i∈M Si is called strategy

space.

3. A utility/payoff function ui : S 7→R, assigned to each player i ∈ M .⇒ ∀s ∈ S : ui(s) ∈ R

A.2 Nash equilibrium and bestanswer

Notations:

1. s ∈ S =∏

i∈M Si ;s = (s1, . . . , sM ) ; si ∈ Si

2. s−i := (s1, . . . , si−1, si+1, . . . , sM ) ;(si, s−i) := s ; (s−i, si) := s

3. S−i =∏

j∈M, j 6=i Sj ;Si × S−i = S ; (si, s−i) ∈ Si × S−i

Definition:

A combination of strategies s∗ ∈ S is calleda Nash equilibrium iff:

∀i ∈ M ∀si ∈ Si :ui(s∗) = ui(s∗i , s

∗−i) ≥ ui(si, s

∗−i)

Definition:

A strategy si ∈ Si is called best answerto a combination of strategies s−i ∈ S−i

iff:

∀si ∈ Si : ui(si, s−i) ≥ ui(si, s−i)⇔ ui(si, s−i) = max{ui(si, s−i) ; si ∈ Si}

It is now easy to see, that, if everyplayer chooses his best answer, this will

constitute a Nash equilibrium, since an-other strategy will not lead to an increaseof the payoff, since the best answer al-ready leads to the maximum payoff.

A.3 Evolutionary stable strate-gies (ESS)

In order for a strategy to be evolutionar-ily stable, it must have the property thatif almost every member of the populationfollows it, no mutant (that is, an individ-ual who adopts a novel strategy) can suc-cessfully invade. This idea can be givena precise characterization as follows: LetE(s1, s2) denote the payoff (measured inDarwinian fitness) for an individual fol-lowing strategy s1 against an opponentfollowing strategy s2, and let W (s) denotethe total fitness of an individual follow-ing strategy s; furthermore, suppose thateach individual in the population has aninitial fitness of W0.

We consider now a population consist-ing mainly of individulas following strat-egy σ which shall be an ESS with a smallfrequency p of some mutants playing µ.Then if each indivudal engages in one con-test

W (σ) = W0 + (1− p)E(σ, σ) + pE(σ, µ),W (µ) = W0 + (1− p)E(µ, σ) + pE(µ, µ).

Since σ is evolutionarily stable, thefitness of an individual following σ mustbe greater than the fitness of an individ-ual following µ (otherwise the mutant fol-lowing µ would be able to invade), and soW (σ) > W (µ). Now, as p is very close to0, this requires either that

E(σ, σ) > E(µ, σ)

or that

E(σ, σ) = E(µ, σ) andE(σ, µ) > E(µ, µ).

A MATHEMATICAL DERIVATION 15

A.4 Replicator equation

The fitness of a population is given by(Ax)i and the total fitness of the entirepopulation is given by xT Ax. Thus, therelative fitness of a population is given by

(Ax)i

xT Ax

Let us assume that the proportion of thepopulation following the strategies in thenext generation is related to the propor-tion of the population following the strate-gies current generation according to therule:

xi(t + ∆t) = xi(t)(Ax)i

xT Ax∆t

for xT Ax 6= 0. Thus

xi(t+∆t)−xi(t) = xi(t)(Ax)i − xT Ax

xT Ax∆t

This yields the differential equation for∆t → 0:

xi =xi[(Ax)i − xT Ax]

xT Ax(2)

for i = 1, . . . , n with xi denoting the deriva-tive of xi after time.

The simplified equation

xi = xi[(Ax)i − xT Ax] (3)

has the same trajectories than (2), sinceevery solution x(t) of (2) delivers accord-ing to the time transformation

t(s) =∫ s

so

x(t)T Ax(t)dt

a solution y(s) := x(t(s)) of (3).Equation (3) is called the replicator

equation.

A.5 Evolutionary stable state ofthe Hawk-Dove game

We want to show, that the replicator dy-namics and ESS yield to the same result

for the Hawk-Dove game.

The replicator equation is given by

xi = xi[(Ax)i − xT Ax)]

Let us denote the population of hawks x1

with p, thus the population of doves x2

will be denoted with 1−p. The first termAx gives( p

2(V − C) + V (1− p)V2 (1− p)

)Since Hawk is denoted with x1, we willuse the first component of the vector Ax.

The second term xT Ax delivers

p2

2(V − C) + p V (1− p) +

V

2(1− p)2

Thus:

p = p [p2(V − C) + V (1− p)− p2

2 (V − C)−p V (1− p)− V

2 (1− p)2]

= p [C2 p2 − 1

2(V + C)p + V2 ]

= p [p2 − V +CC p + V

C ]

In order to be a population evolutionarystable we set the changes of the popula-tion per time to zero, so that there is nochange in time. This gives:

p = 0

⇒ p [p2 − V +CC p + V

C ] = 0

This is certainly true for p = 0. This isthe trivial solution. Two other solutionscan be obtained by evaluating the termin the brackets:

p2 − V + C

Cp +

V

C= 0

This gives

p1/2 = V +C2C ±

√V 2+2V C+C2

4C2 − VC

= V +C2C ±

√V 2−2V C+C2

4C2

= V +C2C ± V−C

2C

A MATHEMATICAL DERIVATION 16

Thus:p1 = 1, p2 =

V

C

p1 = 1 is another trivial solution, thusthe only relevant result is p2 = V

C .

B POGRAM CODE 17

B Pogram Code

B.1 Spatial PD

globals [movie_on?]

patches-own [num_D z z_prev score d score_h neighbor_h]

to setup

ca

;;1/3 will be defectors (z=1, red); 2/3 cooperators (z=0; blue)

ask patches [ifelse ((random 3) < 1) [set pcolor red set z 1][set pcolor blue set z 0]]

;;d=delta*(2.0*(random(1.0)-1.0))

ask patches [set d delta * (2.0 * (random-float 1.0) - 1.0)]

set movie_on? false

end

to single-D

ca

ask patches [set pcolor blue set z 0]

ask patches [set d delta * (2.0 * (random-float 1.0) - 1.0)]

ask patch 0 0 [set pcolor red set z 1]

set movie_on? false

end

to go

play-game

update

if (movie_on?) [movie-grab-view]

end

to play-game

ask patches [set z_prev z]

ask patches

[

; num_D = number of neighbors that are defectors

set num_D nsum z

ifelse z = 1

[

; if patch is defector: score is T times number of cooperators (8-num_D)

set score (T * (8 - num_D)) + (P * (num_D + 1))

]

[

; if patch is cooperator: score is 1=R times number of cooperators (+1=itself)

set score (R * (9 - num_D)) + (S * num_D)

]

]

end

to update

; find the neighbor with the highest payoff

ask patches

[

; neighbor_h = the neighbor with the highest score

set neighbor_h max-one-of neighbors [score]

; score_h = highest score

set score_h score-of neighbor_h

ifelse (score_h >= score + d)

[

; if highest score > own score (+d)

ifelse z_prev = 1

[

; if this patch is defector:

; if neighbor with highest score is cooperator: set patch to cooperator (z=0, green)

; else: stay defector

ifelse (z_prev-of neighbor_h) = 0 [set pcolor green set z 0][set pcolor red]

B POGRAM CODE 18

]

[

; if this patch is cooperator:

; if neighbor with highest score is defector: set patch to defector (z=1, yellow)

; else: stay cooperator

ifelse (z_prev-of neighbor_h) = 1 [set pcolor yellow set z 1][set pcolor blue]

]

set d delta * (2.0 * (random-float 1.0) - 1.0)

]

[

; if own score is the highest:

; if cooperator: set color blue, otherwise red

ifelse z = 0 [set pcolor blue][set pcolor red]

]

]

end

to perturb

if mouse-down?

[

ask patch-at mouse-xcor mouse-ycor [set z 1 - z ifelse z = 0 [set pcolor blue][set pcolor red]]

;need to wait for a while; otherwise the procedure is run a few times after a mouse click

wait 0.5

]

end

to movie_start

movie-start "out.mov"

set movie_on? true

end

to movie_stop

movie-close

set movie_on? false

end

B POGRAM CODE 19

B.2 Hawk-Dove game

breed [hawks hawk]

breed [doves dove]

globals [deltaD deltaH p total counter decrement reproduce_limit]

turtles-own [energy]

to setup

ca

set-default-shape hawks "hawk"

set-default-shape doves "butterfly"

createH n_hawks

createD n_doves

ask turtles [set energy random-float 10.0]

set reproduce_limit 11.0

set decrement 0.05

end

to go

ask turtles

[

move

fight

reproduce

]

while [count turtles > 600]

[ask one-of turtles[die]]

do-plot

end

to createH [num_hawks]

create-custom-hawks num_hawks

[

set color red

set size 1.0

setxy random-xcor random-ycor

]

end

to createD [num_doves]

create-custom-doves num_doves

[

set color white

set size 1.0

setxy random-xcor random-ycor

]

end

to fight

ifelse (is-hawk? self)

[

if ((count other-hawks-here = 1) and (count other-doves-here = 0))

[set energy (energy + 0.5 * (V - C))]

if ((count other-hawks-here = 0) and (count other-doves-here = 1))

[set energy (energy + V)]

]

[

if ((count other-hawks-here = 0) and (count other-doves-here = 1))

B POGRAM CODE 20

[set energy (energy + V / 2)]

]

;set pcolor-of patch-here black

end

to move ;; turtle procedure

rt random-float 50 - random-float 50

fd 1

set energy (energy - decrement)

end

to reproduce

if energy > reproduce_limit

[

set energy (energy / 2)

hatch 1 [rt random 360 fd 1]

]

if energy < 0

[die]

end

to do-plot

set-current-plot "ratio"

set-current-plot-pen "ratio"

if any? turtles

[plot count hawks / count turtles]

;

end

References

• Maynard Smith, J. Evolution and the Theory of Games, Cambridge University Press(1982).

• Hauert, C. and Szabo, G. American Journal of Physics, 73, pp. 405-414 (2005).

• Hofbauer, J. and Sigmund, K. Bulletin of the American Mathematical Society, 40,pp. 479-519 (2003).

• Nowak, M. and May, R. Nature, 359, pp. 826-829 (1992).

• Nowak, M., Page, K. and Sigmund, K. Science, 289, pp. 1773-1775 (2000).

• Nowak, M., and Sigmund, K. Science, 303, pp. 793-799 (2004).

• Kerr, B., Riley, M. A., Feldman, M. W. and Bohannan, B. J. M. Nature, 418, pp.171-174 (2000).

• http://cse.stanford.edu/class/sophomore-college/projects-98/game-theory/neumann.html.

• http://plato.stanford.edu/entries/game-evolutionary/.

• http://www.economist.com/printedition/displayStory.cfm?Story ID=1045223.

• http://www.dklevine.com/general/cogsci.htm