deviations and fluctuations for mean-field games · deviations and fluctuations for mean-field...

Deviations and Fluctuations for Mean-Field Games

Kavita Ramanan,Brown University

AMS Short CourseJMM, Denver, Colorado

January 14, 2020

Kavita Ramanan, Brown University AMS Short Course JMM, Denver, Colorado January 14, 2020Mean-Field Games 1 / 45

Outline

I. Interacting Diffusions: Fluctuations

II. From Interacting Diffusions to MFG: Fluctuations

III. Large Deviations for Interacting Diffusions and MFG

IV. Open Problems


I. Interacting Diffusions:Fluctuations


Interacting Diffusions and the McKean-Vlasov limit

Consider n diffusions interacting through their empirical measure:

dX n,it = b(X n,i

t , µnt )dt + dW i

t , µnt =

1n

n∑k=1

δX n,kt,

where {X n,i0 = ξi }i∈N are iid with common law µ0 and (W i)n

i=1 are iidd-dimensional Brownian motions.

Under suitable regularity conditions, X n,i ⇒ X and µnt → µt , where X is

a non-linear Markov (or McKean-Vlasov) process:

dXt = b(Xt , µt)dt + dBt , µt = Law(Xt),

with X0 ∼ µ0 independent of B, a d-dimensional Brownian motion.Alternatively, µ solves the (nonlinear) Fokker-Planck equation

ddt〈µt , ϕ〉 =

⟨µt ,b(·, µt) · ∇ϕ+

12∆ϕ

⟩, ∀ϕ ∈ C∞

c .




dX n,it = b(X n,i

t , µnt )dt + dW i

t , µnt =

1n

n∑k=1

δX n,kt,






with X0 ∼ µ0 independent of B, a d-dimensional Brownian motion.

Alternatively, µ solves the (nonlinear) Fokker-Planck equation

ddt〈µt , ϕ〉 =

⟨µt ,b(·, µt) · ∇ϕ+

12∆ϕ

⟩, ∀ϕ ∈ C∞

c .




dX n,it = b(X n,i

t , µnt )dt + dW i

t , µnt =

1n

n∑k=1

δX n,kt,






with X0 ∼ µ0 independent of B, a d-dimensional Brownian motion.Alternatively, µ solves the (nonlinear) Fokker-Planck equation

ddt〈µt , ϕ〉 =

⟨µt ,b(·, µt) · ∇ϕ+

12∆ϕ

⟩, ∀ϕ ∈ C∞

c .


Rate of Convergence to the McKean-Vlasov limit

dX n,it = b(X n,i

t , µnt )dt + dW i

t , X n,i0 = ξi

µnt =

1n

n∑k=1

δX n,kt

and µn =1n

n∑k=1

δX n,k (·)

1 How can one measure distance between µnt and µt , µn and µ?

2 Some notation:Define Cd = C([0,T ] : Rd ), equipped with the uniform topologyGiven a separable Banach space (E , ‖ · ‖), let P(E) denote thespace of Borel probability measures on ENote that µn

t , µt ∈ P(R) and µn, µ ∈ P(Cd ),Given p ∈ [1,∞), let Pp(E) = {ν ∈ P :

∫E ‖x‖

pν(dx) <∞} equippedwith the p-Wasserstein metric:

Wp,E(ν, ν′) = inf

π

(∫E×E

||x − y ||pπ(dx ,dy))1/p

,

where the infimum is over all probability measures π on E × E withfirst and second marginals µ and ν respectively.



dX n,it = b(X n,i

t , µnt )dt + dW i

t , X n,i0 = ξi

µnt =

1n

n∑k=1

δX n,kt

and µn =1n

n∑k=1

δX n,k (·)


2 Some notation:Define Cd = C([0,T ] : Rd ), equipped with the uniform topology

Given a separable Banach space (E , ‖ · ‖), let P(E) denote thespace of Borel probability measures on ENote that µn


∫E ‖x‖



π

(∫E×E

||x − y ||pπ(dx ,dy))1/p

,




dX n,it = b(X n,i

t , µnt )dt + dW i

t , X n,i0 = ξi

µnt =

1n

n∑k=1

δX n,kt

and µn =1n

n∑k=1

δX n,k (·)



t , µt ∈ P(R) and µn, µ ∈ P(Cd ),

Given p ∈ [1,∞), let Pp(E) = {ν ∈ P :∫

E ‖x‖pν(dx) <∞} equipped

with the p-Wasserstein metric:


π

(∫E×E

||x − y ||pπ(dx ,dy))1/p

,




dX n,it = b(X n,i

t , µnt )dt + dW i

t , X n,i0 = ξi

µnt =

1n

n∑k=1

δX n,kt

and µn =1n

n∑k=1

δX n,k (·)




∫E ‖x‖



π

(∫E×E

||x − y ||pπ(dx ,dy))1/p

,



Fluctuations around the McKean-Vlasov limit

dX n,it = b(X n,i

t , µnt )dt + dW i

t , µnt =

1n

n∑k=1

δX n,kt

andddt〈µt , ϕ〉 =

⟨µt ,b(·, µt) · ∇ϕ+

12∆ϕ

⟩, ∀ϕ ∈ C∞

c .

Recall µ = (µt)t≥0 is the McKean-Vlasov limit.We are interested in the limit of the signed measures capturingfluctuations:

Snt :=

√n(µn

t − µt), t ≥ 0.

Do you expect this sequence to converge to another signed measure?


A Simple ExampleHow bad can the weak limit of a signed measure be?

Let ν ∈ P(R) be a probability measure with a cdf that is notdifferentiable at any point:

F (x) := ν((−∞, x ]), x ∈ R.

For n ∈ N, define νn to be a measure with cdfFn(x) := F (x + n−1/2):

νn((−∞, x ]) = ν((−∞, x + n−1/2]) = F (x + n−1/2),

and define the signed measure

νn :=√

n[νn − ν].

Then, since F is non-differentiable,

limn→∞ νn((−∞, x ]) = lim

n→∞√

n[F(

x +1√n

)− F (x)

]doesn’t exist!


A Simple Example (contd.)For any measure ν and integrable function ϕ, define

〈ϕ,ν〉 =∫ϕ(x)ν(dx)

Recall that νn((−∞, x ]) = ν((−∞, x + n−1/2]), and so

〈ϕ,νn〉 =∫ϕ(x)νn(dx) =

∫ϕ

(x −

1n

)ν(dx) =

⟨ϕ

(·− 1

n

), ν

⟩Recall that νn :=

√n[νn − ν].

Thus, for any infinitely differentiable function ϕ : R 7→ R withcompact support,

So limn→∞〈νn, ϕ〉 = lim

n→∞√

n[〈νn, ϕ〉− 〈ν,ϕ〉]

= limn→∞

√n⟨ν,

[ϕ

(·− 1√

n

)−ϕ(·)

]⟩= −〈ν,ϕ′〉.






∫ϕ

(x −

1n

)ν(dx) =

⟨ϕ

(·− 1

n

), ν

⟩

Recall that νn :=√

n[νn − ν].



n→∞√

n[〈νn, ϕ〉− 〈ν,ϕ〉]

= limn→∞

√n⟨ν,

[ϕ

(·− 1√

n

)−ϕ(·)

]⟩= −〈ν,ϕ′〉.






∫ϕ

(x −

1n

)ν(dx) =

⟨ϕ

(·− 1

n

), ν

⟩Recall that νn :=

√n[νn − ν].



n→∞√

n[〈νn, ϕ〉− 〈ν,ϕ〉]

= limn→∞

√n⟨ν,

[ϕ

(·− 1√

n

)−ϕ(·)

]⟩= −〈ν,ϕ′〉.


A Primer on the Theory of Distributions

ν ↔ ϕ 7→ 〈ν,ϕ〉Let D be a space of test functions, e.g.,

D = {f ∈ C∞ : supp f is compact }

equipped with a suitable topology: ϕn → ϕ in D if there exists acompact set K such that each ϕn and ϕ have support in K and∂αϕn →∂αϕ uniformly on K for all α.

Definition. A mapping ν : D 7→ R is said to be a linear functional if

ν(α1ϕ1 + α2ϕ2) = α1ν(ϕ1) + α2ν(ϕ2), ∀αi ∈ R, ϕi ∈ D, i = 1,2.

Let D′ denote the corresponding space of distributions, defined tobe a linear functional ν : D 7→ R, that also satisfies the followingcontinuity property:

ϕn → ϕ in D ⇒ ν(ϕn) → ν(ϕ) in R, ∀ϕ ∈ D.


A Primer on the Theory of Distributions

ν ↔ ϕ 7→ 〈ν,ϕ〉Let D be a space of test functions, e.g.,

D = {f ∈ C∞ : supp f is compact }

equipped with a suitable topology: ϕn → ϕ in D if there exists acompact set K such that each ϕn and ϕ have support in K and∂αϕn →∂αϕ uniformly on K for all α.

Definition. A mapping ν : D 7→ R is said to be a linear functional if

ν(α1ϕ1 + α2ϕ2) = α1ν(ϕ1) + α2ν(ϕ2), ∀αi ∈ R, ϕi ∈ D, i = 1,2.

Let D′ denote the corresponding space of distributions, defined tobe a linear functional ν : D 7→ R, that also satisfies the followingcontinuity property:

ϕn → ϕ in D ⇒ ν(ϕn) → ν(ϕ) in R, ∀ϕ ∈ D.Kavita Ramanan, Brown University AMS Short Course JMM, Denver, Colorado January 14, 2020Mean-Field Games 9 / 45

A Primer on the Theory of Distributions (contd.)

Examples of Distributions

1 The space of distributions is clearly a vector space on R

2 Given a σ-finite measure ν on R, the linear functionalϕ 7→ 〈ν,ϕ〉 = ∫

ϕ(x)ν(dx) defines a distributionExercise: Prove this by verifying the continuity propertyNote: For general ν ∈ D′, ν(ϕ) is often written as 〈ν,ϕ〉.

3 Given a distribution ν ∈ D′, and a C∞ function ψ, ψν denotes thedistribution

(ψν)(ϕ) = 〈ν,ψϕ〉, ∀ϕ ∈ D

This is clearly a linear functional.Exercise: Verify the continuity property to show ψν is adistribution.



Examples of Distributions

1 The space of distributions is clearly a vector space on R2 Given a σ-finite measure ν on R, the linear functionalϕ 7→ 〈ν,ϕ〉 = ∫

ϕ(x)ν(dx) defines a distributionExercise: Prove this by verifying the continuity propertyNote: For general ν ∈ D′, ν(ϕ) is often written as 〈ν,ϕ〉.

3 Given a distribution ν ∈ D′, and a C∞ function ψ, ψν denotes thedistribution

(ψν)(ϕ) = 〈ν,ψϕ〉, ∀ϕ ∈ D

This is clearly a linear functional.Exercise: Verify the continuity property to show ψν is adistribution.



Exercise: Which of the following maps w : D(R) 7→ R are distributions:here, ϕ(k) is the k th derivative of ϕ

1 〈w , ϕ〉 :=∫R f (x)ϕ(x)dx for a locally integrable function f .

2 〈w , ϕ〉 :=∑∞

k=0ϕ(k).3 〈w , ϕ〉 :=

∑∞k=0ϕ

(k)(√

2),4 〈w , ϕ〉 :=

∫Rϕ

2(x)dx

5 〈w , ϕ〉 :=∫∞

0ϕ(x)−ϕ(−x)

x dx

Exercise: Show that for any k ∈ N, 〈w , ϕ〉 := (−1)k 〈w , ϕ(k)〉 is adistribution.


Convergence of Distributions

Let (ν`) be a sequence in D′ and ν ∈ D′.1 Then (ν`) is said to converge to ν in D′, denoted ν` → ν if

lim`→∞〈ν`, ϕ〉 = 〈ν,ϕ〉.

2 Moreover, (ν`) is Cauchy in D′ if ∀ϕ ∈ D, (〈ν`, ϕ〉) is Cauchy in R.

Returning to the Example:

νn((−∞, x ]) = ν((−∞, x + n−1/2]), νn :=√

n[νn − ν].

We showed that for all ϕ ∈ D,

limn→∞〈νn, ϕ〉 := −〈ν,ϕ′〉,

The linear functional ϕ 7→ −〈ν,ϕ′〉 lies in D′ (by the last exercise).It is in fact denoted by ∂ν and is said to be the derivative of ν.Thus, we showed that

νn → ∂ν in D′.


Convergence of Distributions

Let (ν`) be a sequence in D′ and ν ∈ D′.1 Then (ν`) is said to converge to ν in D′, denoted ν` → ν if

lim`→∞〈ν`, ϕ〉 = 〈ν,ϕ〉.

2 Moreover, (ν`) is Cauchy in D′ if ∀ϕ ∈ D, (〈ν`, ϕ〉) is Cauchy in R.

Returning to the Example:

νn((−∞, x ]) = ν((−∞, x + n−1/2]), νn :=√

n[νn − ν].

We showed that for all ϕ ∈ D,

limn→∞〈νn, ϕ〉 := −〈ν,ϕ′〉,

The linear functional ϕ 7→ −〈ν,ϕ′〉 lies in D′ (by the last exercise).It is in fact denoted by ∂ν and is said to be the derivative of ν.Thus, we showed that

νn → ∂ν in D′.Kavita Ramanan, Brown University AMS Short Course JMM, Denver, Colorado January 14, 2020Mean-Field Games 12 / 45

A Stochastic Example

Let (Bk )k∈N be a sequence of independent 1-dimensional BMswith initial distribution π0.For t > 0, A ∈ B(R), define

Nnt (A) :=

1n

n∑k=1

I{Bkt ∈A}.

Exercise: Calculate limn→∞ Nnt (A).

Soln. Since (I{Bkt ∈A})k∈N are iid Bernoulli random variables, the

SLLN tells us that Nnt (A) → γt(A) almost surely, where γt is a

centered Gaussian distribution with variance t , because

γt(A) = E[I{B1

t ∈A}

]= P(B1

t ∈ A) = P(Bkt ∈ A).

Note: In fact, one can prove convergence in P(Rd): a.s.,

limn→∞ Nn

t (·) → γt


Stochastic Example (contd.)

Nnt (A) =

1n

n∑k=1

I{Bkt ∈A}, E[Nn

t (·)] = γt , A ∈ B(R), t ≥ 0.

Exercise*: Find the limit of the (random) signed measure-valued proc.:

Snt (A) :=

√n (Nn

t (A) − γt(A)) , t ≥ 0,A ∈ B(R).

Soln. View Sn as a distribution-valued process: for t > 0, ϕ ∈ D,

Snt (ϕ) :=

∫Rϕ(x)Sn

t (dx) =√

n[∫

Rϕ(x)Nn

t (dx) −∫

Rϕ(x)γt(dx)

],

is a random variable, in fact it is equal to

Snt (ϕ) = n−1/2

n∑k=1

(ϕ(Bk

t ) − 〈ϕ,γt〉)= n−1/2

n∑k=1

(ϕ(Bk

t ) − E[ϕ(Bkt )])

• Can show ∀ϕ ∈ D, t 7→ Snt (ϕ) is a.s. continuous and that, t 7→ Sn

t (·)is a continuous D′-valued process.


Stochastic Example (contd.)

Snt (A) :=

√n (Nn

t (·) − γt(·)) .

Use the multidimensional CLT to show there exists a centeredGaussian D′-valued process (St(ϕ))t ,ϕ such that every finite-dim.distribution of Sn

t = (Snt (ϕ))t ,ϕ converges to the corresponding

finite-dim. distribution of the solution S = (St(ϕ))t ,ϕ to the S(P)DE

dSt = (∂ ◦√πt)dbt +

12∂2Stdt ,

dSt(ϕ) = (∂ ◦√πt)dbt(ϕ) +

12∂2St(ϕ)dt , ϕ ∈ D,

where{bt } = {bt(ϕ), ϕ ∈ H2} is a standard Wiener H′2-valued process.πt = π0 ∗ γt√πt ∈ C∞(R) is viewed as a multiplication operator in D′

∂ is differentiation in D′

∂ ◦ √πt denotes a composition of these operators.Kavita Ramanan, Brown University AMS Short Course JMM, Denver, Colorado January 14, 2020Mean-Field Games 15 / 45

Recap so far

We started with interacting diffusions:

dX n,it = b(X n,i

t , µnt )dt + dW i

t , µnt =

1n

n∑k=1

δX n,kt

and recalled the McKean-Vlasov limit µ that satisfies:

ddt〈µt , ϕ〉 =

⟨µt ,b(·, µt) · ∇ϕ+

12∆ϕ

⟩, ∀ϕ ∈ C∞

c .

To capture rate of convergence, we wanted to understand the limit of

Snt :=

√n(µn

t − µt), t ≥ 0.


Recap so far (contd.)

dX n,it = b(X n,i

t , µnt )dt + dW i

t , µnt =

1n

n∑k=1

δX n,kt

ddt〈µt , ϕ〉 =

⟨µt ,b(·, µt) · ∇ϕ+

12∆ϕ

⟩, ∀ϕ ∈ C∞

c .

Snt :=

√n(µn

t − µt), t ≥ 0.

To understand the form of potential limits of processes such as Snt , we

1. considered sequences of scaled centered deterministic signedmeasures of a similar form, and showed that their limits are oftendistributions, not (signed) measures;

2. provided a brief introduction to the theory of distributions;


Back to: Fluctuations around the McKean-Vlasov limit

dX n,it = b(X n,i

t , µnt )dt + dW i

t , µnt =

1n

n∑k=1

δX n,kt

ddt〈µt , ϕ〉 =

⟨µt ,b(·, µt) · ∇ϕ+

12∆ϕ

⟩, ∀ϕ ∈ C∞

c .

Snt :=

√n(µn

t − µt), t ≥ 0.

To understand the form of potential limits of processes such as Snt , we

3. considered the simplest stochastic case, namely to study the limitof fluctuations (or CLT – central limit theorems) for non-interactingdiffusions, that is, where b ≡ 0, and characterized the limit as asolution to distribution-valued process, governed by an “SPDE”.

4. We now consider the interacting case, b 6= 0 and, in analogy withthe non-interacting case, will view Sn as a suitabledistribution-valued process.


Some CLT Results for Interacting Particle Systems

References assuming affine dependence of drift on the empiricalmeasure

1 H. Tanaka and M. Hitsuda. Central limit theorem for a simplediffusion model of interacting particles. Hiroshima MathematicalJournal 11 (1981), no. 2, 415–423.

2 A.S. Sznitman. A fluctuation result for nonlinear diffusions.Infinite-dimensional analysis and Stochastic Processes (1985),145–160.

3 S. Méléard. Asymptotic behaviour of some interacting particlesystems: McKean-Vlasov and Boltzman models, Probabilisticmodels for nonlinear partial differential equations, Lecture Notesin Math, vol. 1627, Springer, 1996, pp. 42–95.


Some CLT Results for Interacting Particle Systems

References that consider more general dependence on the empiricalmeasure

1 T.G. Kurtz and J. Xiong. A stochastic evolution equation arisingfrom fluctuations of a class of interacting particle systems,Communications in Mathematical Sciences 2 (2004) no. 3,325–358.Comment: Only in the case where each particle takes values in R– one-dimensional case

2 F. Delarue, D. Lacker and K.R., “From the master equation tomean field game limit theory: a central limit theorem”, Electron. J.Probab., Volume 24 (2019), paper no. 51, 54 pp.Comment 1: This covers more general dependence and particlestaking values in Rd for general d ∈ N.Comment 2: The precise space in which the limit process liesends up depending on the dimension d .


General CLT for Interacting Particle Systems

dX n,it = b(X n,i

t , µnt )dt + dW i

t , µnt =

1n

n∑k=1

δX n,kt

and, with µ the McKean-Vlasov limit,

Snt :=

√n(µn

t − µt), t ≥ 0.

Theorem: Under suitable regularity assumptions on b, Sn convergesweakly to S = (St(ϕ))t ,ϕ in H ′d , where H ′d is a suitable distributionspace with test function space Hd , where S solves the SPDE:

d〈St , ϕ〉 = 〈St ,At ,µtϕ〉dt + dWt(ϕ), ϕ ∈ Hd

where W is a centered H′d -valued continuous centered Gaussianprocess with covariance functional

E[Wt(ϕ1)Ws(ϕ2)] =

∫ s∧t

0〈µr ,Dxϕ1 · Dxϕ2〉dr , ϕ1, ϕ2 ∈ Hd ,

and where At ,µt is some suitable (nonlocal) operator.


General CLT for Interacting Particle Systems

dX n,it = b(X n,i

t , µnt )dt + dW i

t , µnt =

1n

n∑k=1

δX n,kt

and, with µ the McKean-Vlasov limit,

Snt :=

√n(µn

t − µt), t ≥ 0.

Theorem: Under suitable regularity assumptions on b, Sn convergesweakly to S = (St(ϕ))t ,ϕ in H ′d , where H ′d is a suitable distributionspace with test function space Hd , where S solves the SPDE:

d〈St , ϕ〉 = 〈St ,At ,µtϕ〉dt + dWt(ϕ), ϕ ∈ Hd

where W is a centered H′d -valued continuous centered Gaussianprocess with covariance functional

E[Wt(ϕ1)Ws(ϕ2)] =

∫ s∧t

0〈µr ,Dxϕ1 · Dxϕ2〉dr , ϕ1, ϕ2 ∈ Hd ,

and where At ,µt is some suitable (nonlocal) operator.Kavita Ramanan, Brown University AMS Short Course JMM, Denver, Colorado January 14, 2020Mean-Field Games 21 / 45

Interacting Particle Systems

2. From Interacting Diffusionsto MFG:

Fluctuations


Motivation and Context

Multi-agent or Many-player Systems


Symmetric n-player Differential Games

W 1, . . . ,W n ind. d-dim BMs

Polish action space

drift functional b : Rd × Pp(Rd)× 7→ Rd

State Dynamics

dX n,it = b(X n,i

t , µnt , α

i(t ,X t))dt + dW it , µn

t =1n

n∑k=1

δX n,kt

where αi : [0,T ]× (Rd)n → is a Markovian control that is chosen tominimize the i th objective function

Jni (α

1, . . . , αn) = E

[∫T

0f (X i

t , µnt , α

i(t ,X t))dt + g(X iT ,m

nX T

)

],

for suitable cost functionals f and g.


Nash Equilibria

Definition A (closed-loop) Nash equilibrium is defined in the usual wayas a vector of feedback functions or controls (α1, . . . , αn), whereαi : [0,T ]× (Rd)n → are such that the SDE

dX n,it = b(X n,i

t , µnt , α

i(t ,X t))dt + dW it , µn

t =1n

n∑k=1

δX n,kt

is unique in law, and

Jn,i(α1, . . . , αn) ≤ Jn,i(α1, . . . , αi−1, α, αi+1, . . . , αn),

for any alternative choice of feedback control α.


Characterization of n-player Nash equilibriaMain Point (Cardialaguet et al ’15)

A verification theorem tells us that if we have a unique solution{vn,i }i=1,...,n to a coupled system of n PDEs called the Nash systemsuch that vn,i lies in C1,2 for each i = 1, . . . ,n, then the controls

(0,T ]× (Rd)n 3 (t ,x) 7→ αn,i(

x ,mnX ,Dxi v

n,i(t ,x))

form a closed-loop Nash equilibrium, where mnx = 1

n∑n

i=1 δxi

The corresponding Nash equilibrium dynamics, given by

dX n,it = b(X n,i

t , µnt ,Dxi v

n,i(t ,X nt ))dt + dW i

t , µnt =

1n

n∑k=1

δX n,kt

defines a collection of interacting diffusions, with

b(x ,m, y) = b(x ,m, α(x ,m, y)),

being the Nash equilibrium drift, where α takes the explicit form:

α(x ,m, y) ∈ arg mina∈A

[b(x ,m,a) · y + f (x ,m,a)] .




(0,T ]× (Rd)n 3 (t ,x) 7→ αn,i(

x ,mnX ,Dxi v

n,i(t ,x))


n∑n

i=1 δxi


dX n,it = b(X n,i

t , µnt ,Dxi v


t , µnt =

1n

n∑k=1

δX n,kt

defines a collection of interacting diffusions,

with

b(x ,m, y) = b(x ,m, α(x ,m, y)),



[b(x ,m,a) · y + f (x ,m,a)] .




(0,T ]× (Rd)n 3 (t ,x) 7→ αn,i(

x ,mnX ,Dxi v

n,i(t ,x))


n∑n

i=1 δxi


dX n,it = b(X n,i

t , µnt ,Dxi v


t , µnt =

1n

n∑k=1

δX n,kt

defines a collection of interacting diffusions, with

b(x ,m, y) = b(x ,m, α(x ,m, y)),



[b(x ,m,a) · y + f (x ,m,a)] .


Nash Equilibrium n-player dynamics

Recall that the corresponding Nash equilibrium dynamics has the form

dX n,it = b(X n,i

t , µnt ,Dxi v


t , µnt =

1n

n∑k=1

δX n,kt

In other words, it is a system of weakly interacting diffusions:

dX it = bn(t ,X i

t , µnt )dt + σdBi

t , i = 1, . . . ,n,

wherebn(t , x ,m) = b(x ,m,Dxi v

n,i(t , x))

But the drift is n-dependent, so this is not of the form we consideredearlier. Instead, replace the n-dependent control vn,i by a quantitycoming from the master equation.


An Approximating System

Recall: Interacting diffusions describing Nash equilibrium dynamics:

dX n,it = b(X n,i

t , µnt ,Dxi v


t , µnt =

1n

n∑k=1

δX n,kt

Instead: Consider the modified system coming from the limit system:“Replace” vn,i by un,i , where

un,i(t , x1, . . . , xn) = U(t , xi ,mnx), mn

x =1n

n∑k=1

δxi .

the dependence of un,i on n is only through the empiricalmeasureThat is, consider the sequence of IPS:

dX n,it = b(t , X n,i

t ,mnX t) + dW i

t , i = 1, . . . ,n,

whereb(t , x ,m) = b(x ,m,DxU(t , x ,m))


Overall Philosophy:Transferring LLN/CLT Results

Interacting diffusions describing Nash equilibrium dynamics:

dX n,it = b(X n,i

t , µnt ,Dxi v


t , µnt =

1n

n∑k=1

δX n,kt

Approximating diffusions in the form of an IPS


t , µnt ) + dW i

t , µnt =

1n

n∑k=1

δX n,kt

b(t , x ,m) = b(x ,m,DxU(t , x ,m))

1 Analyze master equation + Nash PDE to proveE[W2,Cd (µn, µn)] = O(n−2)

2 Invoke IPS results to deduce LLN/CLT for {µn}.3 Use estimate in 1. to deduce LLN/CLT for {µn}.


Interacting Particle Systems

3. Large Deviations forInteracting Particle Systems

and MFG


Large Deviations Theory

Large deviations (LD) is an asymptotic theory that characterizesthe asymptotic probability of rare events

When a sequence of probabilities of events decay to 0, largedeviations characterizes the asymptotic exponential decay rate:suppose νn, n ∈ N take values in (some topological space) E , andsatisfies for all “nice” A ⊂ S,

P(νn ∈ A) ∼ e−snI(A),

where I : E 7→ [0,∞] is lowersemicontinuous with compact levelsets, and I(A) := infs∈A I(s).One says in this case that {νn} satisfies a large deviation principle(LDP) on E with speed {sn} and good rate function (GRF) IThus the rate of decay of the probabilities is expressed in terms ofa variational problem. Often I(a) itself is also expressed in termsof a variational problem.



Large deviations (LD) is an asymptotic theory that characterizesthe asymptotic probability of rare eventsWhen a sequence of probabilities of events decay to 0, largedeviations characterizes the asymptotic exponential decay rate:suppose νn, n ∈ N take values in (some topological space) E , andsatisfies for all “nice” A ⊂ S,


where I : E 7→ [0,∞] is lowersemicontinuous with compact levelsets, and I(A) := infs∈A I(s).

One says in this case that {νn} satisfies a large deviation principle(LDP) on E with speed {sn} and good rate function (GRF) IThus the rate of decay of the probabilities is expressed in terms ofa variational problem. Often I(a) itself is also expressed in termsof a variational problem.





where I : E 7→ [0,∞] is lowersemicontinuous with compact levelsets, and I(A) := infs∈A I(s).One says in this case that {νn} satisfies a large deviation principle(LDP) on E with speed {sn} and good rate function (GRF) I

Thus the rate of decay of the probabilities is expressed in terms ofa variational problem. Often I(a) itself is also expressed in termsof a variational problem.





where I : E 7→ [0,∞] is lowersemicontinuous with compact levelsets, and I(A) := infs∈A I(s).One says in this case that {νn} satisfies a large deviation principle(LDP) on E with speed {sn} and good rate function (GRF) IThus the rate of decay of the probabilities is expressed in terms ofa variational problem. Often I(a) itself is also expressed in termsof a variational problem.


A Rigorous Statement of the Large Deviation Principle

• E - topological space• {νn} - sequence of E-valued random elements

Definition (Large Deviation Principle (LDP))

{νn} is said to satisfy a large deviations principle (LDP) with speed {sn}

and rate function I : R 7→ [0,∞) if for all measurable A,

− infw∈A◦

I(w)≤ lim infn→∞ 1

s(n)logP(νn ∈ A)

≤ lim supn→∞

1s(n)

logP(νn ∈ A) ≤ − infw∈A

I(w),

where I is lower semicontinuous and has compact level sets.

In short, the LDP says that for all “nice” sets A ⊂ E ,

P(νn ∈ A) ≈ e−sn infw∈A I(w)



The Contraction Principle

Theorem. Let Y and Y ′ be topological spaces and let F : Y 7→ Y ′ bea continuous mapping. Suppose a sequence {Yn} of Y-valued randomvariables satisfies an LDP with rate function I : Y 7→ [0,∞]. Then thesequence {Y ′n := F (Yn)} satisfies an LDP with rate functionJ : Y ′ 7→ [0,∞], given by

J(y ′) = inf{I(y) : F (y) = y ′ for some y ∈ Y}.

Exercise 3: Prove the contraction principle.


Theory of Large Deviations (Sanov’s Theorem)Suppose Yi , i = 1, . . . , are iid on some Polish space Y with commondistribution θ ∈ P(Y), and define

νn :=1n

n∑i=1

δYi

Also, define relative entropy: given ν, µ ∈ P(Y),

H(ν|θ) :=

∫Y

ln(

dνdθ

(x))ν(dx).

if ν� θ and H(ν|θ) = ∞ otherwise.

Sanov’s Theorem

Then {νn} satisfies an LDP in P1(Y) with good rate function H(·|θ).

Exercise 4: Prove Sanov’s theorem when Yi take values in a finitestate space.


Large Deviations in the non-interacting Case (b = 0)

X n,it = X n,i

0 + W it , Qn

t =1n

n∑k=1

δX n,k0 ,W i ,

{X i0}i∈N iid with common distribution µ0; {W i }i∈N iid Brownian motions.

Theorem: As an immediate consequence of Sanov’s theorem wehave {Qn} satisfies an LDP on P1(Rd × Cd

0) with rate function

R(ν|µ0 ×W),

where recall µ0 is the initial distribution of X i0 and W is d-dimensional

Wiener measure on Cd0


References (LDPs for Interacting Particle Systems)

dX n,it = b(t ,X n,i

t , µnt )dt + dW i

t , µnt =

1n

n∑k=1

δX n,kt,

{X i0}i∈N iid with common distribution µ0; {W i }i∈N iid Brownian motions

D. Dawson and J. Gärtner, “Large deviations from theMcKean-Vlasov limit for weakly interacting diffusions”,Stochastics: An International Journal of Probability and StochasticProcesses 20 (1987), 247-308.

A. Budhiraja, P. Dupuis, and M. Fischer, “Large deviationproperties of weakly interacting processes via weak convergencemethods”, Annals of Probability (2012), 74-102.

M.Fischer, “On the form of the large deviation rate function for theempirical measures of weakly interacting systems”, Bernoulli 20(2014), 1765-1801.


References (LDPs for Interacting Particle Systems)


t , µnt )dt + dW i

t , µnt =

1n

n∑k=1

δX n,kt,

{X i0}i∈N iid with common distribution µ0; {W i }i∈N iid Brownian motions

For the ultimate application to MFG, need to consider an extensionbeyond those references that includes

random initial conditions,

time-dependent drift

and a weaker continuity condition on the drift b, namely,continuous as a map from [0,T ]× Rd × P1(Rd) to Rd ,in particular, b need not be continuous in the third variable withrespect to the weak topology

And also allows for common noise, which we do not include here.


Large Deviations in the Interacting Case


t , µnt )dt + dW i

t , µnt =

1n

n∑k=1

δX n,kt,

(X n,i0 )i∈N iid with common distribution µ0.

Idea: To use the contraction principleNeed to express the law of X n as a continuous functional of the law{Qn} of the non-interacting particle system and a Brownian motion

Key Result: Girsanov’s TheoremRelates the law µ of the solution X to the SDE

dXt = dWt

with the law of µb of the solution X b to the SDE with an adapted(suitably regular) drift (rt)t≥0

dX bt = rtdt + Wt


Large Deviations in the Interacting CaseExercise 5. Prove the LDP for {µn

t } via the following steps:

1. Canonical setup: Define the mappings

e : (y × f ) ∈ Rd × Cd0 7→ y

wt : (y × f ) ∈ Rd × Cd0 7→ ft , t ≥ 0.

2. For Q ∈ P1(Rd × Cd0), define the McKean-Vlasov equation map:

xt = e +

∫ t

0b(s, xs,Q ◦ x−1

s )ds + wt , (1)

where e and wt denote the canonical maps on Rd × Cd0 , as

defined above. Here,• Q represents the joint law of the initial condition e and drivingnoise w• Q ◦ x−1

s ∈ P1(Rd) represents the marginal law at time s of thesolution x to equation (1), under Q.





e : (y × f ) ∈ Rd × Cd0 7→ y

wt : (y × f ) ∈ Rd × Cd0 7→ ft , t ≥ 0.


xt = e +

∫ t

0b(s, xs,Q ◦ x−1

s )ds + wt , (1)


defined above.

Here,• Q represents the joint law of the initial condition e and drivingnoise w• Q ◦ x−1






e : (y × f ) ∈ Rd × Cd0 7→ y

wt : (y × f ) ∈ Rd × Cd0 7→ ft , t ≥ 0.


xt = e +

∫ t

0b(s, xs,Q ◦ x−1

s )ds + wt , (1)


defined above. Here,• Q represents the joint law of the initial condition e and drivingnoise w• Q ◦ x−1




Recall: X n,i0 iid, and


t , µnt )dt+dW i

t , µnt =

1n

n∑k=1

δX n,kt, Qn =

1n

n∑k=1

δ(X n,k0 ,W k )

and for Q ∈ P1(Rd × Cd0), the McKean-Vlasov equation map is:

xt = e +

∫ t

0b(s, xs,Q ◦ x−1

s )ds + wt ,

where e and wt denote canonical variables on Rd × Cd0 .

3. Let Φ : P1(Rd × Cd0) 7→ C([0,T ] : P1(Rd)) be the mapping that

takes Q to the flow of marginal measures (Q ◦ x−1t )t≥0, and

observeµn = Φ(Qn).

4. Prove Φ : P1(Rd × Cd0 ) 7→ C([0,T ] : P1(Rd)) is uniformly

continuous.



Recall: X n,i0 iid, and


t , µnt )dt+dW i

t , µnt =

1n

n∑k=1

δX n,kt, Qn =

1n

n∑k=1

δ(X n,k0 ,W k )

and for Q ∈ P1(Rd × Cd0), the McKean-Vlasov equation map is:

xt = e +

∫ t

0b(s, xs,Q ◦ x−1

s )ds + wt ,

where e and wt denote canonical variables on Rd × Cd0 .

3. Let Φ : P1(Rd × Cd0) 7→ C([0,T ] : P1(Rd)) be the mapping that

takes Q to the flow of marginal measures (Q ◦ x−1t )t≥0, and

observeµn = Φ(Qn).

4. Prove Φ : P1(Rd × Cd0 ) 7→ C([0,T ] : P1(Rd)) is uniformly

continuous.Kavita Ramanan, Brown University AMS Short Course JMM, Denver, Colorado January 14, 2020Mean-Field Games 40 / 45

Large Deviations in the Interacting Case: Summary

Given the IPS


t , µnt )dt+dW i

t , µnt =

1n

n∑k=1

δX n,kt, Qn =

1n

n∑k=1

δ(X n,k0 ,W k )

We have shown• {Qn} satisfies an LDP with rate function R(Q|µ0 ×W)

• µn = Φ(Qn)

• Φ : P1(Rd × Cd0 ) 7→ C([0,T ] : P1(Rd)) is uniformly continuous.

5. Apply the contraction principle to conclude that if b is boundedand continuous and Lipschitz continuous in the second and thirdarguments (uniformly in time), then {µn} satisfies an LDP with ratefunction

J(ν) = inf{R(Q|µ0 ×W) : Φ(Q) = ν}.

This concludes Exercise 5 and the proof of the LDP for IPS.


Large Deviations for Mean-Field Games

Large Deviations for Mean-Field Games

Recall the form of interacting diffusions describing Nash equilibriumdynamics:

dX n,it = b(X n,i

t , µnt ,Dxi v


t , µnt =

1n

n∑k=1

δX n,kt,

Aim:To prove an LDP for the sequence (µn)n∈N of empirical measures of

the sequence of Nash equilibria state processes


Principle: Transfer LDP results from IPS to MFG

Interacting diffusions describing Nash equilibrium dynamics:

dX n,it = b(X n,i

t , µnt ,Dxi v


t , µnt =

1n

n∑k=1

δX n,kt

In view of the previous results on LDPs for IPS, recall theapproximating diffusions we considered earlier that were in the form ofan IPS:


t , µnt ) + dW i

t , µnt =

1n

n∑k=1

δX n,kt

b(t , x ,m) = b(x ,m,DxU(t , x ,m))

1 Originally, had only E[W2,Cd (µn, µn)] = O(n−2)

2 Invoke IPS results to get LDP for {µn}.3 Use a sharper exponential estimate in 1. to deduce LDP for {µn}.


Additional References on LDP

Lacker and Ramanan, “Rare Nash equilibria and the price ofanarchy in large static games,” (2019) Mathematics of OperationsResearch 44 (2019) no. 2, 400-422.

Cardaliaguet, Delarue, Lasry and Lions, “The master equationand the convergence problem in mean-field games” (2019)

F. Delarue, D. Lacker and K. Ramanan, “From the master equationto mean field game limit theory: Large deviations andconcentration of measure," (2018) to appear in Annals ofProbability


IV. Open Problems

Study refined convergence theorems for open-loop Nashequilibria.

Investigate if there are cases when the MFG LDP exists, butdiffers from the interacting paricle system LDP obtained from themaster equation.

Establish large deviation principles for stochastic differentialgames in the presence of non-uniqueness (as has been done inthe static case)

Use LDPs to find interesting conditional limit laws in both the staticand stochastic differential settings.

...


deviations and fluctuations for mean-field games · deviations and fluctuations for mean-field...

Documents