1 Large Deviations-Luca Oddis

Exercise 1.1.

Prove that

limb!+1

limn!+1

1

rnln

Z

f�b

ernf dµn = �1, (1)

if there exists ↵ > 1 s.t.

supn2

✓Zern↵f dµn

◆ 1rn

< +1. (2)

Solution.We first apply Holder’s inequality with exponents

�↵; ↵

↵�1

�

1

rnln

Z

f�b

ernf dµn

1

rnln

"✓Z

f�b

ern↵f dµn

◆ 1↵✓Z

f�b

1 dµn

◆↵�1↵

#

=1

↵ln

"✓Z

f�b

ern↵f dµn

◆ 1rn

#+

1

rnlnhµn (f � b)

↵�1↵

i,

(3)

and, using the hypothesis, we have

limn!1

1

↵ln

"✓Z

f�b

ern↵f dµn

◆ 1rn

# lim

n!1

1

↵ln

"✓Zern↵f dµn

◆ 1rn

#= C1 < +1,

(4)while, for the second terms in the r.h.s. of (3), we have the folowing estimate:

1

rnlnhµn (f � b)

↵�1↵

i

↵� 1

↵lnn⇥

e�↵rnbe

�↵rnbµn (f � b)⇤ 1

rn

o

=↵� 1

↵ln

(e�↵rnb

Z

f�b

ern↵b dµn

� 1rn

)

�(↵� 1)b+ ln

(Z

f�b

ern↵b dµn

� 1rn

), (5)

which, again via the hp, implies

limn!1

1

rnlnhµn (f � b)

↵�1↵

i< �(↵� 1)b+ C2. (6)

Now, using (4) and (6), since we have ↵ > 1, we can conlude that

limb!+1

limn!+1

1

rnln

Z

f�b

ernf dµn lim

b!+1

limn!+1

1

rnln

Z

f�b

ernf dµn = �1. (7)

1

Exercise 1.2.

Consider the following game. Let Sn be the number of tails you get whenyou flip n times a fair coin. Determine the asymptotics of

⇥3Sn

⇤.

Solution.Let us write ⇥

3Sn⇤=

⇥eSn ln 3

⇤=

Zenx ln 3

dµn(x), (8)

where µn is the law of Snn , so we define f(x) := x ln 3. Now we aim to use

the extended Varadhan’s Lemma, testing the condition of Exercise 1. Using(8), if ↵ > 1

Ze↵nf

dµn(x) = µn [3↵n] =

nX

k=0

1

2n

⇣n

k

⌘3↵n

kn =

1

2n

nX

k=0

⇣n

k

⌘3↵k1n�k

=1

2n(3↵ + 1)n . (9)

Thus, we take the n-th root and see that the condition of the Exercise 1holds. We can use now both (the first version of) L.D.P and the extendedVaradhan’s Lemma and we can show that

limn!+1

1

nln

⇥3Sn

⇤= sup

x2[0,1]{g(x)} , (10)

where g(x) := x ln 3� ln 2�x ln x� (1�x) ln(1�x). g0(x) has a 0 for x = 34 ,

which is a local maximum on [0, 1] (g(0) = ln 2, g(1) = ln 32). Finally,

limn!+1

1

nln [3Sn ] = g(

3

4) = ln 2. (11)

Exercise 1.3.

Solution.

[Zi] = 11

2+ 2

1

2=

3

2,

which implies, since Zi are i.i.d.

[Wn] =nY

i=1

[Zi] =

✓3

2

◆n

.

2

At this point we can consider the random variables Xi := lnZi, for i 2 ,which are still i.i.d. This sequence satisfies the hypotheses of the L.L.N. and[Xi] =

12 ln 2, thus

1 = limn!+1

(�����1

n

nX

i=1

Xi �1

2ln 2

����� < ✏

)

= limn!+1

8<

:

������ln

2

4

nY

i=1

Zi

! 1n

3

5�1

2ln 2

������< ✏

9=

; , (12)

which, via exponentiation, (actually, ✏ is not the same as the one in thethesis) yields the result.

Exercise 1.4.

Let S be a Polish space. Let {Xn} be a sequence of i.i.d. S-valued randomvariables with common distribution � and let � : S !

d be a continuousfunction. Assume that

⇥ea|�(X1)|

⇤< +1, 8a > 0. (13)

Prove by contraction from Sanov’s theorem that

Sn

n:=

1

n

nX

k=1

�(Xk) (14)

satisfies a LDP with rate n and good and convex rate function I : d!

[0,+1] given by

I(z) := inf{H(⌫|�) : ⌫(�) = z} = sup✓2 d

{z · ✓ � ln�(e✓·�)}. (15)

Solution.Suppose � is bounded. We aim to derive the L.D.P. by contraction. Weknow that if we define

LXn :=

1

n

nX

i=1

�Xi , (16)

the sequence of laws of LXn satisfiy a L.D.P. with rate function H(⌫|�). We

now define the mapT : M1(S) !

d, (17)

such that T (⌫) = (⌫(�1), . . . ⌫(�d)). This map is continuous since all the Tj

( -valued) are continuous. This is due to the fact that we consider the weak

3

topology on M1(S) in which they are continuous by definition. Observe thatSnn = T (LX

n )We can now apply the Contraction Principle and obtain that the laws of Sn

nstatisfy a L.D.P. with rate n and good rate function

I�(x) = inf⌫2M1(S):⌫(�)=x

H(⌫|�), x 2d. (18)

We proceed now by showing the convexity of I�(x). By definition of infimum,fixing x, y 2

d, for ✏ > 0, there exist ⌫x, ⌫y

H(⌫x|�) < I�(x) + ✏,

H(⌫y|�) < I�(y) + ✏,

where ⌫x(�) = x, ⌫

y(�) = y. Now ↵(⌫x(�) + (1 � ↵)⌫y(�) = ↵x + (1 � ↵)y.That implies ↵(⌫x(�) + (1 � ↵)⌫y(�) is in the set in which the infimum istaken,

I�(↵x+ (1� ↵)y) H(↵⌫x + (1� ↵)⌫y|�), (19)

which allows us to use the convexity of the relative entropy and show that

I�(↵x+(1�↵)y) ↵H(⌫x|�)+(1�↵)H(⌫y|�) ↵(I�(x)+✏)+(1�↵)(I�(x)+✏).(20)

We shall now send ✏ to zero and find the right inequality leading to convexity.Now we take the Legendre transform

I⇤(✓) := sup

z2 d

{z · ✓ � I�(z)} ,

which can be rewritten

supz2 d

sup

⌫2M1(S),⌫(�)=z{⌫(� · ✓)�H(⌫|�)}

!. (21)

HenceI⇤(✓) = sup

⌫2M1(S){⌫(� · ✓)�H(⌫|�)} . (22)

At this point we note that, by the above identity,

I⇤(✓) = (H(⌫|�))⇤(� · ✓) = (p⇤)⇤(� · ✓) = p(� · ✓), (23)

which yields, since I is l.s.c. and convex,

I�(✓) = I⇤⇤(✓) = p

⇤(✓), (24)

4

by Fenchel-Moreau Theorem [RAS15]. Hence

I�(z) = sup✓2 d

{z · ✓ � p(� · ✓)} = sup✓2 d

�z · ✓ � ln�(e�·✓)

. (25)

Exercise 1.5.

Exercise 2.16 in [RAS15].

Solution.First, we observe that, since E is closed, we can work inside of it and forgetabout �. Given G ✓ E open and F ✓ E close, we define the two sets G� ✓ G

and F�◆ F in the following way:

G� := {x 2 G|dist(x, @G) > �}, (26)

which is an open subset of E , since G� = {x 2 E|dist(x, @G) > �} \ G anddist(·, G) is continuous. Likewise, we define

F� := {x 2 E|dist(x, F ) �}, (27)

which is a closed set.Now, 8� > 0 there exist n such that d(⇠n(!), ⌘n(!)) < �, 8n > n, 8!.PART A:Indeed, by definition we have

{⌘n 2 G} ◆ {⇠n 2 G�}, n > n, (28)

which implies

lim infn!1

1

rnln ({⌘n 2 G}) � � inf

x2G�

I(x), 8�. (29)

There exists a minimizing (monotonic) sequence {xk} ✓ G for I and thereexist a corresponding sequence {G�k} s.t. 8k 2 xk 2 G�k . (This followsfrom the fact that every x 2 G has positive distance from @G).By definition

xk � infG�k

I(x), (30)

thus, if we take the limits, which are proper limits since both sequences aremonotonic, we get

infG

I(x) � limk!1

infG�k

I(x). (31)

5

Finally, we use the inverse inequality and get

lim infn!1

1

rnln ({⌘n 2 G}) � � inf

x2GI(x), (32)

which is the L.D.P for {⌘n}.PART B:

{⌘n 2 F} ✓ {⇠n 2 F�}, n > n (33)

and hence

lim infn!1

1

rnln ({⌘n 2 F}) � inf

x2F�

I(x), 8�. (34)

It is left to prove that, if �n & 0,

limn!1

infF �n

� infx2F

I(x), (35)

We know, using that the rate function is good, F �\ {I(x) c} is compact

8c 2 , thus we can consider a sequence {xn} 22 F�n \ {I(x) c} s.t.

I(xn)�1

n inf

F �nI(x) (36)

There exists a subsequence {xnk} such that xnk

! x and by lower semicon-tuity we obtain

lim infn!1

infx2F �n

I(x) � lim infn!1

I(xnk) � I(x). (37)

Exercise 1.6.

Exercise 5.20 in [RAS15].

Solution.(a) Whenever p

⇤(z) 6= +1, the set Nz := {⌫ 2 M1(S)| ⌫ [H] = z} isnonempty and there exists ⌫ 2 Nz such that H(⌫|�) = c < +1.Thus,

p⇤(z) = inf{{⌫ 2 Nz} \ {H(⌫|�) c}}, (38)

and the infimum is taken on a compact set, since the former set is closed(it follows from the continuity of •[H]) and the latter is a sublevel set and,therefore, it is compact. Hence, we take in this set a minimizing sequence{⌫n} and we know that it exists a subsequence {⌫nk

} s.t. ⌫nk! ⌫0 2 {{⌫ 2

Nz} \ {H(⌫|�) c}}. We have, by the lower semicontinuity of H(·|�),that

p⇤(z) = lim

k!1

H(⌫nk|�) � H(⌫0|�), (39)

6

thus, the minimum is attained at ⌫0. Suppose now there exist two measures⌫0, ⌫1 when the minimum is attained. By the strict convexity of H(·|�) (seeExercise 5.5 in [RAS15]), if ✓ 2 (0, 1), we have

H(✓⌫0 + (1� ✓)⌫1|�) < ✓H(⌫0|�) + (1� ✓)H(⌫1|�) = p⇤(z). (40)

Observe that ✓⌫0 + (1 � ✓)⌫1 2 Nz. Hence (40) would imply p⇤(z) < p

⇤(z),which is absurd.(b)

p0(t) =

d

dt

�ln �[etH]

�=

ddt

R⌦ e

tHd�(!)R

⌦ etHd�(!)=

R⌦ He

tHd�(!)

Z�t= µ�t [H]. (41)

Similarly, for the second derivative, we obtain

p00(t) =

R⌦ H

2etHd�(!)Z�t �

�R⌦ He

tHd�(!)

�2

(Z�t)2= µ�t [H2]� µ�t [H]2, (42)

where in both (41) and (42) the hypoteses for the di↵erentiation under theintegral sign are satisfied.Now, note that

limt!0

H(!)etH(!) = H(!) 8! 2 ⌦, |HetH| 2||H||1 2 L

1(⌦, d�) if t is small enough,

as well as

limt!0

etH = 1 8! 2 ⌦, |e

tH| 2 2 L

1(⌦, d�) if t is small enough,

limt!0

p0(t) = lim

t!0

R⌦ He

tHd�(!)

Z�t=

limt!0

R⌦ He

tHd�(!)

limt!0 Z�t=

�[H]

1= �[H],

(43)where we used dominated convergence on both numerator and denominator.(c)We solve the exercise for the essential infimum since the other case is analo-gous. W.l.o.g. we can assume t < 0.In order to get only positive terms, p0(t)can be rewritten as

p0(t) = A+

R⌦(H� A)et(H�A)

d�(!)R⌦ et(H�A)d�(!)

. (44)

STEP 1

7

Fix ✏ > 0. We note that fot t going to �1 the integrals in (44) are mostlydetermined by their behaviour near the infimum. More rigorously:

limt!�1

R⌦(H� A)et(H�A)

d�(!)R⌦ et(H�A)d�(!)

= limt!�1

R{AHA+✏}(H� A)et(H�A)

d�(!)R⌦ et(H�A)d�(!)

.

(45)Let us prove (45). We shall work with the complement {H � A+ ✏}, observethat (we recall that t < 0)

Z

{H�A+✏}

(H� A)et(H�A)d�(!) e

t✏||H� A||1�({H � A+ ✏}), (46)

Z

⌦

et(H�A)

d�(!) �

Z

{AHA+ ✏2}

et(H�A)

d�(!) � et ✏2�({A H A+

✏

2}).

(47)Thus, using (46) and (47) we get

limt!�1

R{H�A+✏}(H� A)et(H�A)

d�(!)R⌦ et(H�A)d�(!)

limt!�1

et ✏2||H� A||1�({H � A+ ✏}

�({A H A+ ✏2})

= 0,

(48)8✏ > 0 since, by definition of ess� sup, �({A H A+ ✏

2}) > 0 8✏ > 0.STEP 2

We now compute the limit of the r.h.s. of (45)

limt!�1

R{AHA+✏}(H� A)et(H�A)

d�(!)R⌦ et(H�A)d�(!)

limt!�1

✏R{AHA+✏} e

t(H�A)d�(!)

R{AHA+✏} e

t(H�A)d�(!)= ✏,

(49)which, by the arbitrariety of ✏ yields the result.(d)The condition p

00> 0 gives us that the function p

0(t)|(A,B) is onto and strictlyincreasing, so that there exists a unique � = �z s.t. p0(��) = z. Furthermore,

p⇤(z) = sup

t2{zt� p(t)}. (50)

Define f(t) := zt � p(t), then f0(t) = z � p

0(t) and hence f0(t) = 0 ,

p0(t) = z , t = ��, which yields p

⇤(z) = �z� � p(��). Moreover, sincef0(t) > 0 8t < �� and f

0(t) < 0 8t > ��, �� is a maximum point of f .It remains to prove that ⌫z = µ�. Due to item a) in the exercise, we onlyneed to show that µ� [H] = z and p

⇤(z) = H(µ�|�). The first identity isequivalent to p

⇤(��) = z, by item b, which is true by the definition of �.

8

Exercise 1.7.

Exercise 6.5 in [RAS15].

Solution.Let us suppose, for now, that g depends only on one site of the configuration!, i.e. g(!) = g(!k) k 2

d. We define the set

Vkn := {i 2 Vn|k � i 2 Vn}, (51)

so that, by definition of the periodized configuration, we have

✓i!(n) = !k�i = ✓i!, (52)

8i 2 Vkn . We can then extimate

|Rn(!, g)�Rn(!, g)| 2||g||1|Vn \ V

kn |

|Vn|, (53)

which tends to 0, since |Vn| = (2n�1)d and |Vn\Vkn | ⇠ Cn

d�1. The inequalitystill holds true for the sup.We shall remove, now, the restriction and consider the most general localfunction g(!) = g(!k1 , . . .!kj , . . .!kp). The di↵erence

g(✓i!(n))� g(✓i!) =

pX

j=1

�gi,j, (54)

where�gi,j(!) = g((✓i!)k1 , . . . (✓i!

(n))kj , . . . (✓i!(n))kp)�g((✓i!)k1 , . . . (✓i!)kj , (✓i!

(n))kj+1, . . . (✓i!(n))kp).Thanks to this decomposition we can use the special case we have just dealtwith. We get

|Rn(!, g)�Rn(!, g)| 1

Vn

pX

j=1

�����X

i2Vn

�gi,j(!)

�����

2||g||1|Vn \

Tpj=1 V

kjn |

|Vn|���!n!1

0.

Exercise 1.8.

Exercise 6.10 in [RAS15].

Solution.To prove (a) we use a Lemma for product measure densities:

9

Lemma 1.

Let X and Y be two polish spaces. Let ↵, µ 2 M1(X ) and �, ⌫ 2 M1(Y).Then

↵⌦ � ⌧ µ⌦ ⌫ , ↵ ⌧ µ ^ � ⌧ ⌫. (55)

In this cased(↵⌦ �)

d(µ⌦ ⌫)=

d↵

dµ

d�

d⌫.

Proof.(()Let A 2 X and B 2 Y be measurable sets. By definition of product measurewe have

↵⌦ �(A⇥ B) = ↵(A)�(B) =

Z

A

d↵

dµdµ

Z

B

d�

d⌫d⌫ =

Z

A⇥B

d↵

dµ

d�

d⌫d(µ⌦ ⌫).

Hence, ↵⌦ � ⌧ µ⌦ ⌫ with density d(↵⌦�)d(µ⌦⌫) =

d↵dµ

d�d⌫ .

())Let A 2 X be a measurable set.

↵(A) = ↵(A)�(Y) = ↵⌦ �(A⇥ Y) =

Z

A⇥Y

d(↵⌦ �)

d(µ⌦ ⌫)d(µ⌦ ⌫)

=

Z

A

✓Z

Y

d(↵⌦ �)

d(µ⌦ ⌫)d⌫

◆dµ,

which yields the result by taking d↵dµ =

RY

d(↵⌦�)d(µ⌦⌫)d⌫. The same holds true for

�.

Thanks to Lemma 1 the result holds trivially when one of the relativeentropies is +1, since it implies than the others diverge too. Let, us suppose,therefore that ↵⌦ � ⌧ µ⌦ ⌫.

H(↵⌦ �|µ⌦ ⌫) =

Z

X⇥Y

d(↵⌦ �)

d(µ⌦ ⌫)ln

✓d(↵⌦ �)

d(µ⌦ ⌫)

◆d(µ⌦ ⌫)

=

Z

X

d↵

dµ

Z

Y

d�

d⌫

✓ln

d↵

dµ+ ln

d�

d⌫

◆d⌫dµ

= H(↵|µ) +H(�|⌫).

(b) We apply the previous part

1

|VN |HN(µ

⌦d|�

⌦d) =

1

|VN |HVN (µ

⌦d|�

⌦d) = H(µ|�),

10

which implies

h(µ⌦d|�

⌦d) := lim

N!1

1

|VN |HN(µ

⌦d|�

⌦d) = H(µ|�). (56)

Exercise 1.9.

Let X be a Polish space, let then µ, ⌫ 2 M1(X) such that µ 6= ⌫. Prove thatµ⌦

d? ⌫

⌦d(i.e. µ⌦

dand ⌫

⌦dare mutually singular).

Solution.First, we recall that if µ 6= ⌫, then µ

⌦d6= ⌫

⌦d, since they have di↵er-

ent one-particle marginal distributions. Hence, there exists f 2 Cb(X⌦d)

s.t. µ⌦ d

[f ] 6= ⌫⌦d

[f ]. Futhermore we observe that for the Kolmogorov’szero-one law, both µ

⌦dand ⌫

⌦dare ergodic measures and, thus, applying

Birkho↵’s ergodic theorem we have

Rn(!, f) !µ⌦ d

[f ] µ⌦

d� a.s., (57)

Rn(!, f) !⌫⌦

d

[f ] ⌫⌦

d� a.s. (58)

By defining A := {! 2 X⌦

d|Rn(!, f) ! µ⌦ d

[f ]} and B := AC , (57) and

(58) give us ⌫⌦d(A) = 0 as much as µ⌦

d(B) = 0, i.e. µ⌦

d? ⌫

⌦d.

Exercise 1.10.

Exercise 8.5 in [RAS15].

Solution. Using Th 8.3 in [RAS15], h(·|�) can be written

h(⌫|�) = ⌫ [f�] + h(⌫|�) + P (�), (59)

which is a�ne, since ⌫ 7�!⌫ [f�] is linear, the specific relative entropy

h(⌫|�) is a�ne for Prop 6.8 in [RAS15] and P (�) is a constant.Furthermore h(⌫|�) is lower semicontinuous (Prop. 6.8 in [RAS15]). Hence,if ⌫n ! ⌫0, we have

lim infn!1

h(⌫n|�) =⌫0 [f�] + lim inf

n!1

h(⌫n|�) + P (�) � ⌫0 [f�] + h(⌫n|�) + P (�)

= h(⌫0|�),

where the first equality follows from the definition of weak convergence andthe fact that f� 2 Cb. Finally, note that, since f� 2 Cb, if c 2

{⌫ 2 M✓|h(⌫|�) c} = {⌫ 2 M✓|⌫ [f�] + h(⌫|�) + P (�) c}

✓ {⌫ 2 M✓|� ||f�||1 + h(⌫|�) + P (�) c}

= {⌫ 2 M✓|h(⌫|�) c� P (�) + ||f�||1},

11

and the last set is compact (Proposition 6.8 in [RAS15]). Hence, since lowersemicontinuity yields that {⌫ 2 M✓|h(⌫|�) c} is a closed set 8c, the resultis proven (a closed set in a compact set is compact).

References

[RAS15] F. Rassoul-Agha and T. Seppalainen, A course on large deviationswith an introduction to gibbs measures, 2015.

12