introduction to monte carlo methods (lecture 2)christophe.chorro.fr/docs/mc2.pdf · when x is...
TRANSCRIPT
![Page 1: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/1.jpg)
Introduction to Monte Carlo Methods (Lecture 2)
Christophe Chorro ([email protected])
University Paris 1
June 19 2008
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 1 / 47
![Page 2: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/2.jpg)
Study Plan
Chapitre 1: Simulation of random variablesInversion of the distribution functionTransformation methodRejection MethodSimulation of Gaussian vectorsSimulation of conditional distribution
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 2 / 47
![Page 3: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/3.jpg)
Simulation of random variables
Starting from i.i.d uniform random variables on [0, 1] (Un)n∈N obtained
using one sequence of pseudo-random numbers
or several sequences of quasi-random numbers,
we want to generate random samples from a general distribution (exponential,Cauchy, Gaussian, Poisson....).
Inversion of distribution function
Transformation method
Rejection method
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 3 / 47
![Page 4: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/4.jpg)
Plan
1 Simulation of random variablesInversion of the distribution functionTransformation methodRejection methodSimulation of Gaussian vectorsSimulation of conditional distribution
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 4 / 47
![Page 5: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/5.jpg)
Inversion Method
Let X be a real random variable, the distribution function of X is defined as
FX (x) = P(X ≤ x); x ∈ R.
Properties: non-decreasing, right C0, limx→+∞
F (x) = 1 and limx→−∞
F (x) = 0.
DefinitionWe define the generalized inverse of FX denoted by F−X where ∀u ∈]0, 1[,
F−X (u) = inf{x | FX (x) ≥ u}.
Remark: When FX is strictly increasing and continuous, F−X = F−1X .
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 5 / 47
![Page 6: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/6.jpg)
Inversion Method
PropositionIf U ↪→ U([0, 1]) then F−X (U) has the same distribution then X.
Proof: We just have to remark that ∀u ∈]0, 1[, ∀x ∈ R,
F−X (u) ≤ x ⇔ u ≤ FX (x).�
Figure: Illustration of the definition of F−X
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 6 / 47
![Page 7: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/7.jpg)
Inversion MethodExample 1: Let X be a discrete random variable (Bernoulli, Binomial, Poisson)
such that P(X = xk ) = pk
( ∞∑k=0
pk = 1)
.
One has FX (x) =∞∑
k=0pk 1xk≤x and
F−X (u) = x01u≤p0 +∞∑
k=1
xk 1(k−1Pj=0
pj<u≤kP
j=0pj
).
When X is finite we have the following algorithm:
p = p0, j = 0
while p < U
do j = j + 1, p = p + pj
end
Y = xj
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 7 / 47
![Page 8: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/8.jpg)
Inversion Method
Example 2: Particular continuous distributions
If X ↪→ E(λ), λ > 0 then FX (x) = (1− e−λx)1x≥0 and
−1λ
Log(1− U) ↪→ E(λ).
If X ↪→ C(a), a > 0, then FX (x) = 1π [Arctan( x
a ) + π2 ] and
a tan(
π(U − 12
)
)↪→ C(a).
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 8 / 47
![Page 9: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/9.jpg)
Inversion Method
−2000 0 1500
0.00
000.
0010
0.00
20
0 2 4 6 80.
00.
20.
40.
60.
8
Figure: Empirical density of 10000 C(1) (left) and E(1) obtained by Inversion method
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 9 / 47
![Page 10: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/10.jpg)
Inversion Method
Example 3: When X ↪→ N (0, 1), FX is unknown but we have the followingapproximation:
If u > 0; 5, let t =√−2log(1− u)
F−X (u) ≈ t − c0 + t(c1 + tc2)
1 + t(d1 + t(d2 + td3)).
If u ≤ 0; 5, let t =√−2log(u)
F−X (u) ≈ c0 + t(c1 + tc2)
1 + t(d1 + t(d2 + td3))− t .
Where
c0 = 2.515517, c1 = 0.802853, c3 = 0.010328,
d1 = 1.432788, d2 = 0.189269, d3 = 0.001308.
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 10 / 47
![Page 11: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/11.jpg)
Inversion Method
−4 −2 0 2 4
0.0
0.1
0.2
0.3
0.4
Figure: Empirical density of 10000 standard normal distribution obtained by Inversionmethod
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 11 / 47
![Page 12: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/12.jpg)
Plan
1 Simulation of random variablesInversion of the distribution functionTransformation methodRejection methodSimulation of Gaussian vectorsSimulation of conditional distribution
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 12 / 47
![Page 13: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/13.jpg)
Transformation method
Here we try to express X as a function of another random variable Y easy togenerate.
One of the main tool is the following result
PropositionLet D and ∆ be two open sets of Rd and Φ = (Φ1, ...Φd ) : D → ∆ aC1-diffeomorphism. If g : ∆ → R is measurable and bounded then∫
∆
g(v)dv =
∫D
g(Φ(u)) | JΦ(u)) | du
where JΦ(u) = det[(
∂Φi
∂uj(u))
1≤i,j≤d
].
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 13 / 47
![Page 14: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/14.jpg)
Transformation method
Example 1: Box-Muller method
PropositionIf U1 and U2 are two independent uniform random variables on [0, 1] then
G1 =√−2log(U1)cos(2πU2) and G2 =
√−2log(U1)sin(2πU2)
are two independent N (0, 1).
Proof: Let us define the following C1-diffeomorphism
Φ : (x , y) ∈]0, 1[2→ (u =√−2log(x)cos(2πy), v =
√−2log(x)sin(2πy))
fulfilling |JΦ(x , y)| = 2πx . Since u2 + v2 = −2log(x), according to the change
of variables theorem, one has for F ∈ Cb(R2, R),∫]0,1[2
F (Φ(x , y))dxdy =
∫R2−(R+×{0})
F (u, v)1
2πe−
u2+v22 dudv .�
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 14 / 47
![Page 15: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/15.jpg)
Uniform random variables on particular domain
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
● ●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●●
●
●●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●● ●●
●
●
●●
●●
●
●
●
●
● ●
● ●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●●
●●
●●
●
●●
●
●●
●
●●
●●
●
●●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
● ●
●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
● ●● ●
●
●
●
●
●
● ●● ●
●
●●
●
●
●
●
●
●
●
●
●● ●●
●
●●●
●
●●
●
●
●
●●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●●●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ●●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
● ●
●●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
● ● ●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●●
●●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●●
●●
●
●
●
●●
●
●
●
●●
●
●
●
● ●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
● ●●
●
●● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●●
●
●
●
●●
●
● ●
●
●
●
●●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●
●●
●
●● ●
●
●
●
●
●
● ●
●●
●
●●
●
●●
●
●●
●●
●●
●
● ●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
● ●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
● ●●
●●●
●
● ●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●●
●●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
● ●●●
●
●
●
● ●●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●●
●●●
●
●
●
●●●
●
●
●
●●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
● ●●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●
●
●
●●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●●●
● ●
●
●
●●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●●
●
●
●
● ●
●
●
●
●
●●
●●
●
●
●
●
●●●
●
●●●●
●
●
●
●
●● ●
●●
●
● ●
●
●
●●
●
●
●
●
●
● ● ●
● ●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
● ●
●
●
●
●●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●●
●
●●
●
● ●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
● ●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
● ●●
●●
●
●● ●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●● ●
●●
●
●
●●
●
●●
●
●
●●
●
●
●
●
● ●
●
●●
●
●
●
● ●
●●● ●
● ●
●●●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●●●
●
●
●●
●●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
● ●
●●
●
●
●
●
●
●●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
● ●
●●
●●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
● ●
●
●●
●
●●
●
●
●
● ● ●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●●
●
●
●
●
● ●
●
●●
●
●
● ●
●
● ●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
● ●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●● ●
●
●
●
●
●
●●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●●●
●
●●
●
●●
●
●
●
●
●●
● ●
●● ●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●●●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
● ●
●
●
●
●
● ●
●
●●●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
● ●
●
●
●
●
●●
●
● ●●
●
●●
● ●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●● ●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
● ●
● ●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
● ●
●●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●●
● ●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●●●
●
●●
● ●
●
● ●
●
●
●
●●
●
●
●
●●
● ●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●●
● ●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
● ●
● ●
●
●●
●
●
●
●
● ●
●
●●
●
●
●●
●● ●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
● ●
●●●●
●●
●
●
●
●●●
●
●
●
●●
●
●
●●
● ●●
● ●
●●
●
●●
●
●
●●
●
●
●
● ●
●
● ●
●
●
●●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●●●
●
●
●
●
●
● ●●●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
● ●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
● ●●
●
●
● ●
●●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●● ●
●
●● ●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●● ●
●●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
● ●
● ●
●
●
●
● ●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●
●
●
●
●●
●
●
●
●● ●
●
●
●●
●●● ●
●
● ●
●●
●
●●●●
●
●
●●
●
●
●●
●● ●●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●● ●●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●●
●●
●● ●
●
●
●
●●
●●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●●
● ●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ● ●●
●
●
●
●
● ●
●
●
● ● ● ●●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
● ●●
●
● ●
●
●
●
●●
●
●
● ●
●
● ●
●●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●●
●
●
● ●●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●●
●
●
●●●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●●
●
●●
●
●
●
●
●
●●●
●
●
●
● ●
● ●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●●●
● ●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●● ●
● ●
●●
●
●●
●
●●
●
●
●
●
● ●
●
●●
●
●●
●●●
●
● ●
●
●
●
●●
●●
●
●
●●
●
●● ●
●
●
● ●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●● ●
●●
●
●●
●
●
●
●● ●
●
●
● ●
● ●
●
●●
●
●
● ●
●
●
●●
●
●●
●
●
●
● ●
●
●●
●
●
●
●
● ●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●●●
●●
●
●
●●
●
●
●● ●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
● ●●
●
●
●
●●
●
●
●●
●
●
●
●●
● ●
●●
●
●
●
●
● ●
●
●●
●
●●
●
●
●●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●●
●
●
● ●
●
●●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●
●●
●
●
●●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●●●
●
●●
● ●
●
●
●
●
●●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●●
●
●●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●●
● ●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●●
●●●
● ●
●●
●●
●
●
●●
●
●●●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●●
●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
●●
●
● ●●
●
●
●
●
● ●
●●
●
●
● ●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●●●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
● ●●●
●
●
●
●●●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
● ●
●
● ●
●●●
●
●
●
●
●
−4 −2 0 2 4
−4
−2
02
4
Figure: Simulation of 5000 pairs of independent N (0, 1) by Box-Muller method
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 15 / 47
![Page 16: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/16.jpg)
Transformation method
Example 2: Gauss Marsiglia method
PropositionIf (U1, U2) is uniformly distributed on {(u1, u2) ∈ R2 | u2
1 + u22 < 1} then
G1 = U1
√−2log(U2
1 + U22 )
U21 + U2
2and G2 = U2
√−2log(U2
1 + U22 )
U21 + U2
2
are two independent N (0, 1).
Proof: Exercise.
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 16 / 47
![Page 17: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/17.jpg)
Transformation method
Example 3: From exponential to Poisson distribution
First, some reminder on Gamma distribution:
DefinitionWe say that Z follows a gamma distribution of parameters (a, b) ∈ R2
+
(Z ↪→ G(a, b)) if its density is given by
xa−1e−xb
Γ(a)ba 1R+(x) where Γ(a) =
∫ ∞
0xa−1e−xdx .
PropositionIf Z1 and Z2 are two independent random variables such that Z1 ↪→ G(a1, b)and Z2 ↪→ G(a2, b) then
Z1 + Z2 ↪→ G(a1 + a2, b).
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 17 / 47
![Page 18: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/18.jpg)
Transformation method
PropositionIf (Ei)i≥1 is a sequence of i.i.d E(λ) then
X =∞∑i=1
i 1(iP
k=1Ek≤1<
i+1Pk=1
Ek
) ↪→ P(λ).
Proof: We have ∀n ∈ N,
P(X = i) = P
(i∑
k=1
Ek ≤ 1 <
i+1∑k=1
Ek
)
wherei∑
k=1Ek is a G(i , 1
λ ) independent of Ei+i . The result follows easily. �
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 18 / 47
![Page 19: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/19.jpg)
Transformation method
Exercise: Explain why the following algorithm generate P(λ) distribution:
a = e−λ, X = 0
U < −− Random
while a ≤ U
do U = U ∗ Random, X = X + 1
end
X
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 19 / 47
![Page 20: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/20.jpg)
Inversion Method
0 1 2 3 4 5 6 7 8 9 11 13
Values
Num
ber
od r
ealiz
atio
ns
050
100
150
Figure: 1000 realizations of a P(λ) using the preceding algorithm
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 20 / 47
![Page 21: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/21.jpg)
Plan
1 Simulation of random variablesInversion of the distribution functionTransformation methodRejection methodSimulation of Gaussian vectorsSimulation of conditional distribution
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 21 / 47
![Page 22: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/22.jpg)
Uniform random variables on general domain
PropositionLet (Zn)n≥1 be i.i.d random variables with values in Rd and D ∈ B(Rd ) suchthat P(Z1 ∈ D) > 0. We define
ν1 = inf (k ≥ 1; Zk ∈ D),
νn+1 = inf (k > νn; Zk ∈ D),
Yn = Zνn .
Then (Yn)n≥1 are i.i.d random variables such that
P(Y1 ∈ A) = P(Z1 ∈ A | Z̄1 ∈ D).
Moreover E [ν1] = E [νn+1 − νn] = 1P(Z1∈D) = 1
α .
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 22 / 47
![Page 23: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/23.jpg)
Uniform random variables on general (bounded)domain
CorollaryLet (Zn)n≥1 be i.i.d uniform random variables on [0, M]d and D ∈ B(Rd ) suchthat D ∈ [0, M]d and P(Z1 ∈ D) > 0. We define
ν1 = inf (k ≥ 1; Zk ∈ D),
νn+1 = inf (k > νn; Zk ∈ D),
Yn = Zνn .
Then (Yn)n≥1 are i.i.d uniform random variables on D.
Moreover E [ν1] = E [νn+1 − νn] = vol(D)Md .
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 23 / 47
![Page 24: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/24.jpg)
Uniform random variables on general (bounded)domain
Example: The following algorithm generate a uniform random variable X onD(0, 1) (see Gauss Marsiglia method.)
U < −− 2Random− 1; V < −− 2Random− 1
while U2 + V2 ≥ 1
do U < −− 2Random− 1; V < −− 2Random− 1
end
X = (U, V)
Average number of simulation π4 ≈ 0, 7854
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 24 / 47
![Page 25: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/25.jpg)
Uniform random variables on general (bounded)domain
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
● ●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
● ●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●●
●
●
● ●
●●
●
●●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
● ●
●
●
●●
●
●
●●
●
●
●
●
● ●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
● ●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
● ● ●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●● ●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
● ●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
● ●● ●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●●
●
●
●
●
●
●
●
● ● ●
●
●●
●
●●
●●
●
●
● ●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
● ●●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
● ●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
−1.0 −0.5 0.0 0.5 1.0
−1.
0−
0.5
0.0
0.5
1.0
Figure: Realization of 10000 uniform random variable in the unit disk
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 25 / 47
![Page 26: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/26.jpg)
Uniform random variables on general (bounded)domain
Sketch of the proof of the proposition:
P(ν1 = k) = P(Z1 6∈ D)k−1P(Z1 ∈ D) = α(1− α)k−1; k ≥ 1.
P(Zν1 ∈ A) =∞∑
k=1P(Zν1 ∈ A ∩ ν1 = k) = P(Z1∈A∩D)
P(Z1∈D) .
By induction we have
P
(n⋂
i=1
Zνi ∈ Ai
)=
n∏i=1
P(Z1 ∈ Ai ∩ D)
P(Z1 ∈ D).�
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 26 / 47
![Page 27: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/27.jpg)
Uniform random variables on particular domain
PropositionLet f be a density on Rd and
Df = {(x , u) ∈ Rd × R+ | 0 ≤ u ≤ f (x)}.
Let U ↪→ U([0, 1]) and X be a random variable independent of U with densityf . Then, (X , Uf (X )) ↪→ U(Df ).
Proof: Exercise.
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 27 / 47
![Page 28: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/28.jpg)
Uniform random variables on particular domain
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
−3 −2 −1 0 1 2 3
0.0
0.1
0.2
0.3
0.4
Figure: On the domain
(x , u) ∈ R× R+ | 0 ≤ u ≤q
12π
e−x2
2
ff, realization of 1000
uniform random variables
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 28 / 47
![Page 29: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/29.jpg)
Rejection method (I)
PropositionLet f be a density on Rd and
Df = {(x , u) ∈ Rd × R+ | 0 ≤ u ≤ f (x)}.
Let (X , U) be a random variable on Rd × R+, then,
(X , U) ↪→ U(Df ) ⇔ X has density f and ∀x ∈ R+, U | x ↪→ U([0, f (x)]).
Proof: One has1Df (x , u) = f (x)
1f (x)
1[0,f (x)](u).�
Exercise: When Z is a random variable that have a density fX with compactsupport and bounded, find an algorithm to simulate it.
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 29 / 47
![Page 30: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/30.jpg)
Rejection method (II)
Aim: We want to simulate a random variable X with a known density fX .
Pb: DfX = {(x , u) ∈ Rd × R+ | 0 ≤ u ≤ fX (x)} may be unbounded...
Idea: Find another random variable Y easy to simulate with a known densityfY such that
fX ≤ cfY ; c ∈ R.
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 30 / 47
![Page 31: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/31.jpg)
Rejection method (II)
Let f de a density on R and g another density such that f ≤ cg.
PropositionLet (Zn)n≥1 a sequence i.i.d random variables with density g and (Un)n≥1 asequence of i.i.d uniform random variables on [0, 1] independent of (Zn)n≥1.We define
ν1 = inf (k ≥ 1; f (Zk ) > cUk g(Zk ))
Y = Zν1 .
Then Y has the density f .
Moreover E [ν1] = c.
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 31 / 47
![Page 32: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/32.jpg)
Rejection method (II)
Proof of the proposition:
P(f (Zk ) > cUk g(Zk )) = 1c
P(ν1 = k) = (1− 1c )k−1 1
c ; k ≥ 1.
P(Y ≤ t) =∞∑
k=1(1− 1
c )k−1∫ t−∞ g(z)
∫ f (z)cg(z)
0 dudz =∫ t−∞ f (z)dz.
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 32 / 47
![Page 33: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/33.jpg)
Rejection method (II)
Application 1: From Laplace to Gaussian distribution
Remind that the density of the Laplace distribution is given by
g(x) =12
e−|x|
and that
1√2π
e−x22 ≤
√2eπ
12
e−|x|.
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 33 / 47
![Page 34: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/34.jpg)
Rejection method (II)
The following algorithm generate a standard gaussian random variable X :
U < −− Random; V < −− Random; W < −− Random
if U < 12 then X < −− (−log(V)) else X < −− log(V)
while 1√2π
e−X22 ≤ W
√2eπ
12 e−|X| do
U < −− Random; V < −− Random; W < −− Random
if U < 12 then X < −− (−log(V)) else X < −− log(V)
end
X
Average number of simulation√
2eπ ≈ 1.31
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 34 / 47
![Page 35: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/35.jpg)
Rejection method (II)
Exercise: a) Find an algorithm to generate a gaussian distribution from acauchy one.
b) Is it possible to do the converse?
Exercise: a) Show that f (x) = eλ−x−1eλ−1−λ
is a density on R.
b) Show that f (x) ≤ λ−xλ
eλ−1eλ−1−λ
.
c) Find an algorithm to simulate a random variable with density f .
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 35 / 47
![Page 36: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/36.jpg)
Rejection method (II)
Application 2: The G(a, 1) distribution (0 < a < 1)
The density of the gamma G(a, 1) distribution is given by
f (x) =xa−1e−x
Γ(a)1R+(x)
and we can show that for 0 < a < 1
f (x) ≤ a + eaeΓ(a)
g(x)
where g is a density given by
g(x) =ae
a + e[xa−11[0,1[(x) + e−x1x≥1.
]
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 36 / 47
![Page 37: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/37.jpg)
Rejection method (II)
The distribution function G associated to g is explicitly known and G−1 isgiven on ]0, 1[ by
G−1(z) =
[a + e
ez] 1
a
1{z< ea+e} − log
[a + e
ae(1− z)
]1{z≥ e
a+e}.
Thus, we may
generate a random variable of density g by the inversion method,
generate a G(a, 1) by the rejection method.
Exercise: a) How to generate a general G(a, b) (a > 0, b > 0) distribution?
b) How to generate a β(a, b) (a > 0, b > 0) distribution?
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 37 / 47
![Page 38: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/38.jpg)
Rejection method (II)
0 2 4 6 8 10
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
Figure: Empirical density of 100000 realizations of a G(1, 0.5) by rejection method
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 38 / 47
![Page 39: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/39.jpg)
Plan
1 Simulation of random variablesInversion of the distribution functionTransformation methodRejection methodSimulation of Gaussian vectorsSimulation of conditional distribution
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 39 / 47
![Page 40: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/40.jpg)
Simulation of Gaussian vectors
DefinitionLet X = (X1, ..., Xn) be a random vector of Rn, X is said to be a Gaussianvector if for all x = (x1, ..., xn) in Rn, the real random variable < x , X > isGaussian.
PropositionIf X is a Gaussian vector of Rn then, ∀x ∈ Rn,
ΦX (x) = ei<x,m>e−xt Σx
2
wherem = (E [X1], ..., E [Xn])
andΣ = [cov(Xi , Xj)]1≤i,j≤n.
In this case, we say that X follows a N (m,Σ).
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 40 / 47
![Page 41: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/41.jpg)
Simulation of Gaussian vectors
PropositionLet G = (G1, ..., Gn) be a n-sample of a N (0, 1), m ∈ Rn and A be a n × nmatrix. Then
m + AG ↪→ N (m, AAt).
Algorithm to generate a N (m,Σ)
Generate G = (G1, ..., Gn) (for example by Box-Muller).Find A such that AAt = Σ (Cholesky decomposition).Compute m+A G .
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 41 / 47
![Page 42: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/42.jpg)
Simulation of Gaussian vectors
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●
●
●
●●●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●●
●● ●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
● ●
●
●
●
● ●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
● ●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●
● ● ●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
● ●●
●
●
●
●
●
●
●
●
●●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
● ●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
● ●
●
● ●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●●
●●●
●●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●● ●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●● ●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●●
●
●
●
●
● ●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●
●
●
●
●
●
●●
●
● ●
●
●
●●
●
●
● ●
●
●
●
●
●
●●
●
●
● ●
● ●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●●
●
● ●
●
●
●●
●
●●● ●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●●
●
● ●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●●
●
●
● ●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
● ●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●●
● ●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●● ●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●●
●
●
●
●●
●
●
●●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
● ●
●●
●
●
● ●
●
● ●
●
●
●
●
●
●● ●
●
●●
●
●
●
●
●
●
● ●●●
●
●●
●
●
●
●●
●●●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●●
●
●●
●
●●●
●
●
●
●
●
●
●
●● ●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
● ●
●
●●●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●
●●
●●
●
●
●●●
● ●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●●
●
●●●●
●
●●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●● ●
●
●
●
●
●
● ●
●●●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
● ●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
●●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●●
●
●
●
● ●
●●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●●
●
●
●●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●●
●
● ●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●●●
●
● ●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●
●
●
●●
●
●
●●
●●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
● ●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
● ●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
● ●●●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
● ●●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●●
● ●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
● ●
●
●
●
●
●●
●●
●●
●
●
●
●●
●
●
●
●
●● ●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
● ●
●
●
●
●●
●
●●
●
●
●
● ●
●
●
●
●
● ●
●
●
●●● ●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●●
●
● ●
●●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
● ●●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
● ●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
● ●
● ●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●●●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●●●
●●
●●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●●
●●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●● ●
●
● ●●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●●
●●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●●
●
● ●
●
●
●
●●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●●
● ●
●
● ●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●● ●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●●●
●●
●
●
●
●
● ●
●
●
●
●● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●●
●
●● ●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
● ●
●
●
●●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●●
●●●
●
●● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●●●
●● ●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
● ●
●
●
●
●●
●●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●●
●●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
● ●
●
●
●
●●●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
● ●
● ●
●●
●
●
●
●
●
●
●● ●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
● ●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●●
●
●●●
●
● ●
●
●
●
●
●
●
●
●
●●●
● ●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●● ●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●●
●
●●
●●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
● ●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●
●●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●●
●
● ●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●●
●●
●●
●
●●
●
●
●
●
●
● ● ●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●● ●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●●
●●
●
●
●
●
●
●●
● ●
●
●
●
●
● ●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
● ● ●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ●
●●
●
●●
●●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●●
●
●
●
●
● ●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
● ●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●●● ●●
●
●
●
●
●●
●●
●●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
● ● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●●
●
●
●● ●
●
●
●●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
● ●●
●
●
●
●●●
●
●●
●
● ●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●●
●●
●
●
●●
●●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
● ●
●
●●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●● ● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
● ●●●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
● ●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
● ●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●●
●
●● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●●
●
●
●
●
● ●
●●
● ●● ●●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
● ●
●
●
●●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
● ●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●●
●●
●
●
●●
●
●
● ●●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●●
●●
●●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●● ●
●●
●●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●
●
●
●●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
● ●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
● ●
●●
●
●
●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●● ●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
● ●
●●●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●● ●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
● ●
●
●
●
●
●●
●●
●
●
●
● ●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
● ●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
● ●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●●
●
●
●
●●
●
● ●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●●
●●
●
●●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
−4 −2 0 2 4
−3
−2
−1
01
23
Figure: 10000 realisations of a gaussian vector of covariance matrix
1116
7√
316
7√
316
2516
!
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 42 / 47
![Page 43: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/43.jpg)
Plan
1 Simulation of random variablesInversion of the distribution functionTransformation methodRejection methodSimulation of Gaussian vectorsSimulation of conditional distribution
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 43 / 47
![Page 44: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/44.jpg)
Conditional distribution
We consider a random variable X with density fX . For a < b such thatP(a < X < b) > 0 and g : R → R bounded, we want to evaluate
E [g(X ) | a < X < b] =E [g(X )1{a<X<b}]
P(a < X < b).
First Idea: Compute the numerator and the denominator by Monte Carlomethods.
Pb: In practice P(a < X < b) > 0 is very small ⇒ High number ofsimulations...
Second Idea: Find Y with a distribution equals to the conditional distribution ofX given {a < X < b} and use MC Method.
Pb: How to find Y?
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 44 / 47
![Page 45: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/45.jpg)
Simulation of conditional distribution by rejectionmethod
PropositionLet (Zn)n≥1 a sequence i.i.d random variables with density fX . We define
ν = inf (k ≥ 1; a < Zk < b)
Y = Zν .
Then Y follows the conditional distribution of X given {a < X < b}.
Moreover E [ν] = 1P(a<X<b) .
Pb: In practice P(a < X < b) > 0 is very small ⇒ High number ofsimulations...
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 45 / 47
![Page 46: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/46.jpg)
Simulation of conditional distribution by inversion
It is easy to see that the distribution of Y has to be equal to
FY (y) =
0 if y ≤ a
FX (y)−FX (a)FX (b)−FX (a) if a < y < b
1 if y ≥ b.
PropositionIf U ↪→ U([0, 1]), then
F−X (FX (a) + (FX (b)− FX (a))U)
follows the conditional distribution of X given {a < X < b}.
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 46 / 47
![Page 47: Introduction to Monte Carlo Methods (Lecture 2)christophe.chorro.fr/docs/MC2.pdf · When X is finite we have the following algorithm: p = p 0, j = 0 while p < U do j = j+1, p = p+p](https://reader034.vdocument.in/reader034/viewer/2022050607/5fae8cbf25315b04f91af37c/html5/thumbnails/47.jpg)
Inversion Method
0.0 0.5 1.0 1.5 2.0 2.5 3.0
0.0
0.5
1.0
1.5
2.0
2.5
3.0
Figure: Empirical density of 10000 realization of X given 1 < X < 2 (by the inversionmethod) when X ↪→ E(3)
Christophe Chorro ([email protected]) (University Paris 1)Introduction to Monte Carlo Methods (Lecture 2) June 19 2008 47 / 47