Download - Bayesian static parameter estimation using Multilevel ...mcqmc2018.inria.fr/wp-content/uploads/2018/07/law_pmcmc-mcqmc18.pdf · normalizing constant [HSS13, DKST15, HTL16, BJLTZ17]

Introduction MLMC BIP Coupling PMCMC PMC(ML)MC Numerical simulations Summary

Bayesian static parameter estimation usingMultilevel Monte Carlo

Kody Law

joint with A. Jasra (NUS), K. Kamatani (Osaka), & Y. Zhou

MCQMC 2018Rennes, FR

July 4, 2018


Outline

1 Introduction

2 Multilevel Monte Carlo sampling

3 Bayesian inference problem

4 Approximate coupling

5 Particle Markov chain Monte Carlo

6 Particle Markov chain Multilevel Monte Carlo

7 Numerical simulations

8 Summary


Orientation

Aim: Approximate posterior expectations of the state path andstatic parameters associated to a discretely observed diffusion,which must be finitely approximated.Solution: Apply an approximate coupling strategy so that themultilevel Monte Carlo (MLMC) method can be used within aparticle marginal Metropolis-Hastings (PMMH) algorithm [B02,AR08, ADH10].

MLMC methods reduce cost to error= O(ε) [H00, G08];Recently this methodology has been applied to inference,mostly in cases where target can be evaluated up to anormalizing constant [HSS13, DKST15, HTL16, BJLTZ17].Here it cannot, but using PMMH we are able to sampleconsistently from an approximate coupling of successivetargets [JKLZ18].


Outline

1 Introduction







8 Summary


Example: expectation for SDE [G08]

Estimation of expectation of solution of intractable stochasticdifferential equation (SDE).

dX = f (X )dt + σ(X )dW , X0 = x0 .

Aim: estimate E(g(XT )).We need to(1) Approximate, e.g. by Euler-Maruyama method with

resolution h:

Xn+1 = Xn + hf (Xn) +√

hσ(Xn)ξn, ξn ∼ N(0,1).

(2) Sample {X (i)NT}Ni=1, NT = T/h.


Multilevel Monte Carlo [H00, G08, CGST11]

Assume hl = 2−l , Xl approximates XT with stepsize hl , andthere are α, β, ζ > 0 such that

(i) weak error |E[g(Xl)− g(X )]| = O(hαl ).

(ii) strong error E|g(Xl)− g(X )|2 = O(hβl )⇒ Vl = O(hβl ),(iii) computational cost for a realization of g(Xl)− g(Xl−1),

Cl ∝ h−ζl .Then, it is possible to choose L and {Nl}Ll=0 so as to achieve amean squared error of O(ε2) at a cost

COST ≤ C

ε−2, if β > γ,

ε−2| log(ε)|2, if β = γ,

ε−(2+ γ−βα ), if β < γ.

Compared with O(ε−2−γ/α) for single level.


Outline

1 Introduction







8 Summary


Parameter inference

Estimate the posterior expectation of a function ϕ of the jointpath X1:T and parameters θ, of an intractable SDE

dX = fθ(X )dt + σθ(X )dW , X0 ∼ µθ ,

given noisy partial observations

Yn ∼ gθ(Xn, ·) , n = 1, . . . ,T .

Aim: estimate E[ϕ(θ,X0:T )|y1:T ], where y1:T := {y1, . . . , yT}.

The hidden process {Xn} is a Markov chain.

Discretize with resolution h and denote the transition kernelFθ,h

(xp−1, dxp

)– this can be simulated from, but its density

cannot be evaluated.


Return to ML

The joint measure is

Πh(dθ, dx0:n) ∝ Π(dθ)µθ(dx0)n∏

p=1

gθ(xp, yp)Fθ,h(xp−1, dxp) ,

For +∞ > h0 > · · · > hL > 0, we would like to compute

EΠhL[ϕ(θ,X0:n)] =

L∑l=0

{EΠhl

[ϕ(θ,X0:n)]− EΠhl−1[ϕ(θ,X0:n)]

}where EΠh−1

[·] := 0.


Outline

1 Introduction







8 Summary


Approximate coupling

Consider a single pair EΠh [ϕ(θ,X0:n)]− EΠh′ [ϕ(θ,X0:n)], h < h′.

Let z = (x , x ′) and let Qθ,h,h′(z, dz) be a coupling of(Fθ,h(x , dx),Fθ,h′(x ′, dx ′)).

Let Gp,θ(z) = max{gθ(x , yp),gθ(x ′, yp)}.

We will sample from the joint coarse/fine filter

Πh,h′(dθ, dz0:n) ∝ Π(dθ)νθ(dz0)n∏

p=1

Gp,θ(zp)Qθ,h,h′(zp−1, dzp),

where νθ is the initial coupling

νθ(d(x , x ′)) = µθ(dx)δx (dx ′).


Change of measure

We haveEΠh [ϕ(θ,X0:n)]− EΠh′ [ϕ(θ,X ′0:n)] =

EΠh,h′ [ϕ(θ,X0:n)H1,θ(θ,Z0:n)]

EΠh,h′ [H1,θ(θ,Z0:n)]−

EΠh,h′ [ϕ(θ,X ′0:n)H2,θ(θ,Z0:n)]

EΠh,h′ [H2,θ(θ,Z0:n)]

where

H1,θ(θ, z0:n) =n∏

p=1

gθ(xp, yp)

Gp,θ(zp)

H2,θ(θ, z0:n) =n∏

p=1

gθ(x ′p, yp)

Gp,θ(zp).


Outline

1 Introduction







8 Summary


Particle filter, for fixed θ

Let M ≥ 1 and θ be fixed, and introduce a0:n−1 ∈ {1, . . . ,M}n.The bootstrap particle filter [Del04] approximates

Πh,h′(dz0:n|θ) ∝ νθ(dz0)n∏

p=1

Gp,θ(zp)Qθ,h,h′(zp−1, dzp)

by sampling from

P(a1:M0:n−1, dz1:M

0:n |θ) =( M∏

i=1

νθ(dz i0)) n∏

p=1

M∏i=1

( Gp−1,θ(zai

p−1p−1 )∑M

j=1 Gp−1,θ(z jp−1)

Qθ,h,h′(zai

p−1p−1 , dz i

p)),

where G0,θ := 1,

i.e. zai

p−1p−1 is resampled with the probability

Gp−1,θ(zaip−1

p−1 )∑Mj=1 Gp−1,θ(z j

p−1).


Unbiased estimator, for fixed θ

Draw J with probability proportional to Gn,θ(z in) for i = 1, . . . ,M.

Let zn = z jn, and trace its ancestral lineage

zn−1 = zaj

n−1n−1 , zn−2 = z

aajn−1

n−2n−2 , and so on .

Define pMh,h′(y0:n|θ) =

∏np=1

(1M∑M

j=1 Gp,θ(z jp))

, let E denote

expectation w.r.t. (A1:M0:n−1,Z

1:M0:n , J), and note that [Del04]

E[ϕ(z0:n)pM

h,h′(y0:n|θ)]

=

∫Zn+1

ϕ(z0:n)νθ(dz0)n∏

p=1

Gp,θ(zp)Qθ,h,h′(zp−1, dzp) .

Therefore, one can run a MH chain {θi , z0:n(θi)} targeting∝ pM

h,h′(y0:n|θ)Π(dθ), and it is consistent with respect toΠh,h′(dz0:n, dθ) [B03, AR08, ADH10].


Particle marginal MH (PMMH) [ADH10]

1 Sample θ0 ∼ π(dθ) and (a1:M0:n−1, z

1:M0:n ) from particle filter

P(da1:M0:n−1, dz1:M

0:n |θ0), and store pMh,h′(y0:n|θ0).

2 Select a path z00:n: draw z j

n with probability proportional to Gn,θ0 (z jn), let

z0n = z j

n, and trace back its ancestral lineage

z0n−1 = z

ajn−1

n−1 , z0n−2 = z

aajn−1

n−2n−2 , and so on ; Set i = 1 .

3 Sample θ∗|θi−1 according to R(dθ∗|θi−1) = r(θ∗|θi−1)dθ∗, then samplefrom particle filter P(da1:M

0:n−1, dz1:M0:n |θ∗). Select one path z∗0:n as above.

4 Set θi = θ∗, z i0:n = z∗0:n with probability:

min

{1,

pMh,h′(y0:n|θ∗)

pMh,h′(y0:n|θi−1)

π(θ∗)r(θi−1|θ∗)π(θi−1)r(θ∗|θi−1)

}

otherwise θi = θi−1, z i0:n = z i−1

0:n .5 Set i = i + 1 and return to the start of 3.


PMMH increment estimator

1N∑N

i=1 ϕ(θi , x i0:n)H1,θ(θi , z i

0:n)1N∑N

i=1 H1,θ(θi , z i0:n)

−1N∑N

i=1 ϕ(θi , x ′i0:n)H2,θ(θi , z i0:n)

1N∑N

i=1 H2,θ(θi , z i0:n)

.

−→N→∞

EΠh,h′ [ϕ(θ,X0:n)H1,θ(θ,Z0:n)]

EΠh,h′ [H1,θ(θ,Z0:n)]−

EΠh,h′ [ϕ(θ,X ′0:n)H2,θ(θ,Z0:n)]

EΠh,h′ [H2,θ(θ,Z0:n)]

=

EΠh [ϕ(θ,X0:n)]− EΠh′ [ϕ(θ,X ′0:n)]


Outline

1 Introduction







8 Summary


Multilevel estimatorConsider

L∑l=0

ENll (ϕ) , ENl

l (ϕ) = ENll (ϕ)− El(ϕ) , (1)

where

ENll (ϕ) =

1Nl

∑Nli=1 ϕ(θi , , x i

0:n)H1,θ(θi , z i0:n)

1Nl

∑Nli=1 H1,θ(θi , z i

0:n)−

1Nl

∑Nli=1 ϕ(θi , x ′i0:n)H2,θ(θi , z i

0:n)

1Nl

∑Nli=1 H2,θ(θi , z i

0:n)

is a consistent estimator of

El(ϕ) := EΠhl[ϕ(θ,X0:n)]− EΠhl−1

[ϕ(θ,X0:n)] .

⇒ (1) is a consistent estimator of EΠhL[ϕ(θ,X0:n)].

One must bound

E[(L∑

l=0

ENll (ϕ))2] =

L∑l=0

E[ENll (ϕ)2] .


Assumptions

(A1) ∀ y ∈ T, ∃ C > 0 such that ∀ x ∈ S, θ ∈ Θ,

C ≤ gθ(x , y) ≤ C−1.

And ∀ y ∈ T, gθ(x , y) is globally Lipschitz on S×Θ.

(A2) ∀ 0 ≤ k ≤ n, ∃ β > 0 such that ∀ϕ ∈ Bb(Θ× Sk+1) ∩ Lip(Θ× Sk+1) ∃ C > 0∫

Θ×S2k+2|ϕ(θ, x0:k )−ϕ(θ, x ′0:k )|2Π(dθ)νθ(dz0)

k∏p=1

Qθ,h,h′(zp−1, dzp) ≤ C(h′)β .

(A3) Suppose that ∀ n > 0, ∃ ξ ∈ (0,1) and ν ∈ P(W) such thatfor each w ∈W, ϕ ∈ Bb(W) ∩ Lip(W), h,h′:∫

Wϕ(w ′)K (w ,dw ′) ≥ ξ

∫Wϕ(v)ν(dv).

K is η-reversible, where η is the joint on the extended space.


Main result

Theorem (JKLZ18)Assume (A1-3). Then ∀ n > 0, ∃ β > 0 such that ∀ϕ ∈ Bb(Θ× Sn+1) ∩ Lip(Θ× Sn+1) ∃ C > 0 such that

E[ENll (ϕ)2] ≤

ChβlNl

,

and β is from (A2).


(A4) ∃ γ, α,C > 0 such that the cost to simulate ENll is controlled by

C(ENll ) ≤ CNlh

−γl , and the bias is controlled by

|EΠhL(ϕ(θ,X0:n))− EΠ0 (ϕ(θ,X0:n))| ≤ ChαL .

Corollary

Assume (A1-4). ∀ n > 0 and ϕ ∈ Bb(Θ× Sn+1) ∩ Lip(Θ× Sn+1) ∃C > 0 such that ∀ ε > 0 one can choose (L, {Nl}L

l=0) so

E

[|

L∑l=0

ENll (ϕ)− EΠ0 (ϕ(θ,X0:n))|2

]≤ Cε2 ,

with a total cost (per time step)

COST ≤ C

ε−2, if β > γ,

ε−2| log(ε)|2, if β = γ,

ε−(2+ γ−βα ), if β < γ.


Outline

1 Introduction







8 Summary


Ornstein-Uhlenbeck process

dXt = θ(µ− Xt )dt + σdWt , X0 = x0 ,

Yk |Xδk ∼ N (Xδk , τ2) ,

θ ∼ G(1,1) ,

σ ∼ G(1,0.5) .

N (m, τ2) denotes the Normal with mean m and varianceτ2.G(a,b) denotes the Gamma with shape a and scale b.x0 = 0, µ = 0, δ = 0.5, and τ2 = 0.2.100 observations simulated with θ = 1 and σ = 0.5.


Langevin SDE

dXt =12∇ log π(Xt )dt + σdWt , X0 = x0

Yk |Xk ∼ N (0, τ2 exp Xk ) ,

θ ∼ G(1,1) ,

σ ∼ G(1,0.5) .

π(x) denotes the probability density function of a Student’st-distribution with θ degrees of freedom.x0 = 0.1,000 observations simulated with θ = 10, σ = 1, andτ2 = 1.


Langevin SDE Ornstein-Uhlenbech process

0 10 20 30 0 10 20 30

0.00

0.25

0.50

0.75

1.00

Lag

ACF

Parameter 𝜎 𝜃

Figure: Autocorrelation of a typical PMCMC chain.


Lagenvin SDE

𝜎

Lagenvin SDE

𝜃

Ornstein-Uhlenbech process

𝜎

Ornstein-Uhlenbech process

𝜃

2−8 2−7 2−6 2−5 2−4 2−3 2−2 2−5 2−4 2−3 2−2 2−1 20

2−8 2−7 2−6 2−5 2−4 2−3 2−6 2−5 2−4 2−3 2−2 2−122242628210

22242628210

22242628210

22242628210

Error

CostAlgorithm ML-PMCMC PMCMC

Figure: Cost vs. MSE for the 2 parameters for each of the 2 SDEs.


Model Parameter ML-PMCMC PMCMCOU θ −1.022 −1.463

σ −1.065 −1.522Langevin θ −1.060 −1.508

σ −1.023 −1.481

Table: Estimated rates of convergence of MSE with respect to cost forvarious parameters, fitted to the curves.


Outline

1 Introduction







8 Summary


Summary

New approximate coupling strategy can be used to applyMLMC to PMCMC for static parameter estimation[JKLZ18.i].Same strategy can be employed for multi-index MCMC[JKLZ18.ii].In progress: MISMC2 [JLX18.s]PhD and postdocs wanted – please inquire !


References[G08]: Giles. "Multilevel Monte Carlo path simulation." Op.Res., 56, 607-617 (2008).[H00] Heinrich. "Multilevel Monte Carlo methods." LSSCproceedings (2001).[CGST11] Cliffe, Giles, Scheichl, & Teckentrup. "MLMCand applications to elliptic PDEs." Computing andVisualization in Science, 14(1), 3 (2011).[D04]: Del Moral. "Feynman-Kac Formulae." Springer:New York (2004).[B02] Beaumont. "Estimation of population growth..."Genetics 164(3) 1139-1160 (2003).[AR08] Andrieu & Roberts. "The pseudo-marginalapproach." Annals of Stat. 37(2) 697-725 (2009).[ADH10] Andrieu, Doucet, and Holenstein. "ParticleMCMC methods." JRSSB 72(3) 269-342 (2010).


References[JKLZ18.i]: Jasra, Kamatani, Law, Zhou. "MLMC for staticBayesian parameter estimation." SISC 40, A887-A902(2018).[JKLZ18.ii]: Jasra, Kamatani, Law, Zhou. "A Multi-IndexMarkov Chain Monte Carlo Method." IJUQ 8(1), 61-73(2018).[JLX18.s]: Jasra, Law, Xu. "Multi-index SMC2." Submitted.[BJLTZ15]: Beskos, Jasra, Law, Tempone, Zhou."Multilevel Sequential Monte Carlo samplers." SPA 127:5,1417–1440 (2017).[HSS13]: Hoang, Schwab, Stuart. Inverse Prob., 29,085010 (2013).[DKST13]: Dodwell, Ketelsen, Scheichl, Teckentrup. "Ahierarchical MLMCMC algorithm." SIAM/ASA JUQ 3(1)1075-1108 (2015).


References[CHLNT17] Chernov, Hoel, Law, Nobile, Tempone."Multilevel ensemble Kalman filtering for spatio-temporalprocesses." arXiv:1608.08558 (2017).[JLS17] Jasra, Law, Suciu. "Advanced Multilevel MonteCarlo Methods." arXiv:1704.07272 (2017).[DJLZ16]: Del Moral, Jasra, Law, Zhou. "MultilevelSequential Monte Carlo samplers for normalizingconstants." ToMACS 27(3), 20 (2017).[JKLZ15]: Jasra, Kamatani, Law, Zhou. "Multilevel particlefilter." SINUM 55(6), 3068–3096 (2017).[JLZ16]: Jasra, Law, and Zhou. "Forward and Inverse UQMLMC Algorithms for an Elliptic Nonlocal Equation." IJUQ6(6), 501–514 (2016).[HLT15]: Hoel, Law, Tempone. "Multilevel ensembleKalman filter." SINUM 54(3), 1813–1839 (2016).


Thank you

Download - Bayesian static parameter estimation using Multilevel ...mcqmc2018.inria.fr/wp-content/uploads/2018/07/law_pmcmc-mcqmc18.pdf · normalizing constant [HSS13, DKST15, HTL16, BJLTZ17]

Top Related