agu 2012 bayesian analysis of non gaussian lrd processes
DESCRIPTION
Contributed talk at American Geophysical Union Fall Meetinfg, San Francisco, 2012. Part of work now submitted to Bayesian Analysis, 2014, see eprint: http://arxiv.org/abs/1403.2940TRANSCRIPT
Bayesian Analysis of Non-Gaussian Long range Dependent Processes
Tim Graves (Statistics Laboratory, Cambridge) Christian Franzke (BAS, Cambridge)
Bobby Gramacy (Booth School of Business, Chicago) & Nick Watkins (BAS, LSE & Warwick) [[email protected]]
NG22A-04 11am Tuesday 4th December 2012 Scaling and Correlations and their use in forecasting Natural Hazards I Room 300 Moscone South
Summary: 1. Standard climate noise models have short range dependence
(SRD). Recent evidence of long range dependence in surface temperatures. Example is Antarctic study [Franzke, J. Climate, 2010]. LRD hampers trend identification and quantification of significance.
2. LRD idea originated at same time as H-selfsimilarity, so not always realised that a model doesn’t need to be H-ss to show LRD, e.g. ARFIMA, [Watkins, GRL Frontiers, 2013].
3. Graves PhD has developed MCMC method to perform Bayesian inference on ARFIMA(p,d,q). Treated Gaussian ARFIMA first, tested on model (& real) data. Study dependence of posterior variance of inferred d on length of time series.
4. However, many real datasets not Gaussian. ARFIMA can allow alpha stable innovations. Graves has modified method to allow joint inference.
5 December 2012 2
Short & long range dependence
5 December 2012 3
( ) exp( )ρ τ λτ−
( ) ( )ρ τ δ τ
Delta correlated white noise
2 1( ) dk ckρ −
0 1/ 2d< <
Exponentially correlated red AR(1), SRD
0( )
k
kkρ
=∞
=
= ∞∑LRD: power law correlated, “1/f” noise
LRD & Antarctic temperature trends
4
[Franzke, J. Climate, 2010] found evidence of LRD in station temperature series. 3 have significant EMD residual trend (dashes) against SRD null model. One station [Faraday] still significant against LRD.
LRD and selfsimilarity, common history ...
Mandelbrot (& Wallis), mid 1960s: 2 departures from AR(1), “Biblical geoscience” illustrations, selfsimilarity exponent H
• heavy tails in amplitude, cotton prices. “Noah” effect, 40 days and 40 nights of rain. • long range dependence in Nile level. 7 lean & 7 fat years: “Joseph effect”. 5 December 2012 5
LRD and selfsimilarity, common history ...
Mandelbrot (& Wallis), mid 1960s: 2 departures from AR(1), “Biblical geoscience” illustrations, selfsimilarity exponent H
• heavy tails in amplitude, cotton prices. “Noah” effect, 40 days and 40 nights of rain. • long range dependence in Nile level. 7 lean & 7 fat years: “Joseph effect”. 5 December 2012 6
LRD and selfsimilarity, common history ...
Mandelbrot (& Wallis), mid 1960s: 2 departures from AR(1), “Biblical geoscience” illustrations, selfsimilarity exponent H
• heavy tails in amplitude, cotton prices. “Noah” effect, 40 days and 40 nights of rain. • long range dependence in Nile level. 7 lean & 7 fat years: “Joseph effect”. 5 December 2012 7
… not necessarily common origin • Both fBm & Levy flights are H-selfsimilar, • Frac. Brownian motion: H (here = J) = d + ½ LRD (persistence): 0 < d < ½; “Hurst” exponent
increases from Brownian value of ½ as memory parameter d increases
• Levy flights: H = 1/α α is exponent of pdf heavy tail. • Both are limits of H = 1/ α +d • Many methods (e.g. R/S …) inspired by self-
similarity, & measure d, not α, via geometry. 8
Can model LRD (d) without assuming complete H-selfsimilarity Don’t actually need completely H-selfsimilar
models to exhibit LRD (just asymptotic) In 1980s Granger and Joyeux modified SRD Auto
Regressive Moving Average [ARMA(p,q)] models to allow LRD via Fractional Integration of order d [ARFIMA(p,d,q)].
Physically interesting: High frequency p term(s) that turns nonstationary, H-ss random walk into weakly stationary AR(p) i.e. dissipation
5 December 2012 9
AR(1): 1st order AutoRegressive
5 December 2012 10
1 1t t tX Xφ ε−= +
0 100 200 300 400 500 600 700 800 900 1000-8
-6
-4
-2
0
2
4
6
8Example series of AR(1)
1 0.9φ =
AR(1): 1st order AutoRegressive
5 December 2012 11
1(1( ) ) t tB B Xφ εΦ − ==
1 1t t tX Xφ ε−= +
1ttBX X −=0 100 200 300 400 500 600 700 800 900 1000
-8
-6
-4
-2
0
2
4
6
8Example series of AR(1)
1 0.9φ =
1( ) 1
pj
jj
z zφ=
Φ = −∑
AutoRegressive Fractionally Integrated Moving Average
[ARFIMA(p,d,q)]
5 December 2012 12
( )(1 ) ( )dt tB B X B εΦ − = Θ
Autoregressive term of order p
Moving average of order q
1( ) 1
qj
jj
z zθ=
Θ = +∑
Fractional integration of order d
Granger (& Joyeux), 1980
(1 )dt tB X ε− =Pure LRD
ARFIMA(0,d,0 ):
Exact Bayesian inference on ARFIMA for d
• ARFIMA has parameters μ, σ, d, φ, θ. All but d essentially nuisance parameters here.
• First assume Gaussian innovations. • Assume flat priors for μ, log σ and d … • Even with this, likelihood for d very complex • No analytic posterior --- use MCMC sampling
5 December 2012 13
( | ) ( ) ( | )x p L xψ ψπ ψ ψ ψ∝
Key features
• Don’t want to assume form of p, q – use R J MCMC [Green, Biometrika 1995]
• Reparameterisation of model to enforce stationarity constraints on φ and θ.
• Efficient calculation of Gaussian likelihood (long memory correlation structure prevents use of standard quick methods)
• Necessary use of Metropolis-Hastings requires careful selection of proposal distribution
• Parameter correlation (φ,d) requires blocking 5 December 2012 14
“Calibration”
5 December 2012 15
~ 1/d nσ
• Have studied how posterior variance of d depends on sample size n, c.f. Kiyani et al, PRE, 2009. study of structure functions etc • Looked at standard test series like Nile river, find d of about .4 and ARFIMA(0,d,0) most probable model. Confims e.g Beran, 1994. • Looked at CET. Dependence more complicated and a model incorporating seasonality performs better.
Approximate inference in more general case
• Drop Gaussianity assumption. • Go to more general distribution (α-stable). • Seek joint inference on d, α • Approximate long memory process as very
high order AR • Construct likelihood sequentially • Use auxiliary variables to integrate out
unknown history 5 December 2012 16
Pure symmetric α-stable ARFIMA
5 December 2012 17
0.15(1 ) t tB X ε− = 1.5α =
Posterior estimates of d, α
5 December 2012 18
1.5α =0.15d =
Scatter of d and α
5 December 2012 19 1 1t t tX Xφ ε−= +
Good estimation of all parameters. Posteriors of d and α are independent.
Conclusions: 1. Standard climate noise model AR(1). Discretises Ornstein-Uhlenbeck
physics. Short range dependence (SRD). However recent evidence of long range dependence (d nonzero) in surface temperatures. Example is multistation Antarctic study [Franzke, J. Climate, 2010]. LRD hampers trend identification and quantification of significance.
2. LRD idea originated at same time as H-selfsimilarity. Not always realised in physics that model doesn’t need to be H-ss to show LRD, e.g. ARFIMA, [Watkins, GRL Frontiers, 2013]. Corollary is that SRD can blur classic LRD methods, range of methods desirable [Franzke et al, Phil. Trans. Roy. Soc A, 2012].
3. Graves PhD: Develop MCMC method to perform Bayesian inference on ARFIMA(p,d,q). Gaussian first, tested on model (& real) data.
4. However, many real datasets not Gaussian. ARFIMA can allow alpha stable innovations. Modify method to allow joint inference of d,alpha.
5. Study dependence of posterior variance of inferred d on length of time series.
5 December 2012 20
Models in Physics & Time Series Analysis
“Models in physics are like Austrian train timetables. Trains in Austria are always late, but without a timetable we wouldn’t know how late they are”.
--- attributed to Pauli, in Kleppner & Kolenkow, “An Introduction to Mechanics” “Remember that all models are wrong; the practical
question is how wrong do they have to be [in order] to not be useful”.
--- Box & Draper, “Empirical Model Building”
21