bayesian hierarchical modelling of rainfall ifd curves · 3 | bayesian hierarchical modelling of...
TRANSCRIPT
Bayesian Hierarchical Modelling of Rainfall IFD Curves Eric A. Lehmann | Aloke Phatak, Rex Lau, Joanne Chia, Mark Palmer
CSIRO Computational Informatics, Perth, Australia
HCSS Team Meeting, May 2014
2 |
Presentation overview
• Some background information • Intensity—frequency—duration (IFD) curves
• Bayesian modelling
• Etc.
• My experiences with BHM implementation • Computational issues
• Coding issues
• Etc.
• Discussion …
Bayesian Hierarchical Modelling of Rainfall IFD Curves
Bayesian Hierarchical Modelling of Rainfall IFD Curves 3 |
IFD curve example • Intensity (or depth)
How much rain falls (per unit of time).
• Duration Period over which the rain falls.
• Frequency How often we expect to observe such an event, e.g., level exceeded once in 100 years.
Background • Climate extremes (not just
means), under climate change
• Civil engineers, design of hydraulic infrastructures, ...
• Australian Rainfall & Runoff project (Engineers Australia)
Source: Australian Rainfall & Runoff (R87) (http://www.bom.gov.au)
Bayesian Hierarchical Modelling of Rainfall IFD Curves 4 |
Typical dataset
• 252 stations around the Greater Sydney region.
• Annual maxima extracted between 1960 and 2000.
• Pluviometer data registered at 5 min “resolution”.
• 12 different accumulation durations: 5 min to 72 h.
• Under certain conditions, the maximum value of a sequence of random variables has a generalized extreme value (GEV) distribution.
• If z is the annual max. rainfall:
with GEV parameters: location , scale , and shape .
• Calculate IFD curves based on estimated GEV parameters (distribution quantiles).
Bayesian Hierarchical Modelling of Rainfall IFD Curves 5 |
Modelling rainfall extremes
0,,,)(
1exp)(GEV
1
zz
Bayesian Hierarchical Modelling of Rainfall IFD Curves 6 |
Spatial modelling of extremes (BHM) Data model: Y GEV(, , )
Process model:
MVN{ XT , (, ) } log() MVN{ XT , (, ) } MVN{ XT , (, ) } where: [(,)]ij = exp( xi - xj / )
Priors: for {, , } IG(, ) Gamma(, ) MVN(,
)
Inference: model fitting via MCMC simulations (Metropolis-Hastings within Gibbs), 150’000 iterations; prototyping in R code (~0.9s per iteration).
parameters vary smoothly according to a spatial stochastic process
Data model: Y GEV(, , )
Process model:
MVN{ XT , (, ) } log() MVN{ XT , (, ) } MVN{ XT , (, ) } where: [(,)]ij = exp( xi - xj / )
Priors: for {, , } IG(, ) Gamma(, ) MVN(,
)
Inference: model fitting via MCMC simulations (Metropolis-Hastings within Gibbs), 150’000 iterations; prototyping in R code (~0.9s per iteration).
Bayesian Hierarchical Modelling of Rainfall IFD Curves 7 |
Spatial modelling of extremes (BHM)
Bayesian Hierarchical Modelling of Rainfall IFD Curves 8 |
Spatial modelling of extremes (BHM) Data model: Y GEV(, , )
Process model:
MVN{ XT , (, ) } log() MVN{ XT , (, ) } MVN{ XT , (, ) } where: [(,)]ij = exp( xi - xj / )
Priors: for {, , } IG(, ) Gamma(, ) MVN(,
)
Inference: model fitting via MCMC simulations (Metropolis-Hastings within Gibbs), 150’000 iterations; prototyping in R code (~0.9s per iteration).
parameters vary smoothly according to a spatial stochastic process
Bayesian Hierarchical Modelling of Rainfall IFD Curves 9 |
Bayesian IFD curves
2 different return periods gauged vs. ungauged
Example: 100-year return level/intensity of precipitation (mean and 95% credible/confidence intervals).
Bayesian Hierarchical Modelling of Rainfall IFD Curves 10 |
Spatial return levels
Example: 100-year return level/intensity of precipitation (mean and 95% credible interval).
Bayesian Hierarchical Modelling of Rainfall IFD Curves 11 |
BHM summary • Provide estimates of uncertainty • BHM allows us to borrow strength spatially and over durations • Provide estimates of extremes at gauged and ungauged locations • Unified and flexible framework ... Features / extensions to this framework
– Koutsoyiannis parameterisation: * /
– duration dependence for GEV parameter: d d / ( d + )
– r-largest order statistics (use top r maxima)
– combination of pluvio and daily data
– integration of different covariates
– influence of climate change (RCM/GCM outputs)
– use of max-stable approach ...
For more info, please request copy of MODSIM’2013 and HWRS’2014 papers...
Bayesian Hierarchical Modelling of Rainfall IFD Curves 12 |
BHM implementation issues
MCMC simulation • Coding relatively straightforward (once you’ve worked out the
maths...), though many bells and whistles to account for (BHM)
• Computationally intensive and memory hungry!
• Which software?... • BUGS / WinBUGS / openBUGS / JAGS / WBDev
• PyMC
• many functions in R: BRugs, R2WinBUGS, rbugs, rjags, R2jags, runjags, etc.
• R packages: mcmc, MCMCpack, coda, BOA, etc.
• LibBi, spTimer, etc.
Need for full flexibility (modelling & coding), previous BHM code available, avoid new SW learning curve, etc. own implementation “from scratch” in R...
Bayesian Hierarchical Modelling of Rainfall IFD Curves 13 |
BHM implementation in R
First tests: ~40 s. per iteration!... (~46 days per run, 100’000 iterations)
not great for prototyping!...
Profiling R code With the help of:
• Rprof: enable profiling of the execution of R code (built-in)
• R packages: profr, proftools, ggplot2, Rgraphviz
Identify “slow” sections of code, then consider optimisation, Rcpp, parallelisation, etc.
Bayesian Hierarchical Modelling of Rainfall IFD Curves 14 |
BHM implementation in R
Bayesian Hierarchical Modelling of Rainfall IFD Curves 15 |
BHM implementation in R
Bayesian Hierarchical Modelling of Rainfall IFD Curves 16 |
Bayesian Hierarchical Modelling of Rainfall IFD Curves 17 |
BHM implementation in R: latest code
Bayesian Hierarchical Modelling of Rainfall IFD Curves 18 |
BHM implementation in R: latest code
Bayesian Hierarchical Modelling of Rainfall IFD Curves 19 |
BHM implementation in R: latest code
Bayesian Hierarchical Modelling of Rainfall IFD Curves 20 |
BHM implementation in R
Speeding up your code without altering it: super-computers and optimised libraries...
Benchmark cross-product matrix
(b = a’ * a)
20 iterations of BHM code
Self time of %*% operations
Bragg-l 1.233s 2m 49s 65%
BANNISTER-FL 12.95s 4m 46s 65%
CMIS-08 16.48s 7m 26s 76%
CMIS1-PER 4.95s 10m 47s 74%
CSIRO Computational Informatics Dr. Eric A. Lehmann
T +61 8 9333 6123 E [email protected] W www.csiro.au
CSIRO COMPUTATIONAL INFORMATICS
Thank you – feedback welcome!...