School of Earth and Environment
Sensitivity analysis and calibration of a global aerosol model
Lindsay Lee [email protected]
Ken Carslaw1, Carly Reddington1, Kirsty Pringle1, Jeff Pierce3, Graham Mann1, Jill Johnson1,
Dominick Spracklen1, Philip Stier2
1. University of Leeds 2. University of Oxford 3. Colorado State University
1. Motivation
http://earthobservatory.nasa.gov/Features/CALIPSO/CALIPSO2.php
2. GLOMAP
• We use the global aerosol model GLOMAP (Mann et al. 2010)
• A microphysical modal model simulating the evolution of global aerosol including sulphate, sea-salt, dust and black carbon
http://www.pnl.gov/atmospheric/research/aci/aci_aerosol_indeffects.stm
3. Uncertainty in modelling
• Structural uncertainty
• Compare multiple models of the same type
• Compare model to observations
• AEROCOM • Aerosol Comparisons between Observations and
Models
• >14 models with the same emissions for 2000 and pre-industrial
5
4. GLOMAP in AEROCOM
6
4. GLOMAP in AEROCOM
7
4. GLOMAP in AEROCOM
8
4. GLOMAP in AEROCOM
• More complex models = more uncertain parameters
• How do these uncertain parameters affect model predictions?
• Which processes in the model are dominating the prediction?
• Together with AEROCOM find the most important uncertainties
9
5. Parametric uncertainty
• Ideally want to carry out sensitivity analysis
• Need a good estimation of the output distribution given uncertain inputs
• Consider GLOMAP output Y = η(X)
• X is uncertain so Y is uncertain
• Want to know f(Y) given G(X)
• Monte Carlo sampling impossible
• Have to use
10
6. Parametric uncertainty in a complex computer model
)(ˆ Yf
• found using Bayesian statistics
• Use a limited number of model runs – training data
• Make some assumptions about the model behaviour – prior distribution
• Find the posterior distribution of the model – and use the mean as
11
7. Bayesian statistics
)(ˆ Yf
)(ˆ Yf
8. The procedure
9. The model output
• Focus on Cloud condensation nuclei (CCN) concentration
• Soluble aerosol (>50nm) form cloud droplets at a given supersaturation
• Uncertainty in CCN concentration due to: • uncertainty in the emission,
• nucleation of new aerosol,
• growth to 50nm,
• solubility,
• loss of aerosol.
10. Expert elicitation
Elicitation: Ask the experts
‘We think these are the uncertain parameters and their values are very unlikely to fall outside of these ranges’
11. Statistical design
– Need maximum information in fewest runs
– Space-filling in 28-dimensions
– Maximin Latin Hypercube used
–good marginal coverage
–good space-filling properties
– Number of runs 168 + 84 validation
– Emulator validation to highlight design issues
• Interpolate ‘well-spaced’ model runs to estimate at untried points
• Gaussian Process - conditional probability • Non-parametric
– Key assumption is that parameter settings give information about model behaviour close by in parameter space
12. Filling in the gaps - emulation
GP mean True Function
X1
X2
Linear Model
13. Filling in the gaps - emulation
Emulator mean
X2
X1
Emulated output distribution
Marginal functions X1 X2
Ou
tpu
t
Input
14. Emulator validation
Is the emulator output a good approximate of the model
output?
YES – use the emulator mean instead
15. Emulator validation I
16. Emulator validation II
17. Estimated CCN and its uncertainty concentration in every surface grid box
January
July
18. Variance-based sensitivity analysis
ji
pij
i
i VVVYVar 12
1
)(
)),|(( iXi XYEVarVi
))|(( ijXij XYEVarVij
)(YVar
VS i
i
112
1
ji
pij
i
i SSS
Main effect sensitivity:
Variance decomposition:
Main effects +
Interactions:
Variance due to each parameter:
Marginal functions
X1 X2
Input
Ou
tpu
t
Emulated output distribution
19. Variance-based sensitivity analysis
)),|(( iXi XYEVarVi
)(YVar
24
20. Contributions to uncertainty in every grid box – January CCN
25
20. Contributions to uncertainty in every grid box – January CCN
ACT_DIAM
AIT_WIDTH
ANTH_SOA
SO2O3_CLEAN
DRYDEP_ACC
BB_EMS
PRIM_SO4_DIAM
BB_DIAM
DMS_FLUX
Surface parameter sensitivities 21. Contributions to uncertainty in every grid box
January January July July
σCCN/µCCN σCCN
22. Summarising global maps
23. Seasonal parameter sensitivities
24. Multi-parameter analysis
24. Multi-parameter analysis
BIO_SOA
BL_NUC
25. Towards structural uncertainty and calibration
• Structural uncertainty
• Considering the model uncertainty is any model configuration near to observations
26. AEROCOM versus AEROS
ACT_DIAM BIO_SOA ANTH_SOA
27. Reduce global problem
Jan CCN
Jul CCN
• PCA and cluster analysis Similar patterns seen in both
28. Cluster analysis
Fossil fuel emissions
Model parameters – Structure and cloud processing
ACT_DIAM
AIT_WIDTH
SO2O3_CLEAN
BB_EMS
BB_DIAM
29. Summary
• Sensitivity analysis provides deeper understanding of model behaviour
• Quantifying parametric uncertainty can help identify structural errors/model errors/modeller errors
• Can help direct research areas/observation strategies
• Will help calibration
30. Calibration discussion points
• Calibration of global means not satisfactory • How do we calibrate the global model with known
structural errors? • Do we expect one set of calibrated parameters given we
know model behaviour is variable? • How can we use the sensitivity information?
– to reduce dimensions • regional and temporal information • identify active parameters
• How do we specify discrepancy? – Use AEROCOM – Use grid box variability
References
• Lee, L. A., Carslaw, K. S., Pringle, K. J., Mann, G. W., and Spracklen, D. V.: Emulation of a complex global aerosol model to quantify sensitivity to uncertain parameters, Atmos. Chem. Phys., 11, 12253-12273, doi:10.5194/acp-11-12253-2011, 2011.
• Lee, L. A., Carslaw, K. S., Pringle, K. J., and Mann, G. W.: Mapping the uncertainty in global CCN using emulation, Atmos. Chem. Phys., 12, 9739-9751, doi:10.5194/acp-12-9739-2012, 2012.
• Lee, L. A., Pringle, K. J., Reddington, C. L., Mann, G. W., Stier, P., Spracklen, D. V., Pierce, J. R., and Carslaw, K. S.: The magnitude and causes of uncertainty in global model simulations of cloud condensation nuclei, Atmos. Chem. Phys. Discuss., 13, 6295-6378, doi:10.5194/acpd-13-6295-2013, 2013.
• Mann, GW; Carslaw, KS; Spracklen, DV; Ridley, DA; Manktelow, PT; Chipperfield, MP; Pickering, SJ; Johnson, CE (2010) Description and evaluation of GLOMAP-mode: a modal global aerosol microphysics model for the UKCA composition-climate model, GEOSCI MODEL DEV, 3, pp.519-551. doi:10.5194/gmd-3-519-2010
• Partridge, D. G., Vrugt, J. A., Tunved, P., Ekman, A. M. L., Struthers, H., and Sorooshian, A.: Inverse modelling of cloud-aerosol interactions – Part 2: Sensitivity tests on liquid phase clouds using a Markov chain Monte Carlo based simulation approach, Atmos. Chem. Phys., 12, 2823-2847, doi:10.5194/acp-12-2823-2012, 2012.
• Roustant, O., Ginsbourger, D., and Deville, Y. (2010). DiceKriging: Kriging methods for computer experiments. R package version 1.1. http://CRAN.R-project.org/package=DiceKriging
• Bastos, L. S. and O'Hagan, A. (2009). Diagnostics for Gaussian process emulators. Technometrics 51, 425-438.
• Pujol, G. (2008). sensitivity: Sensitivity Analysis. R package version 1.4-0. • Saltelli, A., Chan, K., Scott, M. Editors, 2000, Sensitivity Analysis, John Wiley & Sons publishers, Probability
and Statistics series.
38