global sensitivity analysis to support model calibration ... · to support model calibration,...
TRANSCRIPT
Global Sensitivity Analysis to support model calibration, evaluation, uncertainty propagation and robust decision-making: a toolbox for access to methods and workflows!Francesca Pianosi and Thorsten Wagener Department of Civil EngineeringUniversity of Bristol!! NE/J017450/1
credible.bris.ac.uk
Global Sensitivity Analysis is a set of statistical techniques that provide a structured approach to tackle several types of uncertaintyassociated with the development and application of numerical models!GSA is useful for: :: more efficient model calibration :: better understanding of model response:: prioritizing efforts for uncertainty reduction (e.g. acquisition of new data):: assessing robustness to modeling assumptions :: …!!
NUMERICAL MODEL!
resolution!
parameters!
forcing inputs!
structure(equations)!
FC!LP!BETA!
time!
output!
pdf!
forcing inputs!
parameters!
interactions!
resolution!
Global Sensitivity Analysis investigates how the variation in the output of a numerical model can be attributed to variations of its input factors!!!
Outline!
:: Key concepts underlying GSA techniques !:: Examples of GSA applications :: SAFE: a Matlab/Octave/R toolbox for GSA !
Steps of sampling-based GSA!
3 POST PROCESSING!
2 MODEL EVALUATION!
1 INPUT SAMPLING!
x1 x2 x3!
sens
itivi
ty!
!!
1 !!
0.5!!
0!
Sensitivity indices!
Latin-hypercubeQuasi-random sampling…!
FC!LP!BETA!output!
pdf!
Steps of sampling-based GSA!
3 POST PROCESSING!
2 MODEL EVALUATION!
1 INPUT SAMPLING!
x1 x2 x3!
sens
itivi
ty!
!!
1 !!
0.5!!
0!
Sensitivity indices!
Sensitivity measured by: :: multiple-start derivatives/differences (e.g. Morris method, DELSA, …)!
:: correlation between inputs and outputs !
:: properties of input distributions after conditioning outputs (Monte Carlo filtering)!!
:: properties of output distribution after conditioning inputs (e.g. Sobol’ method, density-based methods, …)!
Steps of sampling-based GSA!
3 POST PROCESSING!
2 MODEL EVALUATION!
1 INPUT SAMPLING!
x1 x2 x3!
sens
itivi
ty!
!!
1 !!
0.5!!
0!
Sensitivity indices!
!:: Computational cost of post-processingis negligible wrt to model evaluation !:: Required number of model evaluations grows with number of input factors !:: Growth rate differs from method to method!
Choice of GSA method: classification system!
Correlation & Regression Analysis!
Variance- based !&!Density-based!
Multiple-starts derivatives!
Monte-Carlofiltering!
Num
ber o
f mod
el e
valu
atio
ns! >1
0 x
M!
>100
x M
!>1
000
x M!
PAWN!
FAST!
VBSA (Sobol’)!
EET (Morris)!
Regional Sensitivity Analysis!
CART!
Screening! Ranking! Mapping!Specific purpose!
M = number of input factors!
Pianosi et al. 2016 Env.Mod&Soft!
implemented in the SAFEToolbox!
Supporting calibration of a land surface model!
!Which parameters mostly affect the model performance? Which parameters have littleinfluence and can be set todefault values?!
latent heat!
soil heat!
surface skin temperature!
soiltemperature!
sensibleheat!
water on canopy!
transpiration!
rainfall!
evaporation!
rainfall!
soilmoisture!
drainage!
evaporation!
runoff!
runoff!
ENERGY!WATER!
The Joint UK Land Environment Simulator (JULES)!
Application tothe Santa Rita creosotesite in the US!
with J. Iwema, R. Rosolem
Supporting calibration of a land surface model!
1-9: parameters: b sathh satcon sm_sat sm_crit sm_wilt hcap hcon albsoil 10-12: initial conditions: tsar_tile sthuf t_soil!
0.2
0.4 1. Sensible heat
sens
itivity
2. Latent heat
1 2 3 4 5 6 7 8 9 10 11 12
0.2
0.4 3. Soil moisture (TDT)
sens
itivity
1 2 3 4 5 6 7 8 9 10 11 12
4. Soil moisture (CRNS)
Sensitivity of RMSE of different simulated variables to uncertain parameters and initial conditions!
with J. Iwema, R. Rosolem
Supporting calibration of a land surface model!
0.2
0.4 1. Sensible heat
sens
itivity
2. Latent heat
1 2 3 4 5 6 7 8 9 10 11 12
0.2
0.4 3. Soil moisture (TDT)
sens
itivity
1 2 3 4 5 6 7 8 9 10 11 12
4. Soil moisture (CRNS)
Sensitivity of RMSE of different simulated variables to uncertain parameters and initial conditions!
with J. Iwema, R. Rosolem
1-9: parameters: b sathh satcon sm_sat sm_crit sm_wilt hcap hcon albsoil 10-12: initial conditions: tsar_tile sthuf t_soil!
Supporting calibration of a land surface model!
samples that improve model performances wrt default set-up default set-up values!
with J. Iwema, R. Rosolem
parameters! initial conditions!
b sathh satcon sm−sat sm−crit sm−wilt hcap hcon albsoil tstar−tile sthuf t−soil
0.3
11.1
0.1
833.3
0
0.1
0.4
0.7
0
0.7
0
0.7
1000019.1
2999900.8
0.1
3
0.1
0.5
260
320
0
1
260
320
Supporting calibration of a land surface model!
parameters! initial conditions!
b sathh satcon sm−sat sm−crit sm−wilt hcap hcon albsoil tstar−tile sthuf t−soil
0.3
11.1
0.1
833.3
0
0.1
0.4
0.7
0
0.7
0
0.7
1000019.1
2999900.8
0.1
3
0.1
0.5
260
320
0
1
260
320
with J. Iwema, R. Rosolem
samples that improve model performances wrt default set-up default set-up values!
Investigating uncertainties in a flood inundation model!
!How important is the choiceof the model’s spatial resolution for flood simulations with respect to other uncertain factors? !
with J. Savage, P. Bates, J. Freer
LISFLOOD-FP model applied to Imera Basin, Italy!
Investigating uncertainties in a flood inundation model!
Flood extent =percentage of wet cells (water depth > 0.10 m)!!!Uncertain input factors:!- spatial Resolution - Channel friction (parameter)- Floodplain friction (parameter)- Forcing Hydrograph (boundary condition)- DEM: Digital Elevation Model!!!
uncertainty in predicted flood extent!
time (hours)!
with J. Savage, P. Bates, J. Freer
most important contributor to uncertainty (Sobol’)!
time (hours)!
Investigating uncertainties in a flood inundation model!
Flood extent =percentage of wet cells (water depth > 0.10 m)!!!Uncertain input factors:!- spatial Resolution - Channel friction (parameter)- Floodplain friction (parameter)- Forcing Hydrograph (boundary condition)- DEM: Digital Elevation Model!!!
uncertainty in predicted flood extent!
time (hours)!
with J. Savage, P. Bates, J. Freer
Investigating uncertainties in a flood inundation model!
!!!!!!!Uncertain input factors:!spatial Resolution ! !Channel friction !Floodplain friction Forcing Hydrograph !DEM!!
with J. Savage, P. Bates, J. Freer
Finding key drivers of slope failure!
What are the dominant drivers of landslides in a slope with properties known at different level of certainty?!!Slope properties (geometry&soil):!!!!!!Design-storm > deeply uncertain:!!!!!!
with S. Almeida and L. Holcombe
evaporation
runoff
rainfall
water table
slip circle
the Combined Hydrology And Slope Stability Model (CHASM)!www.chasm.info!
Slope angle (degrees)! [27,30]!
Thickness of top soil (m)! [2,6]!
etc. (25 in total)!
Rain
fall!
Time!
Duration!
Intensity!
Finding key drivers of slope failure!
Results of CART analysis!
Cohesion/Thickness
top soil
Stable
>2.0 <2.0
>11 <11
>1.5 <1.5
>3 <3
>47 <47
Stable
>18 <18
Depth WT
>80 <80
Rainfall intensity
<5 >5
Stable
<3.2 >3.2
Stable
Thickness top soil
Stable
Fail
Fail
Rainfall duration
Cohesion/Thickness
top soil
Rainfall duration
Rainfall intensity
Stable >7.5 <7.5
Cohesion/Thickness
top soil
Fail Stable
Rainfall intensity
Fail
with S. Almeida and L. Holcombe
Finding key drivers of slope failure!
Results of CART analysis!!:: the dominant drivers of landslides in this slope are:!1 cohesion of the top soil2 thickness of the top soil3 rainfall duration 4 rainfall intensity5 depth of water table!!
Cohesion/Thickness
top soil
Stable
>2.0 <2.0
>11 <11
>1.5 <1.5
>3 <3
>47 <47
Stable
>18 <18
Depth WT
>80 <80
Rainfall intensity
<5 >5
Stable
<3.2 >3.2
Stable
Thickness top soil
Stable
Fail
Fail
Rainfall duration
Cohesion/Thickness
top soil
Rainfall duration
Rainfall intensity
Stable >7.5 <7.5
Cohesion/Thickness
top soil
Fail Stable
Rainfall intensity
Fail
with S. Almeida and L. Holcombe
Finding key drivers of slope failure!
Results of CART analysis!!:: the dominant drivers of landslides in this slope are:!1 cohesion of the top soil2 thickness of the top soil3 rainfall duration 4 rainfall intensity5 depth of water table!!:: thresholds for these drivers that would lead to slope failure!!
Cohesion/Thickness
top soil
Stable
>2.0 <2.0
>11 <11
>1.5 <1.5
>3 <3
>47 <47
Stable
>18 <18
Depth WT
>80 <80
Rainfall intensity
<5 >5
Stable
<3.2 >3.2
Stable
Thickness top soil
Stable
Fail
Fail
Rainfall duration
Cohesion/Thickness
top soil
Rainfall duration
Rainfall intensity
Stable >7.5 <7.5
Cohesion/Thickness
top soil
Fail Stable
Rainfall intensity
Fail
with S. Almeida and L. Holcombe
Characteristics of the SAFE Toolbox!
:: It works under Matlab/Octave (an R version is also available)!!:: flexible, modular structure
!> easy to integrate with models running outside matlab!:: tutorial scripts (workflows) to get started … more in our introductory paper on Env. Mod & Soft (2015)!
Uptake in academia!
Freely available for non-commercial use since December, 2014www.bris.ac.uk/cabot/resources/safe-toolbox/!!Introductory paper published on Env. Mod & Soft in May, 2015!!About 300 academic users so far!!!About 300 academic users so far!
Uptake in industry!
Download requests for a closed-code 3-months trial version by: - E.ON Energy (Uncertainty in yield prediction for wind farms)- Pfizer (Physiological Based Pharmacokinetic models)- EDF (Thermochemical Heat Storage)!Ongoing collaboration with:!- Risk Management Solutions
!> support calibration of rainfall-runoff models !> find key drivers of loss models
- Airbus!> uncertainty in aircraft design models
- JBA Trust !> support long-term investment plans for flood risk reduction!
!
Conclusions!
Our group @UoB is very active in GSA research, contributing to both methodological advances and development of GSA tools!!The SAFE Toolbox is freely available for non-commercial use from: www.bris.ac.uk/cabot/resources/safe-toolbox/!