a toolbox for comprehensive, efficient, and robust ...€¦ · razavi, s. and gupta, h.v., (2019),...
TRANSCRIPT
A Toolbox for Comprehensive, Efficient, and Robust
Sensitivity and Uncertainty Analysis – Version 2Saman Razavi
Second IMPC Annual General Meeting, June 12-19, 2019
Fast Reservoir
Slow Reservoir
Snow Storage
Soil Storage
Rainfall, x
Ru
no
ff, y
Pre
dic
tio
n
Inte
rval
Uncertainty Propagation
Runoff (t)
Pro
bab
ility
Den
sity
Exp
ecte
d
Val
ue
Confidence Limits
Optimization(single- and multi-objective)
E.g.,Razavi & Tolson (2013 WRR)Razavi et. (2012 WRR)Razavi et. (2012 EMS)Razavi et al. (2010 WRR)
Sensitivity Analysis
53%
30%
15%2%
p1p2
p3
p4
E.g.,Haghnegahdar, Razavi et al. (2017 HP)Sheikholeslami & Razavi (2017 EMS)Haghnegahdar & Razavi (2017 EMS)Yassin, Razavi et al. (2017 HP)Gupta & Razavi (2018 WRR)Razavi & Gupta (2016b WRR)Razavi & Gupta (2016a WRR)Razavi & Gupta (2015 WRR)
2
Fast Reservoir
Slow Reservoir
Snow Storage
Soil StorageDiagnostic Testing:How is the fidelity of the model structure, conceptualization, and parameterization?
Factor and Model Reduction:What model component or parameter might be uninfluential or redundant?
Uncertainty Apportionment:When and how does uncertainty matter?
… and how can SensitivityAnalysis (SA) be useful?
Razavi and Gupta (2015), What do we mean by sensitivity analysis?The need for comprehensive characterization of ‘‘global’’ sensitivity inEarth and Environmental systems models, Water Resources Research.
For more see:
3
Sensitivity Analysis
53%
30%
15%2%
p1p2
p3
p4
4
1. Ambiguous Definition of Sensitivity?! Non-unique, conflicting, incomprehensive.
2. Ignorance of “Structure” ofResponse Surfaces Surface shape, covariance, multi-modality.
3. Computational InefficiencyLarge numbers of samples required.
2015
5. Misrepresentation ofDynamical BehavioursNeed for proper “time-varying” and “time-aggregate” sensitivity analysis.
2018
4. Identifiability Analysis or Sensitivity Analysis?!Mixed understanding and characterization.
𝜃1
𝜃2
1
2
3
4
56
VARS is a “unifying theory”, generating a ‘spectrum’ ofinformation on sensitivity, while as limiting cases, itreduces to derivative- and variance-based approaches.
(Razavi & Gupta, 2016a)
VARS is super-efficient (1-2 orders of magnitude more efficientthan alternatives), because it is based on the informationcontained in pairs of points, rather than in individual points.
* Number of pairs grows as ~n2, where n is rate of increase of points.
(Razavi & Gupta, 2016b)
IVA
RS
Ind
ice
s
Derivative-based Indices
Variance-based Indices
Integrated VariogramsAcross a Range of Scales
VARS
DataSamples from
Response Surface
5
VARS-TOOL , by incorporating a diversity of tools and features within a single platform, conveniently provides the user with
the ingredients necessary for conducting exploratory research with a view to discovering new directions for advancing the
field of sensitivity and uncertainty analysis.(Razavi et al., 2019 EMS)
6
High-Dimensional Problems: Factor GroupingAn innovative strategy that employs a clustering mechanism
enabled with bootstrap to handle problems involving hundreds of factors and group them based on their sensitivity and
function (Sheikholeslami, Razavi et al., 2018 EMS).
Handling Model Crashes: Model EmulationStrategies to generate surrogate model responses when a model fails
during simulation under particular parameter values, such that the analysis can be completed, while minimizing the impact of those failures
on the results. (Sheikholeslami, Razavi et al., 2019 GMDD)
Test Functions and Real-World Case StudiesThe ready-to-use collection of many test functions, the HBV-SASK rainfall-runoff model, and MESH land surface
model enables the use of VARS-TOOL for a range of learning, teaching, and research purposes.
(Razavi et al., 2018 EMS)
-1.2
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
-0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1
y 1, ..
., y
6
x1, ..., x6
g(x1) g(x2)
g(x3) g(x4)
g(x5) g(x6)
x1x2
y
(a) (b)
S2 Q2=K2.S2
Q1=K1.S1α
FRAC.R(1-FRAC).R
P
SF RFIf T < TT If T ≥ TT
MS=C0.(T-TT)
R=(RF+MS).(SM/FC)β
AET= PET.(SM/LP)
SM
FCLP
Q=Q1 +Q2
SWESnow Storage
Soil Storage
S1Fast Reservoir
Slow Reservoir
0
20
40
60
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
020406080
100
2000
-01-
0120
00-0
2-05
2000
-03-
1120
00-0
4-15
2000
-05-
2020
00-0
6-24
2000
-07-
2920
00-0
9-02
2000
-10-
0720
00-1
1-11
2000
-12-
1620
01-0
1-20
2001
-02-
2420
01-0
3-31
2001
-05-
0520
01-0
6-09
2001
-07-
1420
01-0
8-18
2001
-09-
2220
01-1
0-27
2001
-12-
0120
02-0
1-05
2002
-02-
0920
02-0
3-16
2002
-04-
2020
02-0
5-25
2002
-06-
2920
02-0
8-03
2002
-09-
0720
02-1
0-12
2002
-11-
1620
02-1
2-21
2003
-01-
2520
03-0
3-01
2003
-04-
0520
03-0
5-10
2003
-06-
1420
03-0
7-19
2003
-08-
2320
03-0
9-27
2003
-11-
0120
03-1
2-06
P (
mm
)
Time
050
100150200250
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
Time
SM (
mm
)
Time
S2 (
mm
)
050
100150200250
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
TimeSWE
(mm
)
020406080
100
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
Time
S1 (
mm
)
UBAS
State Variables
05
1015202530
20
00
-01
-01
20
00
-02
-06
20
00
-03
-13
20
00
-04
-18
20
00
-05
-24
20
00
-06
-29
20
00
-08
-04
20
00
-09
-09
20
00
-10
-15
20
00
-11
-20
20
00
-12
-26
20
01
-01
-31
20
01
-03
-08
20
01
-04
-13
20
01
-05
-19
20
01
-06
-24
20
01
-07
-30
20
01
-09
-04
20
01
-10
-10
20
01
-11
-15
20
01
-12
-21
20
02
-01
-26
20
02
-03
-03
20
02
-04
-08
20
02
-05
-14
20
02
-06
-19
20
02
-07
-25
20
02
-08
-30
20
02
-10
-05
20
02
-11
-10
20
02
-12
-16
20
03
-01
-21
20
03
-02
-26
20
03
-04
-03
20
03
-05
-09
20
03
-06
-14
20
03
-07
-20
20
03
-08
-25
20
03
-09
-30
20
03
-11
-05
20
03
-12
-11
TimeAET
(m
m)
0
50
100
150
200
250
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
Q (
cms)
Time
Output Fluxes
Input Flux
Q1route
route
References:Razavi, S. and Gupta, H.V., (2016), A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research.
Razavi, S. and Gupta, H.V., (2016), A new framework for comprehensive, robust, and efficient global sensitivity analysis: 2. Application. Water Resources Research.
Gupta, H.V. and Razavi, S., (2018), Revisiting the Basis of Sensitivity Analysis for Dynamical Earth System Models, Water Resources Research.
Razavi, S. and Gupta, H.V., (2019), A Multi-Method Generalized Global Sensitivity Matrix Approach to Accounting for the Dynamical Nature of Earth and Environmental Systems Models. Environmental Modelling & Software.
Razavi, S., Sheikholeslami, R., Gupta, H. and Haghnegahdar, A., (2019), VARS-TOOL: A Toolbox for Comprehensive, Efficient, and Robust Sensitivity and Uncertainty Analysis. Environmental Modelling & Software.
Sheikholeslami, R., Razavi, S., Gupta, H.V., Becker, W. and Haghnegahdar, A., (2018), Global sensitivity analysis for high-dimensional problems: How to objectively group factors and measure robustness and convergence while reducing computational cost. Environmental Modelling & Software.
Sheikholeslami, R. and Razavi, S., (2017), Progressive Latin Hypercube Sampling: An efficient approach for robust sampling-based analysis of environmental models.Environmental Modelling & Software.
HBV-SASK
or
Derivative-Based Approach Variance-based Approach
Variogram
Covariogram
Summary Derivations:
If
If
“Elementary Effects” based Metrics of Morris
Variance of Response Surface
“Total-Order Effects” of Sobol’
A “Unifying”, Multi-Approach FrameworkSimultaneous generation of a range of sensitivity indices, includingones based on derivative, variance, and variogram concepts, from asingle sample (Razavi and Gupta,2016a&b WRR).
0
5
10
15
20
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
-30-20-10
0102030
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
0
20
40
60
80
100
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
0.01
0.1
1
10
100
1000
10000
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
Time (daily)
0.01
0.1
1
10
100
1000
10000
20
05
-10
-01
20
05
-10
-15
20
05
-10
-29
20
05
-11
-12
20
05
-11
-26
20
05
-12
-10
20
05
-12
-24
20
06
-01
-07
20
06
-01
-21
20
06
-02
-04
20
06
-02
-18
20
06
-03
-04
20
06
-03
-18
20
06
-04
-01
20
06
-04
-15
20
06
-04
-29
20
06
-05
-13
20
06
-05
-27
20
06
-06
-10
20
06
-06
-24
20
06
-07
-08
20
06
-07
-22
20
06
-08
-05
20
06
-08
-19
20
06
-09
-02
20
06
-09
-16
20
06
-09
-30
Time (daily)
TTETFPMK2
Time-Varying Sensitivity AnalysisProperly accounting for the dynamical nature of Earth and environmentalsystem models, and providing means to compress the full spectrum ofsensitivity information across temporal orspatio-temporal domains.(Gupta and Razavi, 2018 WRR; Razavi and Gupta, 2019 EMS)
Uncertainty in Output Space
Model/Process Response Surface
(Uncertain)
Co
rrelatio
n
Effects
Inte
raction
s Effe
cts
Uncertainty in Input Space
Joint Probability Distribution
Correlation Effects and Non-Uniformity of Factors
Handling non-uniformly distributed and/or correlated factors efficiently and generating of a range of sensitivity indices, including ones based on derivative, variance, and variogram concepts.(Do and Razavi, In Review)
θ1
θ2
θ1
θ2
θ1
θ2
(a) Slice 1 (b) Slice 1 + Slice 2 (c) Slice 1 + Slice 2 + Slice 3
Highly Efficient Sampling TechniquesIncluding the Progressive Latin Hypercube Sampling (PLHS) and STAR Sampling strategies that maximize
robustness and rapid convergence to stable sensitivity estimates (Sheikholeslami and Razavi, 2017 EMS).
∆h
θ1θ2
θ3
StarPoints
StarCenter
PLHS:
STAR:
VARS-TOOL , by incorporating a diversity of tools and features within a single platform, conveniently provides the user with
the ingredients necessary for conducting exploratory research with a view to discovering new directions for advancing the
field of sensitivity and uncertainty analysis.(Razavi et al., 2019 EMS)
7
8
High-Dimensional Problems: Factor GroupingAn innovative strategy that employs a clustering mechanism
enabled with bootstrap to handle problems involving hundreds of factors and group them based on their sensitivity and
function (Sheikholeslami, Razavi et al., 2018 EMS).
Handling Model Crashes: Model EmulationStrategies to generate surrogate model responses when a model fails
during simulation under particular parameter values, such that the analysis can be completed, while minimizing the impact of those failures
on the results. (Sheikholeslami, Razavi et al., 2019 GMDD)
Test Functions and Real-World Case StudiesThe ready-to-use collection of many test functions, the HBV-SASK rainfall-runoff model, and MESH land surface
model enables the use of VARS-TOOL for a range of learning, teaching, and research purposes.
(Razavi et al., 2018 EMS)
-1.2
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
-0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1
y 1, ..
., y
6
x1, ..., x6
g(x1) g(x2)
g(x3) g(x4)
g(x5) g(x6)
x1x2
y
(a) (b)
S2 Q2=K2.S2
Q1=K1.S1α
FRAC.R(1-FRAC).R
P
SF RFIf T < TT If T ≥ TT
MS=C0.(T-TT)
R=(RF+MS).(SM/FC)β
AET= PET.(SM/LP)
SM
FCLP
Q=Q1 +Q2
SWESnow Storage
Soil Storage
S1Fast Reservoir
Slow Reservoir
0
20
40
60
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
020406080
100
2000
-01-
0120
00-0
2-05
2000
-03-
1120
00-0
4-15
2000
-05-
2020
00-0
6-24
2000
-07-
2920
00-0
9-02
2000
-10-
0720
00-1
1-11
2000
-12-
1620
01-0
1-20
2001
-02-
2420
01-0
3-31
2001
-05-
0520
01-0
6-09
2001
-07-
1420
01-0
8-18
2001
-09-
2220
01-1
0-27
2001
-12-
0120
02-0
1-05
2002
-02-
0920
02-0
3-16
2002
-04-
2020
02-0
5-25
2002
-06-
2920
02-0
8-03
2002
-09-
0720
02-1
0-12
2002
-11-
1620
02-1
2-21
2003
-01-
2520
03-0
3-01
2003
-04-
0520
03-0
5-10
2003
-06-
1420
03-0
7-19
2003
-08-
2320
03-0
9-27
2003
-11-
0120
03-1
2-06
P (
mm
)
Time
050
100150200250
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
Time
SM (
mm
)
Time
S2 (
mm
)
050
100150200250
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
TimeSWE
(mm
)
020406080
100
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
Time
S1 (
mm
)
UBAS
State Variables
05
1015202530
20
00
-01
-01
20
00
-02
-06
20
00
-03
-13
20
00
-04
-18
20
00
-05
-24
20
00
-06
-29
20
00
-08
-04
20
00
-09
-09
20
00
-10
-15
20
00
-11
-20
20
00
-12
-26
20
01
-01
-31
20
01
-03
-08
20
01
-04
-13
20
01
-05
-19
20
01
-06
-24
20
01
-07
-30
20
01
-09
-04
20
01
-10
-10
20
01
-11
-15
20
01
-12
-21
20
02
-01
-26
20
02
-03
-03
20
02
-04
-08
20
02
-05
-14
20
02
-06
-19
20
02
-07
-25
20
02
-08
-30
20
02
-10
-05
20
02
-11
-10
20
02
-12
-16
20
03
-01
-21
20
03
-02
-26
20
03
-04
-03
20
03
-05
-09
20
03
-06
-14
20
03
-07
-20
20
03
-08
-25
20
03
-09
-30
20
03
-11
-05
20
03
-12
-11
TimeAET
(m
m)
0
50
100
150
200
250
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
Q (
cms)
Time
Output Fluxes
Input Flux
Q1route
route
References:Razavi, S. and Gupta, H.V., (2016), A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research.
Razavi, S. and Gupta, H.V., (2016), A new framework for comprehensive, robust, and efficient global sensitivity analysis: 2. Application. Water Resources Research.
Gupta, H.V. and Razavi, S., (2018), Revisiting the Basis of Sensitivity Analysis for Dynamical Earth System Models, Water Resources Research.
Razavi, S. and Gupta, H.V., (2019), A Multi-Method Generalized Global Sensitivity Matrix Approach to Accounting for the Dynamical Nature of Earth and Environmental Systems Models. Environmental Modelling & Software.
Razavi, S., Sheikholeslami, R., Gupta, H. and Haghnegahdar, A., (2019), VARS-TOOL: A Toolbox for Comprehensive, Efficient, and Robust Sensitivity and Uncertainty Analysis. Environmental Modelling & Software.
Sheikholeslami, R., Razavi, S., Gupta, H.V., Becker, W. and Haghnegahdar, A., (2018), Global sensitivity analysis for high-dimensional problems: How to objectively group factors and measure robustness and convergence while reducing computational cost. Environmental Modelling & Software.
Sheikholeslami, R. and Razavi, S., (2017), Progressive Latin Hypercube Sampling: An efficient approach for robust sampling-based analysis of environmental models.Environmental Modelling & Software.
HBV-SASK
or
Derivative-Based Approach Variance-based Approach
Variogram
Covariogram
Summary Derivations:
If
If
“Elementary Effects” based Metrics of Morris
Variance of Response Surface
“Total-Order Effects” of Sobol’
A “Unifying”, Multi-Approach FrameworkSimultaneous generation of a range of sensitivity indices, includingones based on derivative, variance, and variogram concepts, from asingle sample (Razavi and Gupta,2016a&b WRR).
0
5
10
15
20
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
-30-20-10
0102030
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
0
20
40
60
80
100
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
0.01
0.1
1
10
100
1000
10000
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
Time (daily)
0.01
0.1
1
10
100
1000
10000
20
05
-10
-01
20
05
-10
-15
20
05
-10
-29
20
05
-11
-12
20
05
-11
-26
20
05
-12
-10
20
05
-12
-24
20
06
-01
-07
20
06
-01
-21
20
06
-02
-04
20
06
-02
-18
20
06
-03
-04
20
06
-03
-18
20
06
-04
-01
20
06
-04
-15
20
06
-04
-29
20
06
-05
-13
20
06
-05
-27
20
06
-06
-10
20
06
-06
-24
20
06
-07
-08
20
06
-07
-22
20
06
-08
-05
20
06
-08
-19
20
06
-09
-02
20
06
-09
-16
20
06
-09
-30
Time (daily)
TTETFPMK2
Time-Varying Sensitivity AnalysisProperly accounting for the dynamical nature of Earth and environmentalsystem models, and providing means to compress the full spectrum ofsensitivity information across temporal orspatio-temporal domains.(Gupta and Razavi, 2018 WRR; Razavi and Gupta, 2019 EMS)
Uncertainty in Output Space
Model/Process Response Surface
(Uncertain)
Co
rrelatio
n
Effects
Inte
raction
s Effe
cts
Uncertainty in Input Space
Joint Probability Distribution
Correlation Effects and Non-Uniformity of Factors
Handling non-uniformly distributed and/or correlated factors efficiently and generating of a range of sensitivity indices, including ones based on derivative, variance, and variogram concepts.(Do and Razavi, In Review)
θ1
θ2
θ1
θ2
θ1
θ2
(a) Slice 1 (b) Slice 1 + Slice 2 (c) Slice 1 + Slice 2 + Slice 3
Highly Efficient Sampling TechniquesIncluding the Progressive Latin Hypercube Sampling (PLHS) and STAR Sampling strategies that maximize
robustness and rapid convergence to stable sensitivity estimates (Sheikholeslami and Razavi, 2017 EMS).
∆h
θ1θ2
θ3
StarPoints
StarCenter
PLHS:
STAR:
VARS-TOOL , by incorporating a diversity of tools and features within a single platform, conveniently provides the user with
the ingredients necessary for conducting exploratory research with a view to discovering new directions for advancing the
field of sensitivity and uncertainty analysis.(Razavi et al., 2019 EMS)
0
5
10
15
20
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
-30-20-10
0102030
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
0
20
40
60
80
100
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
0.01
0.1
1
10
100
1000
10000
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
Time (daily)
0.01
0.1
1
10
100
1000
10000
20
05
-10
-01
20
05
-10
-15
20
05
-10
-29
20
05
-11
-12
20
05
-11
-26
20
05
-12
-10
20
05
-12
-24
20
06
-01
-07
20
06
-01
-21
20
06
-02
-04
20
06
-02
-18
20
06
-03
-04
20
06
-03
-18
20
06
-04
-01
20
06
-04
-15
20
06
-04
-29
20
06
-05
-13
20
06
-05
-27
20
06
-06
-10
20
06
-06
-24
20
06
-07
-08
20
06
-07
-22
20
06
-08
-05
20
06
-08
-19
20
06
-09
-02
20
06
-09
-16
20
06
-09
-30
Time (daily)
TTETFPMK2
Time-Varying Sensitivity AnalysisProperly accounting for the dynamical nature of Earth and environmentalsystem models, and providing means to compress the full spectrum ofsensitivity information across temporal orspatio-temporal domains.(Gupta and Razavi, 2018 WRR; Razavi and Gupta, 2019 EMS)
9
High-Dimensional Problems: Factor GroupingAn innovative strategy that employs a clustering mechanism
enabled with bootstrap to handle problems involving hundreds of factors and group them based on their sensitivity and
function (Sheikholeslami, Razavi et al., 2018 EMS).
Handling Model Crashes: Model EmulationStrategies to generate surrogate model responses when a model fails
during simulation under particular parameter values, such that the analysis can be completed, while minimizing the impact of those failures
on the results. (Sheikholeslami, Razavi et al., 2019 GMDD)
Test Functions and Real-World Case StudiesThe ready-to-use collection of many test functions, the HBV-SASK rainfall-runoff model, and MESH land surface
model enables the use of VARS-TOOL for a range of learning, teaching, and research purposes.
(Razavi et al., 2018 EMS)
-1.2
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
-0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1
y 1, ..
., y
6
x1, ..., x6
g(x1) g(x2)
g(x3) g(x4)
g(x5) g(x6)
x1x2
y
(a) (b)
S2 Q2=K2.S2
Q1=K1.S1α
FRAC.R(1-FRAC).R
P
SF RFIf T < TT If T ≥ TT
MS=C0.(T-TT)
R=(RF+MS).(SM/FC)β
AET= PET.(SM/LP)
SM
FCLP
Q=Q1 +Q2
SWESnow Storage
Soil Storage
S1Fast Reservoir
Slow Reservoir
0
20
40
60
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
020406080
100
2000
-01-
0120
00-0
2-05
2000
-03-
1120
00-0
4-15
2000
-05-
2020
00-0
6-24
2000
-07-
2920
00-0
9-02
2000
-10-
0720
00-1
1-11
2000
-12-
1620
01-0
1-20
2001
-02-
2420
01-0
3-31
2001
-05-
0520
01-0
6-09
2001
-07-
1420
01-0
8-18
2001
-09-
2220
01-1
0-27
2001
-12-
0120
02-0
1-05
2002
-02-
0920
02-0
3-16
2002
-04-
2020
02-0
5-25
2002
-06-
2920
02-0
8-03
2002
-09-
0720
02-1
0-12
2002
-11-
1620
02-1
2-21
2003
-01-
2520
03-0
3-01
2003
-04-
0520
03-0
5-10
2003
-06-
1420
03-0
7-19
2003
-08-
2320
03-0
9-27
2003
-11-
0120
03-1
2-06
P (
mm
)
Time
050
100150200250
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
Time
SM (
mm
)
Time
S2 (
mm
)
050
100150200250
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
TimeSWE
(mm
)
020406080
100
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
Time
S1 (
mm
)
UBAS
State Variables
05
1015202530
20
00
-01
-01
20
00
-02
-06
20
00
-03
-13
20
00
-04
-18
20
00
-05
-24
20
00
-06
-29
20
00
-08
-04
20
00
-09
-09
20
00
-10
-15
20
00
-11
-20
20
00
-12
-26
20
01
-01
-31
20
01
-03
-08
20
01
-04
-13
20
01
-05
-19
20
01
-06
-24
20
01
-07
-30
20
01
-09
-04
20
01
-10
-10
20
01
-11
-15
20
01
-12
-21
20
02
-01
-26
20
02
-03
-03
20
02
-04
-08
20
02
-05
-14
20
02
-06
-19
20
02
-07
-25
20
02
-08
-30
20
02
-10
-05
20
02
-11
-10
20
02
-12
-16
20
03
-01
-21
20
03
-02
-26
20
03
-04
-03
20
03
-05
-09
20
03
-06
-14
20
03
-07
-20
20
03
-08
-25
20
03
-09
-30
20
03
-11
-05
20
03
-12
-11
TimeAET
(m
m)
0
50
100
150
200
250
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
Q (
cms)
Time
Output Fluxes
Input Flux
Q1route
route
References:Razavi, S. and Gupta, H.V., (2016), A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research.
Razavi, S. and Gupta, H.V., (2016), A new framework for comprehensive, robust, and efficient global sensitivity analysis: 2. Application. Water Resources Research.
Gupta, H.V. and Razavi, S., (2018), Revisiting the Basis of Sensitivity Analysis for Dynamical Earth System Models, Water Resources Research.
Razavi, S. and Gupta, H.V., (2019), A Multi-Method Generalized Global Sensitivity Matrix Approach to Accounting for the Dynamical Nature of Earth and Environmental Systems Models. Environmental Modelling & Software.
Razavi, S., Sheikholeslami, R., Gupta, H. and Haghnegahdar, A., (2019), VARS-TOOL: A Toolbox for Comprehensive, Efficient, and Robust Sensitivity and Uncertainty Analysis. Environmental Modelling & Software.
Sheikholeslami, R., Razavi, S., Gupta, H.V., Becker, W. and Haghnegahdar, A., (2018), Global sensitivity analysis for high-dimensional problems: How to objectively group factors and measure robustness and convergence while reducing computational cost. Environmental Modelling & Software.
Sheikholeslami, R. and Razavi, S., (2017), Progressive Latin Hypercube Sampling: An efficient approach for robust sampling-based analysis of environmental models.Environmental Modelling & Software.
HBV-SASK
or
Derivative-Based Approach Variance-based Approach
Variogram
Covariogram
Summary Derivations:
If
If
“Elementary Effects” based Metrics of Morris
Variance of Response Surface
“Total-Order Effects” of Sobol’
A “Unifying”, Multi-Approach FrameworkSimultaneous generation of a range of sensitivity indices, includingones based on derivative, variance, and variogram concepts, from asingle sample (Razavi and Gupta,2016a&b WRR).
0
5
10
15
20
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
-30-20-10
0102030
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
0
20
40
60
80
100
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
0.01
0.1
1
10
100
1000
10000
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
Time (daily)
0.01
0.1
1
10
100
1000
10000
20
05
-10
-01
20
05
-10
-15
20
05
-10
-29
20
05
-11
-12
20
05
-11
-26
20
05
-12
-10
20
05
-12
-24
20
06
-01
-07
20
06
-01
-21
20
06
-02
-04
20
06
-02
-18
20
06
-03
-04
20
06
-03
-18
20
06
-04
-01
20
06
-04
-15
20
06
-04
-29
20
06
-05
-13
20
06
-05
-27
20
06
-06
-10
20
06
-06
-24
20
06
-07
-08
20
06
-07
-22
20
06
-08
-05
20
06
-08
-19
20
06
-09
-02
20
06
-09
-16
20
06
-09
-30
Time (daily)
TTETFPMK2
Time-Varying Sensitivity AnalysisProperly accounting for the dynamical nature of Earth and environmentalsystem models, and providing means to compress the full spectrum ofsensitivity information across temporal orspatio-temporal domains.(Gupta and Razavi, 2018 WRR; Razavi and Gupta, 2019 EMS)
Uncertainty in Output Space
Model/Process Response Surface
(Uncertain)
Co
rrelatio
n
Effects
Inte
raction
s Effe
cts
Uncertainty in Input Space
Joint Probability Distribution
Correlation Effects and Non-Uniformity of Factors
Handling non-uniformly distributed and/or correlated factors efficiently and generating of a range of sensitivity indices, including ones based on derivative, variance, and variogram concepts.(Do and Razavi, In Review)
θ1
θ2
θ1
θ2
θ1
θ2
(a) Slice 1 (b) Slice 1 + Slice 2 (c) Slice 1 + Slice 2 + Slice 3
Highly Efficient Sampling TechniquesIncluding the Progressive Latin Hypercube Sampling (PLHS) and STAR Sampling strategies that maximize
robustness and rapid convergence to stable sensitivity estimates (Sheikholeslami and Razavi, 2017 EMS).
∆h
θ1θ2
θ3
StarPoints
StarCenter
PLHS:
STAR:
VARS-TOOL , by incorporating a diversity of tools and features within a single platform, conveniently provides the user with
the ingredients necessary for conducting exploratory research with a view to discovering new directions for advancing the
field of sensitivity and uncertainty analysis.(Razavi et al., 2019 EMS)
Uncertainty in Output Space
Model/Process Response Surface
(Uncertain)C
orre
lation
Effe
ctsIn
teractio
ns
Effects
Uncertainty in Input Space
Joint Probability Distribution
Correlation Effects and Non-Uniformity of Factors
Handling non-uniformly distributed and/or correlated factors efficiently and generating of a range of sensitivity indices, including ones based on derivative, variance, and variogram concepts.(Do and Razavi, In Review)
10
High-Dimensional Problems: Factor GroupingAn innovative strategy that employs a clustering mechanism
enabled with bootstrap to handle problems involving hundreds of factors and group them based on their sensitivity and
function (Sheikholeslami, Razavi et al., 2018 EMS).
Handling Model Crashes: Model EmulationStrategies to generate surrogate model responses when a model fails
during simulation under particular parameter values, such that the analysis can be completed, while minimizing the impact of those failures
on the results. (Sheikholeslami, Razavi et al., 2019 GMDD)
Test Functions and Real-World Case StudiesThe ready-to-use collection of many test functions, the HBV-SASK rainfall-runoff model, and MESH land surface
model enables the use of VARS-TOOL for a range of learning, teaching, and research purposes.
(Razavi et al., 2018 EMS)
-1.2
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
-0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1
y 1, ..
., y
6
x1, ..., x6
g(x1) g(x2)
g(x3) g(x4)
g(x5) g(x6)
x1x2
y
(a) (b)
S2 Q2=K2.S2
Q1=K1.S1α
FRAC.R(1-FRAC).R
P
SF RFIf T < TT If T ≥ TT
MS=C0.(T-TT)
R=(RF+MS).(SM/FC)β
AET= PET.(SM/LP)
SM
FCLP
Q=Q1 +Q2
SWESnow Storage
Soil Storage
S1Fast Reservoir
Slow Reservoir
0
20
40
60
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
020406080
100
2000
-01-
0120
00-0
2-05
2000
-03-
1120
00-0
4-15
2000
-05-
2020
00-0
6-24
2000
-07-
2920
00-0
9-02
2000
-10-
0720
00-1
1-11
2000
-12-
1620
01-0
1-20
2001
-02-
2420
01-0
3-31
2001
-05-
0520
01-0
6-09
2001
-07-
1420
01-0
8-18
2001
-09-
2220
01-1
0-27
2001
-12-
0120
02-0
1-05
2002
-02-
0920
02-0
3-16
2002
-04-
2020
02-0
5-25
2002
-06-
2920
02-0
8-03
2002
-09-
0720
02-1
0-12
2002
-11-
1620
02-1
2-21
2003
-01-
2520
03-0
3-01
2003
-04-
0520
03-0
5-10
2003
-06-
1420
03-0
7-19
2003
-08-
2320
03-0
9-27
2003
-11-
0120
03-1
2-06
P (
mm
)
Time
050
100150200250
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
Time
SM (
mm
)
Time
S2 (
mm
)
050
100150200250
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
TimeSWE
(mm
)
020406080
100
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
Time
S1 (
mm
)
UBAS
State Variables
05
1015202530
20
00
-01
-01
20
00
-02
-06
20
00
-03
-13
20
00
-04
-18
20
00
-05
-24
20
00
-06
-29
20
00
-08
-04
20
00
-09
-09
20
00
-10
-15
20
00
-11
-20
20
00
-12
-26
20
01
-01
-31
20
01
-03
-08
20
01
-04
-13
20
01
-05
-19
20
01
-06
-24
20
01
-07
-30
20
01
-09
-04
20
01
-10
-10
20
01
-11
-15
20
01
-12
-21
20
02
-01
-26
20
02
-03
-03
20
02
-04
-08
20
02
-05
-14
20
02
-06
-19
20
02
-07
-25
20
02
-08
-30
20
02
-10
-05
20
02
-11
-10
20
02
-12
-16
20
03
-01
-21
20
03
-02
-26
20
03
-04
-03
20
03
-05
-09
20
03
-06
-14
20
03
-07
-20
20
03
-08
-25
20
03
-09
-30
20
03
-11
-05
20
03
-12
-11
TimeAET
(m
m)
0
50
100
150
200
250
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
Q (
cms)
Time
Output Fluxes
Input Flux
Q1route
route
References:Razavi, S. and Gupta, H.V., (2016), A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research.
Razavi, S. and Gupta, H.V., (2016), A new framework for comprehensive, robust, and efficient global sensitivity analysis: 2. Application. Water Resources Research.
Gupta, H.V. and Razavi, S., (2018), Revisiting the Basis of Sensitivity Analysis for Dynamical Earth System Models, Water Resources Research.
Razavi, S. and Gupta, H.V., (2019), A Multi-Method Generalized Global Sensitivity Matrix Approach to Accounting for the Dynamical Nature of Earth and Environmental Systems Models. Environmental Modelling & Software.
Razavi, S., Sheikholeslami, R., Gupta, H. and Haghnegahdar, A., (2019), VARS-TOOL: A Toolbox for Comprehensive, Efficient, and Robust Sensitivity and Uncertainty Analysis. Environmental Modelling & Software.
Sheikholeslami, R., Razavi, S., Gupta, H.V., Becker, W. and Haghnegahdar, A., (2018), Global sensitivity analysis for high-dimensional problems: How to objectively group factors and measure robustness and convergence while reducing computational cost. Environmental Modelling & Software.
Sheikholeslami, R. and Razavi, S., (2017), Progressive Latin Hypercube Sampling: An efficient approach for robust sampling-based analysis of environmental models.Environmental Modelling & Software.
HBV-SASK
or
Derivative-Based Approach Variance-based Approach
Variogram
Covariogram
Summary Derivations:
If
If
“Elementary Effects” based Metrics of Morris
Variance of Response Surface
“Total-Order Effects” of Sobol’
A “Unifying”, Multi-Approach FrameworkSimultaneous generation of a range of sensitivity indices, includingones based on derivative, variance, and variogram concepts, from asingle sample (Razavi and Gupta,2016a&b WRR).
0
5
10
15
20
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
-30-20-10
0102030
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
0
20
40
60
80
100
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
0.01
0.1
1
10
100
1000
10000
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
Time (daily)
0.01
0.1
1
10
100
1000
10000
20
05
-10
-01
20
05
-10
-15
20
05
-10
-29
20
05
-11
-12
20
05
-11
-26
20
05
-12
-10
20
05
-12
-24
20
06
-01
-07
20
06
-01
-21
20
06
-02
-04
20
06
-02
-18
20
06
-03
-04
20
06
-03
-18
20
06
-04
-01
20
06
-04
-15
20
06
-04
-29
20
06
-05
-13
20
06
-05
-27
20
06
-06
-10
20
06
-06
-24
20
06
-07
-08
20
06
-07
-22
20
06
-08
-05
20
06
-08
-19
20
06
-09
-02
20
06
-09
-16
20
06
-09
-30
Time (daily)
TTETFPMK2
Time-Varying Sensitivity AnalysisProperly accounting for the dynamical nature of Earth and environmentalsystem models, and providing means to compress the full spectrum ofsensitivity information across temporal orspatio-temporal domains.(Gupta and Razavi, 2018 WRR; Razavi and Gupta, 2019 EMS)
Uncertainty in Output Space
Model/Process Response Surface
(Uncertain)
Co
rrelatio
n
Effects
Inte
raction
s Effe
cts
Uncertainty in Input Space
Joint Probability Distribution
Correlation Effects and Non-Uniformity of Factors
Handling non-uniformly distributed and/or correlated factors efficiently and generating of a range of sensitivity indices, including ones based on derivative, variance, and variogram concepts.(Do and Razavi, In Review)
θ1
θ2
θ1
θ2
θ1
θ2
(a) Slice 1 (b) Slice 1 + Slice 2 (c) Slice 1 + Slice 2 + Slice 3
Highly Efficient Sampling TechniquesIncluding the Progressive Latin Hypercube Sampling (PLHS) and STAR Sampling strategies that maximize
robustness and rapid convergence to stable sensitivity estimates (Sheikholeslami and Razavi, 2017 EMS).
∆h
θ1θ2
θ3
StarPoints
StarCenter
PLHS:
STAR:
VARS-TOOL , by incorporating a diversity of tools and features within a single platform, conveniently provides the user with
the ingredients necessary for conducting exploratory research with a view to discovering new directions for advancing the
field of sensitivity and uncertainty analysis.(Razavi et al., 2019 EMS)
High-Dimensional Problems: Factor GroupingAn innovative strategy that employs a clustering mechanism
enabled with bootstrap to handle problems involving hundreds of factors and group them based on their sensitivity and
function (Sheikholeslami, Razavi et al., 2018 EMS).
Handling Model Crashes: Model EmulationStrategies to generate surrogate model responses when a model fails
during simulation under particular parameter values, such that the analysis can be completed, while minimizing the impact of those failures
on the results. (Sheikholeslami, Razavi et al., 2019 GMDD)
High-Dimensional Problems: Factor GroupingAn innovative strategy that employs a clustering mechanism
enabled with bootstrap to handle problems involving hundreds of factors and group them based on their sensitivity and
function (Sheikholeslami, Razavi et al., 2018 EMS).
Handling Model Crashes: Model EmulationStrategies to generate surrogate model responses when a model fails
during simulation under particular parameter values, such that the analysis can be completed, while minimizing the impact of those failures
on the results. (Sheikholeslami, Razavi et al., 2019 GMDD)
Test Functions and Real-World Case StudiesThe ready-to-use collection of many test functions, the HBV-SASK rainfall-runoff model, and MESH land surface
model enables the use of VARS-TOOL for a range of learning, teaching, and research purposes.
(Razavi et al., 2018 EMS)
-1.2
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
-0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1
y 1, ..
., y
6
x1, ..., x6
g(x1) g(x2)
g(x3) g(x4)
g(x5) g(x6)
x1x2
y
(a) (b)
S2 Q2=K2.S2
Q1=K1.S1α
FRAC.R(1-FRAC).R
P
SF RFIf T < TT If T ≥ TT
MS=C0.(T-TT)
R=(RF+MS).(SM/FC)β
AET= PET.(SM/LP)
SM
FCLP
Q=Q1 +Q2
SWESnow Storage
Soil Storage
S1Fast Reservoir
Slow Reservoir
0
20
40
60
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
020406080
100
2000
-01-
0120
00-0
2-05
2000
-03-
1120
00-0
4-15
2000
-05-
2020
00-0
6-24
2000
-07-
2920
00-0
9-02
2000
-10-
0720
00-1
1-11
2000
-12-
1620
01-0
1-20
2001
-02-
2420
01-0
3-31
2001
-05-
0520
01-0
6-09
2001
-07-
1420
01-0
8-18
2001
-09-
2220
01-1
0-27
2001
-12-
0120
02-0
1-05
2002
-02-
0920
02-0
3-16
2002
-04-
2020
02-0
5-25
2002
-06-
2920
02-0
8-03
2002
-09-
0720
02-1
0-12
2002
-11-
1620
02-1
2-21
2003
-01-
2520
03-0
3-01
2003
-04-
0520
03-0
5-10
2003
-06-
1420
03-0
7-19
2003
-08-
2320
03-0
9-27
2003
-11-
0120
03-1
2-06
P (
mm
)
Time
050
100150200250
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
Time
SM (
mm
)
Time
S2 (
mm
)
050
100150200250
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
TimeSWE
(mm
)
020406080
100
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
Time
S1 (
mm
)
UBAS
State Variables
05
1015202530
20
00
-01
-01
20
00
-02
-06
20
00
-03
-13
20
00
-04
-18
20
00
-05
-24
20
00
-06
-29
20
00
-08
-04
20
00
-09
-09
20
00
-10
-15
20
00
-11
-20
20
00
-12
-26
20
01
-01
-31
20
01
-03
-08
20
01
-04
-13
20
01
-05
-19
20
01
-06
-24
20
01
-07
-30
20
01
-09
-04
20
01
-10
-10
20
01
-11
-15
20
01
-12
-21
20
02
-01
-26
20
02
-03
-03
20
02
-04
-08
20
02
-05
-14
20
02
-06
-19
20
02
-07
-25
20
02
-08
-30
20
02
-10
-05
20
02
-11
-10
20
02
-12
-16
20
03
-01
-21
20
03
-02
-26
20
03
-04
-03
20
03
-05
-09
20
03
-06
-14
20
03
-07
-20
20
03
-08
-25
20
03
-09
-30
20
03
-11
-05
20
03
-12
-11
TimeAET
(m
m)
0
50
100
150
200
250
20
00
-01
-01
20
00
-02
-05
20
00
-03
-11
20
00
-04
-15
20
00
-05
-20
20
00
-06
-24
20
00
-07
-29
20
00
-09
-02
20
00
-10
-07
20
00
-11
-11
20
00
-12
-16
20
01
-01
-20
20
01
-02
-24
20
01
-03
-31
20
01
-05
-05
20
01
-06
-09
20
01
-07
-14
20
01
-08
-18
20
01
-09
-22
20
01
-10
-27
20
01
-12
-01
20
02
-01
-05
20
02
-02
-09
20
02
-03
-16
20
02
-04
-20
20
02
-05
-25
20
02
-06
-29
20
02
-08
-03
20
02
-09
-07
20
02
-10
-12
20
02
-11
-16
20
02
-12
-21
20
03
-01
-25
20
03
-03
-01
20
03
-04
-05
20
03
-05
-10
20
03
-06
-14
20
03
-07
-19
20
03
-08
-23
20
03
-09
-27
20
03
-11
-01
20
03
-12
-06
Q (
cms)
Time
Output Fluxes
Input Flux
Q1route
route
References:Razavi, S. and Gupta, H.V., (2016), A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research.
Razavi, S. and Gupta, H.V., (2016), A new framework for comprehensive, robust, and efficient global sensitivity analysis: 2. Application. Water Resources Research.
Gupta, H.V. and Razavi, S., (2018), Revisiting the Basis of Sensitivity Analysis for Dynamical Earth System Models, Water Resources Research.
Razavi, S. and Gupta, H.V., (2019), A Multi-Method Generalized Global Sensitivity Matrix Approach to Accounting for the Dynamical Nature of Earth and Environmental Systems Models. Environmental Modelling & Software.
Razavi, S., Sheikholeslami, R., Gupta, H. and Haghnegahdar, A., (2019), VARS-TOOL: A Toolbox for Comprehensive, Efficient, and Robust Sensitivity and Uncertainty Analysis. Environmental Modelling & Software.
Sheikholeslami, R., Razavi, S., Gupta, H.V., Becker, W. and Haghnegahdar, A., (2018), Global sensitivity analysis for high -dimensional problems: How to objectively group factors and measure robustness and convergence while reducing computational cost. Environmental Modelling & Software.
Sheikholeslami, R. and Razavi, S., (2017), Progressive Latin Hypercube Sampling: An efficient approach for robust sampling-based analysis of environmental models.Environmental Modelling & Software.
HBV-SASK
or
Derivative-Based Approach Variance-based Approach
Variogram
Covariogram
Summary Derivations:
If
If
“Elementary Effects” based Metrics of Morris
Variance of Response Surface
“Total-Order Effects” of Sobol’
A “Unifying”, Multi-Approach FrameworkSimultaneous generation of a range of sensitivity indices, includingones based on derivative, variance, and variogram concepts, from asingle sample (Razavi and Gupta, 2016a&b WRR).
0
5
10
15
20
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
-30-20-10
0102030
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
0
20
40
60
80
100
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
0.01
0.1
1
10
100
1000
10000
20
05
-10
-01
20
05
-10
-08
20
05
-10
-15
20
05
-10
-22
20
05
-10
-29
20
05
-11
-05
20
05
-11
-12
20
05
-11
-19
20
05
-11
-26
20
05
-12
-03
20
05
-12
-10
20
05
-12
-17
20
05
-12
-24
20
05
-12
-31
20
06
-01
-07
20
06
-01
-14
20
06
-01
-21
20
06
-01
-28
20
06
-02
-04
20
06
-02
-11
20
06
-02
-18
20
06
-02
-25
20
06
-03
-04
20
06
-03
-11
20
06
-03
-18
20
06
-03
-25
20
06
-04
-01
20
06
-04
-08
20
06
-04
-15
20
06
-04
-22
20
06
-04
-29
20
06
-05
-06
20
06
-05
-13
20
06
-05
-20
20
06
-05
-27
20
06
-06
-03
20
06
-06
-10
20
06
-06
-17
20
06
-06
-24
20
06
-07
-01
20
06
-07
-08
20
06
-07
-15
20
06
-07
-22
20
06
-07
-29
20
06
-08
-05
20
06
-08
-12
20
06
-08
-19
20
06
-08
-26
20
06
-09
-02
20
06
-09
-09
20
06
-09
-16
20
06
-09
-23
20
06
-09
-30
Time (daily)
0.01
0.1
1
10
100
1000
10000
20
05
-10
-01
20
05
-10
-15
20
05
-10
-29
20
05
-11
-12
20
05
-11
-26
20
05
-12
-10
20
05
-12
-24
20
06
-01
-07
20
06
-01
-21
20
06
-02
-04
20
06
-02
-18
20
06
-03
-04
20
06
-03
-18
20
06
-04
-01
20
06
-04
-15
20
06
-04
-29
20
06
-05
-13
20
06
-05
-27
20
06
-06
-10
20
06
-06
-24
20
06
-07
-08
20
06
-07
-22
20
06
-08
-05
20
06
-08
-19
20
06
-09
-02
20
06
-09
-16
20
06
-09
-30
Time (daily)
TTETFPMK2
Time-Varying Sensitivity AnalysisProperly accounting for the dynamical nature of Earth and environmentalsystem models, and providing means to compress the full spectrum ofsensitivity information across temporal orspatio-temporal domains.(Gupta and Razavi, 2018 WRR; Razavi and Gupta, 2019 EMS)
Uncertainty in Output Space
Model/Process Response Surface
(Uncertain)
Co
rrelatio
n
Effects
Inte
raction
s Effe
cts
Uncertainty in Input Space
Joint Probability Distribution
Correlation Effects and Non-Uniformity of Factors
Handling non-uniformly distributed and/or correlated factors efficiently and generating of a range of sensitivity indices, including ones based on derivative, variance, and variogram concepts.(Do and Razavi, In Prep.)
θ1
θ2
θ1
θ2
θ1
θ2
(a) Slice 1 (b) Slice 1 + Slice 2 (c) Slice 1 + Slice 2 + Slice 3
Highly Efficient Sampling TechniquesIncluding the Progressive Latin Hypercube Sampling (PLHS) and STAR Sampling strategies that maximize
robustness and rapid convergence to stable sensitivity estimates (Sheikholeslami and Razavi, 2017 EMS).
∆h
θ1θ2
θ3
StarPoints
StarCenter
PLHS:
STAR:
11
VARS-TOOL , by incorporating a diversity of tools and features within a single platform, conveniently provides the user with
the ingredients necessary for conducting exploratory research with a view to discovering new directions for advancing the
field of sensitivity and uncertainty analysis.(Razavi et al., 2019 EMS)Already more than 250 users from 36 countries.
12