Journal of Hydrology 406 (2011) 54–65

Contents lists available at ScienceDirect

Journal of Hydrology

journal homepage: www.elsevier .com/locate / jhydrol

Uncertainty estimates by Bayesian method with likelihood of AR (1) plus Normalmodel and AR (1) plus Multi-Normal model in different time-scaleshydrological models

Lu Li a,⇑, Chong-Yu Xu a,b, Jun Xia c, Kolbjørn Engeland d, Paolo Reggiani e,f

a Department of Geosciences, University of Oslo, P.O. Box 1047, Blindern, 0316 Oslo, Norwayb Department of Earth Sciences, Uppsala University, Villavägen 16, SE-752 36 Uppsala, Swedenc Key Laboratory of Water Cycle & Related Land Surface Processes, Institute of Geographic Sciences and Natural Resources Research, CAS, Beijing 100101, Chinad SINTEF Energy Research, 7465 Trondheim, Norwaye Section of Hydraulic Structures and Probabilistic Design, Delft University of Technology, P.O. Box 5048, 2600GA Delft, The Netherlandsf Deltares, P.O. Box 177, 2600MH Delft, The Netherlands

a r t i c l e i n f o

Article history:Received 6 October 2010Received in revised form 1 May 2011Accepted 30 May 2011Available online 16 June 2011

This manuscript was handled byDr. A. Bardossy, Editor-in-Chief, with theassistance of Ezio Todini, Associate Editor

Keywords:Uncertainty assessmentHydrological modelsBayesian methodsMulti-NormalTime scales

0022-1694/$ - see front matter � 2011 Elsevier B.V. Adoi:10.1016/j.jhydrol.2011.05.052

⇑ Corresponding author.E-mail address: lu.li@geo.uio.no (L. Li).

s u m m a r y

Bayesian revision is widely used in hydrological model uncertainty assessment. With respect to modelcalibration, parameter estimation and analysis of uncertainty sources, various regression and probabilis-tic approaches have been used in different models calibrated for either daily or monthly time step. Noneof these applications however includes a comparison of uncertainty analysis in hydrological models withrespect to the time periods, at which the models are operated. This study pursues a comprehensive inter-comparison and evaluation of uncertainty assessments by Bayesian revision using the Metropolis Hasting(MH) algorithm with the hydrological model WASMOD with daily and monthly time step. In the dailystep model three likelihood functions are used in combination with Bayesian revision: (i) the AR (1) plusNormal time period independent model (Model 1), (ii) the AR (1) plus Multi-Normal model (Model 2),and (iii) the AR (1) plus Normal time period dependent model (Model 3). In addition an index calledthe percentage of observations bracketed by the Unit Confidence Interval (PUCI) was used for uncertaintyevaluation. The results reveal that it is more important to consider the autocorrelation in daily WASMODrather than monthly WASMOD. Firstly, the resulting goodness of fit of the daily model vs. observations asmeasured by the Nash–Sutcliffe efficiency value is comparable with that calculated by the optimizationalgorithm in monthly WASMOD. Secondly, the AR (1) model is not sufficiently adequate to estimate thedistribution of residuals in daily WASMOD since PUCI shows that Model 2 outperforms Model 1. Further-more, the maximum Nash–Sutcliffe efficiency value of Model 2 is the largest. Thirdly, Model 3 performsbest over the entire flow range, while Model 2 outperforms Model 3 for high flows. This shows that addi-tional statistical parameters reflect the statistical characters of the residuals more efficiently and accu-rately. Fourthly, by considering the difference in terms of application and computational efficiency itbecomes evident that Model 3 performs best for daily WASMOD. Model 2 on the other hand is superiorfor daily time step WASMOD if the auto-correlation of parameters is considered.

� 2011 Elsevier B.V. All rights reserved.

1. Introduction

The Bayesian method is widely used for uncertainty assessmentin hydrologic modelling. During the last two decades several appli-cations to various models and catchments have been pursued inthe literature (Bates and Campbell, 2001; Duan et al., 2007;Engeland and Gottschalk, 2002; Li et al., 2010b; Marshall et al.,2007; Kavetski et al., 2006a,b; Kuczera et al., 2006; Kuczera andParent, 1998; Krzysztofowicz and Kelly, 2000; Liu et al., 2005;

ll rights reserved.

Thiemann et al., 2001; Vrugt et al., 2003). The Bayesian method re-quires a specification of a likelihood function. For it to provideunbiased and reliable results several conditions need to be satis-fied. The likelihood function provides a probability distribution ofall simulation errors. It is therefore necessary to specify a correctdistribution for the simulation errors and a possible mutual corre-lation structure. The most common approach is to transform thedata for them to become normally and identically distributed (con-stant variance).

There are two commonly used transformation methods inuncertainty assessment for hydrological models. One is the Box–Cox method (Engeland et al., 2005; Jin et al., 2010; Wang et al.,

L. Li et al. / Journal of Hydrology 406 (2011) 54–65 55

2009; Yang et al., 2007b) and the other one is the Normal QuantileTransform (NQT) (Krzysztofowicz, 1997, 1999; Todini, 2008; Vander Waerden, 1952, 1953a,b). Krzysztofowicz and Herr (2001) pro-posed the Normal Quantile Transform (NQT) method in his Hydro-logical Uncertainty Processor (HUP) to convert both observationand model output into the Gaussian space with the aim to estimatethe predictive uncertainty. This method is one of the most com-monly used in hydrological uncertainty analysis in deriving thejoint distribution and predictive conditional distribution from amultivariate distribution (Krzysztofowicz and Kelly, 2000; Liuet al., 2005; Maranzano and Krzysztofowicz, 2004; Montanariand Brath, 2004; Reggiani and Weerts, 2008; Todini, 2008). Be-cause hydrological processes have a ‘‘memory effect’’, the modelresiduals are often inter-correlated. In this context the discrete-time and continuous-time autoregressive error models have beenused universally nowadays (Yang et al., 2007b), amongst which,the first order auto-regressive model (AR (1) model) is the mostwidely used. The latter is employed in combination with the trans-formation method to make the residuals satisfy the statisticalassumptions of identical and independent distribution (Engelandet al., 2010; Krzysztofowicz, 2002; Yang et al., 2007a). Mantovanand Todini (2006) proposed a Multi-Normal AR (1) model as like-lihood function by considering the autocorrelation of errors.

All uncertainty assessment methods account for differentsources of uncertainty. Some methods use a statistical model ofthe residuals to address all the uncertainty sources (Engelandand Gottschalk, 2002; Jin et al., 2010; Krzysztofowicz, 2002;Montanari and Grossi, 2008). Others consider all uncertaintysources lumped into parameter uncertainty such as the GLUEmethod and the multi-objective method (Beven and Freer, 2001;Engeland et al., 2006; Yapo et al., 1998). Others consider the inputerror and/or model structure error jointly or separately(Chowdhury and Sharma, 2007; Kavetski et al., 2006a, 2006b;Kuczera et al., 2006). Recently, some criteria have been proposedto evaluate the performance of uncertainty analysis methods: (1)indices for goodness of fit at the level of the posterior distribution(efficiency) based on the Nash–Sutcliffe (NS) coefficient or otherobjective functions (Jin et al., 2010; Yang et al., 2008); (2) indicesused to compare the width of the model uncertainty intervals (res-olution) mainly based on 95% confidence interval of discharge(95CI), which include the width of 95CI, the average distance be-tween the upper and the lower limits of 95% confidence intervals,95PPU (d-factor), (Li et al., 2009; Xiong et al., 2009; Yang et al.,2008); and (3) indices to compare the distribution of the uncer-tainty estimates (reliability) which include the percentage ofobservations bracketed by the 95CI, the Continuous Rank Probabil-ity Score (CRPS). Engeland et al. (2010) proposed a hierarchy of allthe criteria which are reliability, resolution and efficiency.

According to Todini (2007, 2011) all these studies treats theproblem of ‘‘uncertainty’’, which can be distinguished into ‘‘Valida-tion Uncertainty’’ and ‘‘Predictive Uncertainty’’. The ‘‘ValidationUncertainty’’ is the probability that the simulated value (water le-vel, discharge, water volume, etc.) will be less than or equal to aprescribed value given prior knowledge, the historical informationand the observed value, which refers to running the model in his-torical mode based on observations. Numerous approaches forquantifying the ‘‘Validation Uncertainty’’ have been proposed,including the Generalized Likelihood Uncertainty Estimation(GLUE) (Beven and Freer, 2001; Blasone et al., 2008a,b; Vogelet al., 2008; Xiong and O’Connor, 2008) and formal Bayesian meth-ods (Montanari and Brath, 2004; Vrugt et al., 2003; Engeland et al.,2005; Yang et al., 2008; Jin et al., 2010; Li et al., 2010b). In which,the uncertainty has been estimated by 95% confidence interval(95CI) (Engeland et al., 2005; Li et al., 2009, 2010b) or 95% predic-tion uncertainty band (95PPU) (Abbaspouretal et al., 2007; Yanget al., 2008) calculated at the 2.5% and 97.5% levels of the

cumulative distribution function of the model output variables.Krzysztofowicz (1999, 2001) defines ‘‘Predictive Uncertainty’’ asexpressing the uncertainty on what may happen at a future stageconditional on the forecast of model, which was considered as aknown value, based on the all available information, like input,parameter and model structure. It refers to running the model inforecast mode. After this, Todini (2007) redefined the predictiveuncertainty that it is based on the posterior density of parametersand the conditional probability density of a predict and conditionalon the covariates and a set of parameters. According to this, threeapproaches for the assessment of predictive uncertainty have beenmotioned (Todini, 2011), which are Hydrological Uncertainty Pro-cessor (HUP) (Krzysztofowicz, 1999), the Bayesian Model Averag-ing (BMA) (Raftery et al., 2003, 2005) and the Model ConditionalProcesssor (MCP) (Todini, 2008; Coccia and Todini, 2010). This def-inition of predictive uncertainty is restricted to hydrologic fore-casting. It might, however, argued that the predictive uncertaintymight be defined for other applications that include predictions,e.g. streamflow in an ungauged catchment (conditioned on stream-flow in a nearby catchment), or streamflow for a period of missingdata (conditioned on observations outside this period). For thisreason the difference between the ‘‘Predictive Uncertainty’’ and‘‘Validation Uncertainty’’ as defined by Todini might be more re-lated to the problem addressed (forecasting/non-forecasting) andnot to a general Bayesian definition of predictive uncertainty. Inboth types of uncertainty analyses, the Bayesian method requiresa specification of a likelihood function. In order to provide unbi-ased and reliable results several conditions need to be satisfied.The likelihood function provides a probability distribution of allsimulation errors. It is therefore necessary to specify a correct dis-tribution for the simulation errors and a possible mutual correla-tion structure. The most common approach is to transform thedata for them to become normally and identically distributed (con-stant variance).

Hydrological models at monthly time-step have been widelyutilized for long-range streamflow forecasting, for assessing thelong-term climatic change impact on water resources, and for re-gional water resources management (Xu and Vandewiele, 1995;Chen et al., 2007). Whereas daily or shorter time step hydrologicalmodels are required for real-time flood forecasting, for generationof synthetic sequences of hydrologic data for facility design, forstudying the potential impacts of changes in land use or climate,and for interdisciplinary researches. Therefore, both monthly anddaily hydrological modeling approaches are essential in water cy-cle and water resources researches. With respect to model calibra-tion, parameter estimation, uncertainty sources analyses, variousregression and probabilistic approaches have been employed inhydrological applications. Some include monthly water balancehydrological models (Benke et al., 2008; Engeland et al., 2005; Jinet al., 2010; Li et al., 2010b), others are daily conceptual hydrolog-ical models or physically-based distributed hydrological models(Blasone et al., 2008a; Engeland and Gottschalk, 2002; Li et al.,2010a; Liu et al., 2005; Looper et al., 2009; Wang et al., 2009; Yanget al., 2008, 2007b). However, none of the above applicationsseems to include comparisons of uncertainty analyses in time-variant hydrological models. It is common knowledge that thehydrological response variables are dependent on the time scaleof model simulation. However, the question remains how modeltime scales impact on the performance of the uncertainty analysismethods and how models of similar structure operated with differ-ent time steps affect the uncertainty assessment via Bayesianmethods. The objective of this paper is to attempt to fill this gap.

This study performs a comprehensive comparison and evalua-tion of uncertainty estimates by Bayesian methods using theMetropolis Hasting (MH) algorithm in two validated conceptualhydrological models (WASMOD) with monthly and daily time

56 L. Li et al. / Journal of Hydrology 406 (2011) 54–65

steps. It aims at explaining the differences between the Bayesianmethod applied to hydrological models of similar concept andstructure but with different time steps. The Box–Cox transforma-tion in this study was used to obtain variables which are closerto a normal distribution. In daily WASMOD, two likelihood func-tions were used to remove the time dependence of residuals inthe Bayesian method: (i) the AR (1) plus Normal model and (ii)the AR (1) plus Multi-Normal model. We also tested models, wherethe statistical parameters were dependent and/or independent ontime, respectively. The Average Relative Interval Length (ARIL) de-fined by Jin et al. (2010), the percentage of observations bracketedby the Confidence Interval (PCI), and a new index called the Per-centage of observations bracketed by the Unit Confidence Interval(PUCI) were used in this study for evaluation of model perfor-mance. Furthermore, the 95% confidence interval of daily dischargein low flow, medium flow and high flow were selected for compar-ison. Finally, the paper tries to give some advice to users on appli-cation of Bayesian method in validation uncertainty in time-variant hydrological model applications.

2. Study area and hydrological model

2.1. Study area

The selected study area is the Jiuzhou basin (Fig. 1) a tributaryof the Dongjiang River in southern China. The river flows from thenorth-east to the south-west estuary with a drainage area of385 km2 (upstream area of Jiuzhou gauge station). The landscape

Fig. 1. Main river, meteorological stations and

is characterized by hills and plains. The Jiuzhou basin is locatedin a Monsoon region in a tropical and sub-tropical humid climatewith a mean annual temperature of about 21 �C and only occa-sional incidents of winter daily air temperature dropping below0 �C in the mountainous areas of the upper catchment part. Precip-itation is generated mainly from two types of storms: frontal typeand typhoon-type rainfalls. Eighty percent of the annual precipita-tion occurs during the wet season from April to September and themean annual precipitation for the period of 1978–1988 is1802 mm. The mean annual runoff is 993.4 mm or 55% of the an-nual precipitation. Precipitation data from Heduobu, Tangwei andJiuzhou stations for the period of 1978–1988 are used for the studyas the main input data. The potential evapotranspiration was cal-culated through the Penman–Monteith formula using meteorolog-ical data of the Heyuan meteorological station. All data have gonethrough quality control in earlier studies (Jin et al., 2010).

2.2. Wasmod

WASMOD is a well-tested simple conceptual water balancemodel with multiple time scales (Xu, 2002). The input data tothe model include precipitation, potential evapotranspiration.Model output data are fast flow, slow flow, actual evapotranspira-tion and soil moisture. The model can be set up with different spa-tial and temporal scales varying from global to catchment and fromdaily to monthly (Gong et al., 2009; Jin et al., 2010; Widen-Nilssonet al., 2009, 2007; Xu, 2002). For this study a monthly and a dailyversion of the model at the catchment scale are used. The main

discharge station of Jiuzhou River Basin.

Table 1The main equations in WASMOD.

Actual evapotranspiration et ¼minfðsmt�1=Dt þ ptÞð1� expð�a1 � eptÞÞ; eptg 0 6 a1 6 1Slow flow st ¼ a2ðsmt�1Þ2 0 6 a2 6 1

Fast flow ft ¼ a3ðsmt�1Þðpt � eptð1� expð�pt=maxðept ;1ÞÞÞÞ 0 6 a3 6 1Water balance smt ¼ smt�1 þ ðpt � et � dtÞ � DtRouting of fast flow SCt ¼ SCt�1 þ ft � Dt 0 6 a4 6 1

Qt ¼ a4 � SCt

SCt ¼ SCt � Qt � DtTotal runoff dt ¼ st þ Qt

In which, t is t th month or day, pt is precipitation, ept is potential evaporation, smt is t th monthly or daily soil moisture, SCt is channelstorage at time step t, Dt is time step which is either one month for the monthly modal or one day for the daily model. The unit for allthe fluxes is either mm/month for the monthly model and mm/day for the daily model. The unit for the state variables soil moisture andchannel storage is mm.

L. Li et al. / Journal of Hydrology 406 (2011) 54–65 57

model equations are shown in Table 1. Snowmelt is not consideredin the simulation because of the rare appearance of snow in theJiuzhou River basin in the South of China. There are three parame-ters in monthly WASMOD and four parameters in daily WASMOD,with one additional routing parameter with respect to the former.Furthermore, parameter a2 and a3 have multiplied 103 and 104

respectively, aiming to be scaled into [0, 1] in this paper.

3. Methods

3.1. Residual analysis

The variable nt represent the difference of the observed randomvariable gtðytÞ and the simulated random variables gM

t ðyMt Þ:

nt ¼ gtðytÞ � gMt ðyM

t Þ ð1Þ

where t is an index for time, M indicates that the variable is simu-lated, y is the variable in original space and g is the transformedvariable.

In this study, Box–Cox transformation method is used, whichcan be expressed as:

hðy; kÞ ¼yk�1

k k–0lnðyÞ k ¼ 0

(ð2Þ

where k is a transformation parameter. It is often fixed, e.g. at thevalue of 0.3 in Yang et al. (2007) or 0.2 (Engeland et al., 2010).The square-root transformation and log-transformation are specialcases of the Box–Cox transformation (Engeland and Gottschalk,2002; Engeland et al., 2005; Jin et al., 2010; Langsrud et al., 1998;Wang et al., 2009).

The Lilliefors test (Lilliefors, 1967) was used to verify the nor-mality of the distribution of the residuals of the Box–Cox trans-formed variable with k fixed at different values for the monthlyand daily models. The test statistics is called LSTAT and it shouldbe as small as possible. Calculations indicated that the Box–Cox

-6

-4

-2

0

2

4

6

1 2 3 4 5 6mo

Res

idua

l

Fig. 2. Variances of daily runoff residuals in Jiuzhou basin (

transformation yields residuals closest to normal for values ofk = 0.6 in daily and k = 0.2 in monthly WASMOD.

It was also necessary to verify whether nt were stochasticallyindependent. In case of stochastic dependence, an AR (1) modelneeds to be fitted to the residuals as follows:

nt � b ¼ aðnt�1 � bÞ þ et ð3Þ

There is no need to use AR (1) model in monthly WASMOD,since the monthly residuals were not significantly correlated.

On the contrary, daily residuals were more heteroscedastic andautocorrelated. Using the Box–Cox transformation to reduce theheteroscedasticity of residuals increased their degree of autocorre-lation. We therefore used the AR (1) model to make the Box–Coxtransformed residuals independent. However, the AR (1) innova-tions were still auto-correlated. We therefore introduced AR (1)plus Multi-Normal model to account for all autocorrelations. Thismethod has been used before in hydrological applications(Mantovan and Todini, 2006; Romanowicz et al., 1994).

Fig. 2 shows that the variances of AR (1) innovations from dailyresiduals have different values in different seasons. We thereforetested two statistical models: (i) in Model 1 the statistic parame-ters were constant and (ii) in Model 3 the parameters were allowedto depend on the season. Two simulation sub-periods were de-fined: a wet period from April to September and a dry period fromOctober to March.

3.2. Statistical models for the prediction errors

3.2.1. Statistical model for monthly WASMODThe assumption that there is neither bias nor autocorrelation in

monthly residuals results in the following likelihood function formonthly WASMOD:

Lðx; hjnÞ ¼YT

t¼1

1r qðnt

rÞ ð4Þ

where h represents the hydrological model parameters, x repre-sents the statistical parameters including in this case only the

7 8 9 10 11 12nth

1979–1988) after Box–Cox transformation and AR (1).

58 L. Li et al. / Journal of Hydrology 406 (2011) 54–65

standard deviation of the residuals r, n ¼ fn1; . . . ; nTg is the residu-als, t is the time index, and q is the normal density operator.

3.2.2. Statistical models for daily WASMOD

(a) The AR (1) plus Normal model in which statistical parame-ters independent on periods (Model 1)

The likelihood function for the AR (1) plus Normal model for theprediction errors in daily WASMOD were:

Lðx; h nj Þ ¼ 1r

qðn0

rÞ �YT

t¼1

1r

qðnt � b� aðnt�1 � bÞr

Þ ð5Þ

where h represents the hydrological model parameters. x repre-sents the statistical parameters including an autoregressive terma, a bias term b and a standard deviation of the residuals r, n0 isthe initial residual value, t is the time index, and q is the normaldensity operator.

(b) The AR (1) plus Multi-Normal model (Model 2)The likelihood function is built for time steps ranging from 1 to t

(t = n) given the AR (1) plus Multi-Normal model.

Lðx; h nj Þ ¼expð� 1

2 ðn� bÞTP�1ðn� bÞÞ

ð2pÞn=2½detðPÞ�1=2 ð6Þ

whereP

is the covariance matrix that can be approximated with:

X¼ r2

n

1 r1 r21 . . . rn�2

1 rn�11

r1 1 r1 . . . rn�31 rn�2

1

r21 r1 1 . . . rn�4

1 rn�31

. . . . . . . . . . . . . . . . . .

rn�21 rn�3

1 rn�41 . . . 1 r1

rn�11 rn�2

1 rn�31 . . . r1 1

2666666664

3777777775

ð7Þ

where

r2n ¼ var½nt � ð8Þ

r1 ¼

Pn�1

t¼1ðnt � nÞðntþ1 � nÞ

Pnt¼1ðnt � nÞ2

ð9Þ

(c) The AR (1) plus Normal model in which statistical parame-ters depend on periods (Model 3)

The likelihood function for the daily WASMOD for which thestatistical parameters depend on the periods is:

Lðx; h nj Þ ¼ 1r0

qðn0

r0Þ �YT

t¼1

1rt

qðnt � bt � atðnt�1 � bt�1Þrt

Þ ð10Þ

where all statistical parameters (bt , at and rt) are labelled accordingto two periods: the wet period (April to September) and the dry per-iod (rest months):

Table 2The prior distributions of all parameters in two hydrological models.

Hydrological models Monthly WASMOD

Parameter a1 a2 a3Prior distributionsa U[0, 1] U[0, 1] U[0, 1]Hydrological models Daily WASMODParameter a1 a2 a3Prior distributionsa U[0, 1] U[0, 1] U[0, 1]

a U½a; b� means the prior distribution of the parameter is uniform over the interval ½a;b / r�1 means the prior density of the parameter at value r is proportional to r�1.

xtfbt; at;rtg ¼xWfbW ; aW ;rWg t 2 ðApriltoSeptemberÞxDfbD; aD;rDg others

�ð11Þ

3.3. Prior density

There is no information about the distribution of parameters.Parameter r is an unknown model standard deviation and withJeffreys’ uninformative prior it is proportional to r�1 (Bernardoand Smith, 1994; Yang et al., 2007b). Besides, the prior probabilitydensities of other parameters are generally taken as non-informa-tive multi-uniform distribution in hydrological applications (Enge-land et al., 2005; Jin et al., 2010; Liu et al., 2005; Yang et al., 2008).The prior densities and intervals of all parameters in the twohydrological models are shown in Table 2. The posterior densitypðu nj Þ is:

pðujnÞ ¼ Lðu nj Þ � f ðuÞRLðu nj Þ � f ðuÞdu ð12Þ

where f ðuÞ is the prior density, u ¼ fx; hg and n ¼ fn1; . . . ; nTg. Thelikelihood L is given in Eqs. (4), (5), (6), and (10) above.

3.4. Metropolis Hasting algorithm

The Markov chain Monte Carlo method known as the Metropo-lis Hasting (MH) algorithm was used to sample from the posteriordistributions. The Markov chain is initiated from a random startingvalue, whereby a new sample is derived from the proposal distri-bution. To avoid long ‘‘burn in’’ time, the Markov chain is initiatedin correspondence of an approximated maximum Nash–Sutcliffevalue calculated with the help of an optimization algorithm namedVA05A (Hopper, 1978). A new sample was suggested from the pro-posal distribution. The new sample was accepted according to anacceptance probability. If not, the chain kept the old sample. Thenecessary steps of the Metropolis Hasting method have beenexplained in detail in the literature (Chib and Greenberg, 1995;Engeland and Gottschalk, 2002; Kuczera and Parent, 1998).

Here the proposal distributions d were assumed as normal inthe Random walk M–H Chain:

dðxÞ � N½xL; Sx� ð13Þ

dðhÞ � N½hL; Sh� ð14Þ

where x ¼ fa; b;rg are the statistical parameters, h are the hydro-logic parameters, L index the current state of the chain, Sx and Sh

are the covariance matrixes of the proposal distributions. The nor-mal distribution is not appropriate for r > 0. However, in this study,the standard deviation of r was much smaller than the mean of r,which means that the normal distribution density functions aremostly inside the feasible region. Other statistical parameters andmodel parameters in this paper satisfied the same assumption.The algorithm works best if the proposal density matches the targetdistribution which is generally unknown. The tuning of the variancewas performed by calculating the acceptance rate. The ideal

rb

/r�1

a4 a b rb

U[0, 1] U[0, 1) U[�10, 10] /r�1

b�.

0.7 0.8 0.90

0.1

0.2

0.3

0.4

Frequency

A1

0.1 0.2 0.3 0.40

0.1

0.2

0.3

0.4

Frequency

A2

0.1 0.2 0.3 0.40

0.1

0.2

0.3

0.4

Frequency

A3

Fig. 3. Histograms of parameters of monthly WASMOD derived by Metropolis Hasting method.

L. Li et al. / Journal of Hydrology 406 (2011) 54–65 59

acceptance rate is approximately 40–50% for one parameter updat-ing, decreasing to approximately 20–30% for multi-parametersupdating in a one block (Chib and Greenberg, 1995; Engeland andGottschalk, 2002). Furthermore, the Scale Reduction Score

ffiffiffiRp

(Gelman and Rubin, 1992) was used to check whether the MCchains converged.

3.5. Uncertainty confidence intervals estimation

The uncertainty intervals for streamflow due to parameteruncertainty were calculated using all the MH-samples of thehydrologic parameters. The samples were then sorted for everyday in order to get credibility intervals. The empirical distributionPBðy < yiÞ based on the rank i was used to calculate the probabilityusing the sorted sample yi

Table 3The statistic characteristic of parameters estimated by Bayesian method for monthlyWASMOD.

Method Parameters

a1 a2 a3

Min 0.783 0.132 0.148Max 0.872 0.340 0.405Average 0.830 0.224 0.258Variance 1.59E�04 1.38E�03 1.54E�03Skewness �0.134 0.166 0.248Estimated 0.829 0.207 0.2442.50% Quantile 0.804 0.158 0.1865% Quantile 0.809 0.166 0.19550% Quantile 0.830 0.223 0.25795% Quantile 0.850 0.286 0.32597.50% Quantile 0.854 0.300 0.341MNS 0.856

MNS: Maximum Nash–Sutcliffe.

Table 4The statistic characteristic difference of parameters estimated by three methods for daily

Parameter Min Max Mean Variance

Model 1 a1 0.967 0.993 0.981 1.83E�05a2 0.112 0.177 0.143 1.26E�04a3 0.033 0.062 0.045 1.84E�05a4 0.34 0.443 0.393 2.18E�04

Model 2 a1 0.955 0.988 0.974 3.81E�05a2 0.129 0.204 0.17 1.90E�04a3 0.053 0.088 0.07 4.21E�05a4 0.364 0.459 0.411 2.77E�04

Model 3 a1 0.959 0.983 0.972 1.72E�05a2 0.159 0.217 0.184 1.18E�04a3 0.057 0.088 0.073 2.55E�05a4 0.329 0.462 0.394 3.91E�04

MNS: Maximum Nash–Sutcliffe.

PBðy < yiÞ ¼ i=n ð15Þ

where n is the number of sample simulations; PBðy < yiÞ is thequantile of discharge yi.

The 95% confidence intervals due to parameter uncertainty andmodel uncertainty for discharge were calculated by adding themodel residuals in the form of a normally distributed random errorwith zero mean and variance r2 to each of the sample simulationsthat were available for each time step (Eq.(16)). For the daily modelthe bias and autoregressive terms were included as well (Eq.(17)).The inverse of the Box–Cox transformation was then applied.

gi;t ¼ gMi;t þ rnormð0;rÞi ð16Þ

gi;t ¼ gMi;t � bþ a � ðgi;t�1 � gM

i;t�1 � bÞ þ rnormð0;rÞi ð17Þ

yi;t ¼ h�1ðgi;tÞ ð18Þ

where i is the sample index, t is the time index, rnorm is the randomvalue extracted from a normal distribution with zero mean and var-iance r2, and h�1 represents the inverse operation of the Box–Coxtransformation. Then the 5% percentile and 95% percentile of dis-charge due to parameter uncertainty and model uncertainty are de-rived by sorting new discharge values yi;t were sorted at each timestep (Eq. (18)). And Eq. (15) was used to obtain the empiricaldistributions.

3.6. Evaluation of uncertainty interval estimates

Three indices were used to evaluate the resolution, reliabilityand efficiency of the uncertainty interval estimates: (1) the Aver-age Relative Interval Length (ARIL) by Jin et al. (2010), (2) the per-centage of observations bracketed by the Confidence Interval (PCI)

WASMOD.

Skewness 5% Quantile 50% Quantile 95% Quantile MNS

�0.164 0.974 0.981 0.988 0.8050.074 0.125 0.143 0.1620.42 0.038 0.044 0.052�0.022 0.368 0.393 0.416

�0.156 0.963 0.974 0.984 0.831�0.206 0.145 0.17 0.192�0.136 0.059 0.071 0.08�0.038 0.383 0.411 0.437

�0.186 0.965 0.972 0.979 0.8210.715 0.17 0.182 0.206�0.393 0.064 0.074 0.081

0.186 0.363 0.394 0.429

60 L. Li et al. / Journal of Hydrology 406 (2011) 54–65

(Li et al., 2009) and (3) the maximum Nash–Sutcliffe value (MNS)(Nash and Sutcliffe, 1970).

ARIL measures the resolution of the predictive distributions:

ARILp ¼1n

X LimitUpper;t;p � LimitLower;t;p

Robs;tð19Þ

LimitUpper;t;p and LimitLower;t;p are the upper and lower boundary val-ues of the p confidence interval, n is the number of time steps,Robs;t is the observed discharge. ARIL should be as small as possibleand was plotted as a function of p.

The PCI measures the reliability of the predictive distributionsand it is based on the count, the number of the observed data with-in a range of simulated intervals, which is a function of p.

Table 5The Correlation between the parameters for monthly WASMOD estimated byBayesian method.

Correlation a1 a2 a3

a1 1.000 �0.003 �0.029a2 �0.003 1.000 0.725a3 �0.029 0.725 1.000

0.95 0.975 10

0.1

0.2

0.3

0.4

Frequency

A1

0.1 0.2 0.30

0.1

0.2

0.3

0.4

Frequency

A2

0.95 0.975 10

0.1

0.2

0.3

0.4

Frequency

A1

0.1 0.2 0.30

0.1

0.2

0.3

0.4

Frequency

A2

0.95 0.975 10

0.1

0.2

0.3

0.4

Frequency

A1

0.1 0.2 0.30

0.1

0.2

0.3

0.4

Frequency

A2

(a)

(b)

(c)

Fig. 4. Histograms of parameters of daily WASMOD derived by Metropol

PCIp ¼NQ in;p

nð20Þ

NQin;p is the number of observations which are contained within thep confidence interval. PCI was plotted as a function of p and shouldbe close to the 45� diagonal line.

The maximum Nash–Sutcliffe value (MNS) measures the effi-ciency of the model predictions.

MNS ¼maxN

i¼1fNSig ð21Þ

NS ¼ 1�

PTt¼1ðRobs;t � Rsim;tÞ2

PTt¼1ðRobs;t � RobsÞ2

ð22Þ

in which, i is the acceptable sample index, t is the time index, Robs;t isthe observed discharge, Rsim;t is the simulated discharge, and Robs isthe average value of Robs;t . MNS should be as close to 1 as possible.

However, the previous research (Li et al., 2010b) shows that it isnot adequate to judge the uncertainty results by only ARIL or PCI,because they vary simultaneously. To facilitate a more directuncertainty assessment for 95% confidence interval of discharge,

0 0.05 0.10

0.1

0.2

0.3

0.4

Frequency

A3

0.3 0.4 0.50

0.1

0.2

0.3

0.4

Frequency

A4

0 0.05 0.10

0.1

0.2

0.3

0.4

Frequency

A3

0.3 0.4 0.50

0.1

0.2

0.3

0.4

Frequency

A4

0 0.05 0.10

0.1

0.2

0.3

0.4

Frequency

A3

0.3 0.4 0.50

0.1

0.2

0.3

0.4

Frequency

A4

is Hasting method using: (a) Model 1; (b) Model 2 and (c) Model 3.

L. Li et al. / Journal of Hydrology 406 (2011) 54–65 61

this paper proposes a new index based on ARIL and PCI, called thePercentage of observations bracketed by the Unit Confidence Inter-val (PUCI):

PUCI ¼ ð1:0� AbsðPCI � 0:95ÞÞ=ARIL ð23Þ

From Eq. (23), we can see that the PUCI is ranges from zero toinfinity, and the upper boundary is not clear, which is a weak pointof the index. It’s only used in 95% confidence interval evaluation. Infact the larger the PUCI the lower the uncertainty of 95% confidenceinterval of discharge is.

4. Results and discussion

4.1. Parameter estimates and parameter uncertainty

Five parallel simulation chains are used in the MH-algorithmwith 5000 iterations each. The acceptance rate is ranging between20% and 30%. The total number of samples of each parameter is25,000. The Scale Reduction Score

ffiffiffiRp

for all the parameters isapproximately 1.0, indicating convergence of the iterative proce-dure for a chain.

Table 6The Correlation between the parameters for daily WASMOD estimated by Bayesian metho

Model 1 Model 2

a1 a2 a3 a4 a1 a2

a1 1 0.468 �0.617 �0.283 1 0.338a2 0.468 1 0.189 0.259 0.338 1a3 �0.617 0.189 1 0.327 �0.380 0.259a4 �0.283 0.259 0.327 1 �0.272 0.148

0

200

400

600

800

1979-01 1980-01 1981-01 1982-01 1983-01 198Month

Q(m

m)

Fig. 5. The 95% confidence intervals of monthly discharge (1979/1–1988/12) due to parathe posterior distribution (solid) and observed discharge (dots).

(a)

0

0.4

0.8

1.2

1.6

2

0 0.2 0.4 0.6 0.8 1Confidence Interval

AR

IL

Fig. 6. The Average Relative Interval Length (ARIL) and the percentage of observations b(Probability) for 1979–1988 monthly runoff in Jiuzhou river basin.

Fig. 3 shows the marginal posterior parameter densities calcu-lated by Bayesian method for monthly WASMOD. Sharp and peakydistributions show well identifiable parameters, while flat distri-butions indicate a larger parameter uncertainty. Fig. 3 shows thatall parameters have well-defined posterior distributions, fromwhich parameter estimates can be clearly inferred as modal values.The marginal posterior distributions for all parameters are nearlysymmetric as indicated by the skewness coefficients in Table 3and also visible in Fig. 3. Besides, it is evident that the interval ofparameter a1 is very narrow and parameter a3 has the mostskewed distribution.

Correlations between the parameters of monthly WASMODestimated from Bayesian samples are represented in Table 5. Theresults show that the correlation coefficients range between�0.003 and 0.725. The correlation coefficients of all parametersare quite small except between parameter a2 and a3 with a valueof 0.725. It shows that the slow flow and fast flow are highly cor-related since both of them are related to soil moisture content.For daily WASMOD, the marginal posterior parameter densitiescalculated by Models 1–3 are shown in Fig. 4. The following canbe deduced from the Figure: (1) by comparing the posterior distri-butions of parameters derived by the three models, those esti-mated by Model 1 are slightly different; the variances and

d.

Model 3

a3 a4 a1 a2 a3 a4

�0.380 �0.272 1 0.341 �0.488 �0.3630.259 0.148 0.341 1 0.352 0.3051 0.231 �0.488 0.352 1 0.2820.231 1 �0.363 0.305 0.282 1

4-01 1985-01 1986-01 1987-01 1988-01(Y-M)

95% confidence intervalObservedSimulated

meter and model uncertainty (grey bands), simulated discharge at the maximum of

(b)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1Confidence Interval

PCI

racketed by the Confidence Interval (PCI) due to different confidence interval value

0

0.2

0.4

0.6

0.8

1

0 0.4 0.8 1.2 1.6 2ARIL

PCI

Fig. 8. The relation of ARIL and PCI due to different confidence interval value(Probability) for 1979–1988 monthly runoff in Jiuzhou river basin.

62 L. Li et al. / Journal of Hydrology 406 (2011) 54–65

effective parameter space obtained from Model 3 is more similar tothat from Model 1 than Model 2 and (2) the posterior distributionsobtained from Model 1 are somehow sharper and narrower thanthose obtained from Model 2 and Model 3, indicating that a slightlymore optimal parameter set has been identified.

This is confirmed by the variances of parameter samples givenin Table 4, which reports posterior summary statistics for eachparameter estimated by three likelihood functions for daily WAS-MOD. It shows several interesting features: (1) the variance ofparameter samples estimated by Model 1 is smaller than thosefrom Model 2 and Model 3; (2) the interval length for parametera4 is nearly the same for the three methods, while for the otherthree parameters, the samples by Model 2 and Model 3 haveslightly larger intervals than those by Model 1; (3) the marginalposterior distributions for all parameters except for a3 of Model1 and a2 of Model 3, are nearly symmetric as demonstrated bythe skewness coefficients and (4) the maximum Nash–Sutcliffe(MNS) values of Model 1, Model 2 and Model 3 are 0.805, 0.831and 0.821, respectively. Obviously, the MNS from Model 1 is thesmallest. It indicates that Model 1 has low accuracy although ithas the narrowest interval of posterior distribution for parameters.It means that residuals autocorrelation in daily WASMOD is more

(a)

0

10

20

30

40

1 51 101 151Day of ye

Q(m

m)

(b)

0

10

20

30

40

1 51 101 151Day of ye

Q(m

m)

(c)

0

10

20

30

40

1 51 101 151Day of ye

Q(m

m)

Fig. 7. The 95% confidence intervals of daily discharge (1982) due to parameter and modistribution (solid) and observed discharge (dots) for (a) Model 1; (b) Model 2 and (c) M

important, which needs to be considered for uncertainty assess-ment purposes.

Table 6 shows the correlations between the parameters samplesestimated from the three Models. We see that compared to themonthly WASMOD, the average correlation between the parame-ters has increased slightly, but is still in the same range. And

201 251 301 351ar 1982

95% confidence intervalObservedSimulated

201 251 301 351ar 1982

95% confidence intervalObservedSimulated

201 251 301 351ar 1982

95% confidence intervalObservedSimulated

del uncertainty (grey bands), simulated discharge at the maximum of the posteriorodel 3.

Table 7The comparison of uncertainty measures between two hydrological models for the 95% confidence interval.

Monthly WASMOD(79–88) Daily WASMOD (78–82)

Model 1 Model 2 Model 3

Methods ARIL PCI (%) ARIL PCI (%) ARIL PCI (%) ARIL PCI (%)

Bayesian_P 0.250 29.2 0.451 28.2 0.433 26.6 0.290 11.5Bayesian_PM 1.516 94.2 2.642 96.4 2.512 96.2 1.556 92.9

Bayesian_P represents the 95% confidence interval is only due to parameter uncertainty.Bayesian_PM represents the 95% confidence interval is due to parameter and model uncertainty.

Table 8The PUCI based on low, medium, high and all flows due to 95% confidence interval for1978–1982 daily runoffs in Jiuzhou river basin.

Low Flows Medium Flows High Flows All Flows

Model 1 0.255 0.428 0.805 0.373Model 2 0.263 0.442 0.813 0.393Model 3 0.610 0.607 0.675 0.629

L. Li et al. / Journal of Hydrology 406 (2011) 54–65 63

now the largest correlation is between a1 and a3 and not a2 and a3as in the monthly WASMOD.

4.2. Confidence interval of discharge due to parameters and modeluncertainty

For monthly WASMOD, the 95% confidence intervals of dis-charge from 1979 to 1988 due to parameter and model uncertaintyare plotted in Fig. 5. It shows that most of the observed data hasbeen covered by the 95% confidence intervals, which can also beseen more clearly from Fig. 6 presenting the plots of change of Aver-age Relative Interval Length (ARIL) and the percentage of observa-tions bracketed by the Confidence Interval (PCI) due to differentconfidence interval values for monthly WASMOD. It indicates thatboth ARIL and PCI are increasing with the increase of confidenceinterval. Furthermore, the relation of ARIL and PCI due to differentconfidence interval value (Probability) was shown in Fig. 8.

Table 7 reports the uncertainty measures for different likeli-hood models for two different time-scale hydrological models forthe 95% confidence interval. Due to parameter uncertainty andcombined parameter and model uncertainty for monthly WAS-MOD, ARIL values derived by Bayesian method are 0.250 and1.516, respectively, while the PCI values are 29.2% and 94.2%,respectively. This indicates that the uncertainty in the simulateddischarges resulting from parameter uncertainty is less importantthan that resulting from total uncertainty.

The 95% confidence intervals of daily discharge from 1982 esti-mated by three statistical models due to parameter and modeluncertainty for daily WASMOD are plotted in Fig. 7 for illustrativepurpose. Fig. 7a and b shows that there is no difference betweenresults obtained by Model 1 and Model 2 on the basis of visualcomparison, while the confidence intervals estimated by Model 3are different in the two periods (Fig. 7c). It indicates that the con-fidence intervals in dry periods derived by Model 3 are narrowerthan those derived from Model 2 and Model 1.

The difference of 95% confidence intervals between three statis-tical models can be seen more clearly in Table 7, which shows (1)

Table 9Comparison of application and computational efficiency (Computer with Intel(R) Core(TM

Monthly WASMOD (1979–1988)

Difficulty of implement Very easyNumber of runs � number of chains 5000 � 5Time 1 min

ARIL of 95% confidence interval due to parameter and model uncer-tainty derived from Model 2 is narrower than those from Model 1.Furthermore, the PCI obtained by Model 2 is nearly the same asthat of Model 1; (2) ARIL of 95% confidence interval due to param-eter and model uncertainty derived from Model 3 is the smallestwith a value of 1.556, or less than 60% of 2.512 and 2.642 fromModel 2 and Model 1, respectively. Meanwhile, the PCI of Model3 is 92.9% which is quite close to 96.2% of Model 2 and 96.4% ofModel 1. The explanation lies in the fact that the statistical param-eters of Model 3 depend on simulation periods which makes theinterval of dry period discharge to be much narrower and the resultthus more reliable and less uncertain and (3) ARIL of 95% confi-dence interval due to parameter uncertainty is much narrowerthan those due to both of parameter uncertainty and model uncer-tainty for all statistical models in monthly and daily WASMOD.Above results reveal that the parameter uncertainty is not the mostimportant key in uncertainty estimates for both monthly and dailyhydrological models.

However, from results of Table 7, it is difficult to identify whichlikelihood function is the best for daily WASMOD when consider-ing both ARIL and PCI as indices for confidence interval evaluation.For instance Model 1 has the biggest PCI value while Model 3 hasthe smallest ARIL value. In order to settle this problem and makethe comparison easier and clearer, a new index as defined in Eq(23) is used for comparison and the results are shown in Table 8.In which the values of PUCI based on low, medium and high flowsdue to 95% confidence interval for 1978–1982 daily runoffs in theJiuzhou basin are compared for the three likelihood functions. Itcan be clearly seen that (1) Model 3 has the largest PUCI of 0.629and Model 1 has the smallest PUCI of 0.373 for all flows. Conse-quently Model 3 is the best for daily time steps, highlighting theimportance for the time-dependent statistical parameters; (2) thePUCI of Model 2 is the largest based on the high flow, implying thatModel 2 is less uncertain and better in high flow estimate than theother two models and (3) an inter-comparison of Model 1 withModel 2 without time dependence shows in terms of PUCI valuesthat Model 2 is superior to Model 1 over the whole range of flows.It reflects that Model 2 which uses more auto-correlation stepsover the time periods outperforms Model 1.

4.3. Comparison of application and computational efficiency

Table 9 shows the comparison of application and computationalefficiency of statistical models in WASMOD with differenttime-scales. From the running time, we can see that Model 2 is

) 2 Duo CPU T8300 @ 2.40 GHz and 2.39 GHz, 1.96 GB of RAM).

Daily WASMOD (1978–1982)

Model 1 Model 3 Model 2

Little complicated More complicated Very complicated5000 � 5 5000 � 5 2000 � 5141.6 min 188.4 min 23 days

64 L. Li et al. / Journal of Hydrology 406 (2011) 54–65

computationally more demanding than Model 1 and Model 3.Implementation of model 1 is very easy compared to Model 2. Thisis due to the fact that Model 2 considers the autocorrelation ofwhole sample periods, which implies hundreds or thousands ofmatrix calculations for the more complex likelihood function andsubsequent processing. For monthly WASMOD, the computationalefficiency is much higher than that of daily WASMOD. For dailyWASMOD, although it is more efficient for Model 2 to find the cor-rect posterior distribution of parameters than for Model 1 andModel 3, Model 2 is computationally more expensive than Models1 and Model 3, which is certainly the main disadvantage of usingthis model in the Bayesian method for uncertainty assessment.

5. Conclusions

This study performed a comprehensive comparison and evalua-tion of uncertainty estimation through the Bayesian method byapplying the Metropolis Hasting (MH) algorithm to two well-tested conceptual hydrological models (WASMOD) operated withmonthly and daily time step. Several likelihood functions havebeen constructed to supply the uncertainty estimates in monthlyand daily hydrological models. The Box–Cox transformation wasused on the observed and simulated flows in all likelihood func-tions. For monthly WASMOD a statistical Gaussian error modelwas constructed for the residuals after the transformation. For dai-ly WASMOD, the AR (1) plus Multi-Normal model was used afterthe transformation (Model 2). Furthermore, Model 1 and Model 3compared the results derived by two likelihood functions whosestatistical parameters dependent on time and independent ontime, respectively. A novel index called Percentage of observationsbracketed by the Unit Confidence Interval (PUCI) has been proposedfor uncertainty evaluation of discharges. The following conclusionsare drawn from the results of this study:

The autocorrelations of residuals in monthly WASMOD and dai-ly WASMOD are different. In monthly WASMOD, it is not importantto consider the autocorrelation, whereas in daily WASMOD it isnecessary.

In monthly WASMOD, the posterior distributions of parameterswere sharp and peaky which is associated with well identifiablemodel parameter sets. Besides, the goodness of model fit, as mea-sured by the maximum Nash–Sutcliffe efficiency value, is compa-rable to the value calculated by the optimization algorithm.

The AR (1) model is not adequate enough to estimate the distri-bution of residuals in daily WASMOD. The values of PUCI show thatModel 2 outperforms Model 1, although the posterior distributionsof parameters derived by Model 1 are narrower than those ob-tained by the Model 2. Furthermore, the maximum Nash–Sutcliffeefficiency value obtained by Model 2 is 0.831, which is much high-er than 0.805 achieved by Model 1. We conclude that the estimatedparameters derived from Model 2 are more accurate than that ofModel 1.

For daily WASMOD, the values of PUCI show that Model 3 per-forms the best over the entire flow range, while Model 2 is the bestfor high flows. Moreover, the MNS derived from Model 3 is 0.821,which is slightly smaller than that of Model 2 which is in turn lar-ger than that of Model 1. This is because Model 3 uses more statis-tical parameters which reflect more appropriately the statisticalproperties of the residuals than Model 1. In fact Model 3 performssuperior on 95% confidence interval of discharge estimates thanModel 1.

Considering the difference in application and computationalefficiency, we can see that a simple Gaussian error model is suit-able for monthly WASMOD since there is no auto-correlation intransformed monthly residuals, while the value of PUCI shows thatModel 3 is the best for daily WASMOD. But if considering the

uncertainty of high flows and accuracy of estimated parameters,Model 2 is superior compared to Model 1 and Model 3 for dailyWASMOD.

Despite that Model 2 outperforms in high flows for daily WAS-MOD, the PUCI of Model 2 is smaller than that of Model 3 for allflows, which shows that Model 2 is also sub-optimal for dailyWASMOD. It is better for statistical models to split the simulationperiod. Two periods (dry and wet period) for the statistic parame-ters are probably not enough, which means that additional timesections need to be considered for Bayesian analyses in daily WAS-MOD. Furthermore, some low flows and high flows do not fit thenormal distribution after the Box–Cox transformation and AR (1)modelling in daily WASMOD, which implies that further effortsare required to improve the formulation of likelihood functionsused in hydrological applications since the assumption of normaldistribution of the residuals in daily WASMOD over the entire flowrange is inappropriate. Finally, one of the major challenges is tofind an appropriate likelihood function for the error distributionwhen extreme flows are outside the normal distribution. In viewof the computational effort required by Model 2, another impor-tant issue is to find an improved way to account for error auto-cor-relation in daily hydrological models. We observe that althoughthe study has led to interesting findings, the generality of the re-sults requires using additional hydrological models and study re-gions in future research.

Acknowledgments

This study was supported by Key Project of the Natural ScienceFoundation of China (No. 40730632), the Norwegian ResearchCouncil (RCN) for sponsoring project number 171783 (FRIMUF)and the Environment and Development Programme (FRIMUF) ofthe Research Council of Norway, project 190159/V10 (SoCoCA).The authors would like to thank Mario L.V. Martina for his adviceand help in this study. We also appreciate the comments and sug-gestions from two reviewers and the associate editor, Prof. EzioTodini. Their comments greatly helped to improve the quality ofthis paper.

References

Abbaspouretal, K.C. et al., 2007. Modelling hydrology and water quality in the pre-alpine/alpine Thur watershed using SWAT. Journal of Hydrology 333 (2–4),413–430.

Bates, B.C., Campbell, E.P., 2001. A Markov chain Monte Carlo scheme for parameterestimation and inference in conceptual rainfall–runoff modeling. WaterResources Research 37 (4), 937–947.

Benke, K.K., Lowell, K.E., Hamilton, A.J., 2008. Parameter uncertainty, sensitivityanalysis and prediction error in a water-balance hydrological model.Mathematical and Computer Modelling 47 (11–12), 1134–1149.

Bernardo, J.M., Smith, A.F., 1994. Bayesian Theory. Wiley, Chichester.Beven, K.J., Freer, J., 2001. Equifinality, data assimilation, and uncertainty estimation

in mechanistic modelling of complex environmental systems using the GLUEmethodology. Journal of Hydrology 249 (1–4), 11–29.

Blasone, R.S., Madsen, H., Rosbjerg, D., 2008a. Uncertainty assessment of integrateddistributed hydrological models using GLUE with Markov chain Monte Carlosampling. Journal of Hydrology 353 (1–2), 18–32.

Blasone, R.S. et al., 2008b. Generalized likelihood uncertainty estimation (GLUE)using adaptive Markov Chain Monte Carlo sampling. Advances in WaterResources 31 (4), 630–648.

Chen, X., Chen, Y.D., Xu, C.-Y., 2007. A distributed monthly hydrological model forintegrating spatial variations of basin topography and rainfall. HydrologicalProcesses 21 (2), 242–252.

Chib, S., Greenberg, E., 1995. Understanding the Metropolis–Hastings algorithm.The American Statistician 49, 327–335.

Chowdhury, S., Sharma, A., 2007. Mitigating parameter bias in hydrologicalmodelling due to uncertainty in covariates. Journal of Hydrology 340 (3–4),197–204.

Coccia, G., Todini, E., 2010. Recent developments in predictive uncertaintyassessment based on the model conditional processor approach. Hydrologyand Earth System Sciences and Discussion 7, 9219–9270.

L. Li et al. / Journal of Hydrology 406 (2011) 54–65 65

Duan, Q., Ajami, N.K., Gao, X., Sorooshian, S., 2007. Multi-model ensemblehydrologic prediction using Bayesian model averaging. Advances in WaterResources 30 (5), 1371–1386.

Engeland, K., Braud, I., Gottschalk, L., Leblois, E., 2006. Multi-objective regionalmodelling. Journal of Hydrology 327 (3–4), 339–351.

Engeland, K., Gottschalk, L., 2002. Bayesian estimation of parameters in a regionalhydrological model. Hydrology and Earth System Sciences 6 (5), 883–898.

Engeland, K., Renard, B., Steinsland, I., Kolberg, S., 2010. Evaluation of statisticalmodels for forecast errors from the HBV model. Journal of Hydrology 384 (1–2),142–155.

Engeland, K., Xu, C.Y., Gottschalk, L., 2005. Assessing uncertainties in a conceptualwater balance model using Bayesian methodology. Hydrological SciencesJournal 50 (1), 45–63.

Gelman, A., Rubin, D.B., 1992. Inference from iterative simulation using multiplesequences. Statistical Science 7, 457–472.

Gong, L., Widen-Nilsson, E., Halldin, S., Xu, C.Y., 2009. Large-scale runoff routingwith an aggregated network-response function. Journal of Hydrology 368 (1–4),237–250.

Hopper, M.J., 1978. A Catalogue of Subroutines. Harwell Subroutine Library, UKAtomic Energy Authority, Harwell.

Jin, X., Xu, C.-Y., Zhang, Q., Singh, V.P., 2010. Parameter and modeling uncertaintysimulated by GLUE and a formal Bayesian method for a conceptual hydrologicalmodel. Journal of Hydrology 383 (3–4), 147–155.

Kavetski, D., Kuczera, G., Franks, S.W., 2006a. Bayesian analysis of input uncertaintyin hydrological modeling. 1. Theory. Water Resources Research 42 (3), W03407.

Kavetski, D., Kuczera, G., Franks, S.W., 2006b. Bayesian analysis of input uncertaintyin hydrological modeling. 2. Application. Water Resources Research 42 (3),W03408.

Krzysztofowicz, R., 1997. Transformation and normalization of variates withspecified distributions. Journal of Hydrology 197 (1–4), 286–292.

Krzysztofowicz, R., 1999. Bayesian theory of probabilistic forecasting viadeterministic hydrologic model. Water Resources Research 35 (9), 2739–2750.

Krzysztofowicz, R., 2002. Bayesian system for probabilistic river stage forecasting.Journal of Hydrology 268 (1–4), 16–40.

Krzysztofowicz, R., Herr, H.D., 2001. Hydrologic uncertainty processor forprobabilistic river stage forecasting: precipitation-dependent model. Journalof Hydrology 249 (1–4), 46–68.

Krzysztofowicz, R., Kelly, K.S., 2000. Hydrologic uncertainty processor forprobabilistic river stage forecasting. Water Resources Research 36 (11), 3265–3277.

Kuczera, G., Kavetski, D., Franks, S., Thyer, M., 2006. Towards a Bayesian total erroranalysis of conceptual rainfall–runoff models: characterising model error usingstorm-dependent parameters. Journal of Hydrology 331 (1–2), 161–177.

Kuczera, G., Parent, E., 1998. Monte Carlo assessment of parameter uncertainty inconceptual catchment models: the Metropolis algorithm. Journal of Hydrology211 (1–4), 69–85.

Langsrud, Ø., Frigessi, A., Høst, G., 1998. Pure model error for the HBV-model.Norwegian Water Resources and Energy Directorate. Hydra Note 4/1998, Oslo,28pp.

Li, L., Xia, J., Xu, C.-Y., Chu, J., Wang, R., 2009. Analyse the sources of equifinality inhydrological model using GLUE methodology. In: Hydroinformatics inHydrology, Hydrogeology and Water Resources (Proc. of Symposium JS.4 atthe Joint IAHS & IAH Convention, Hyderabad, India, September 2009) IAHSPubl., no. 331, pp. 130–138.

Li, Z., Shao, Q., Xu, Z., Cai, X., 2010a. Analysis of parameter uncertainty in semi-distributed hydrological models using bootstrap method: a case study of SWATmodel applied to Yingluoxia watershed in northwest China. Journal ofHydrology 385 (1–4), 76–83.

Li, L., Xia, J., Xu, C.-Y., Singh, V.P., 2010b. Evaluation of the subjective factors of theGLUE method and comparison with the formal Bayesian method in uncertaintyassessment of hydrological models. Journal of Hydrology 390 (3–4), 210–221.

Lilliefors, H.W., 1967. On the Kolmogorov–Smirnov test for normality with meanand variance unknown. Journal of the American Statistical Association 62 (318),399–402.

Liu, Z., Martina, M.L.V., Todini, E., 2005. Flood forecasting using a fully distributedmodel: application of the TOPKAPI model to the Upper Xixian Catchment.Hydrology and Earth System Sciences 9 (4), 347–364.

Looper, J.P., Vieux, B.E., Moreno, M.A., 2009. Assessing the impacts of precipitationbias on distributed hydrologic model calibration and prediction accuracy.Journal of Hydrology, doi:10.1016/j.jhydrol.2009.09.048.

Mantovan, P., Todini, E., 2006. Hydrological forecasting uncertainty assessment:incoherence of the GLUE methodology. Journal of Hydrology 330 (1–2), 368–381.

Maranzano, C.J., Krzysztofowicz, R., 2004. Identification of likelihood and priordependence structures for hydrologic uncertainty processor. Journal ofHydrology 290 (1–2), 1–21.

Marshall, L., Nott, D., Sharma, A., 2007. Towards dynamic catchment modelling: aBayesian hierarchical mixtures of experts framework. Hydrological Processes21 (7), 847–861.

Montanari, A., Brath, A., 2004. A stochastic approach for assessing the uncertainty ofrainfall–runoff simulations. Water Resources Research 40, W01106.

Montanari, A., Grossi, G., 2008. Estimating the uncertainty of hydrological forecasts:a statistical approach. Water Resources Research 44, W00B08.

Nash, J.E., Sutcliffe, J.V., 1970. River flow forecasting through conceptual models.Part 1. A discussion of principles. Journal of hydrology 10 (3), 282–290.

Raftery, A.E., Balabdaoui, F., Gneiting, T., Polakowski, M., 2003. Using BayesianModel Averaging to Calibrate Forecast Ensembles. Tech. Rep. 440, Departmentof Statistics, University of Washington, Seattle.

Raftery, A.E., Gneiting, T., Balabdaoui, F., Polakowski, M., 2005. Using Bayesianmodel averaging to calibrate forecast ensembles. Monthly Weather Review 133(5), 1155–1174.

Reggiani, P., Weerts, A.H., 2008. A Bayesian approach to decision-making underuncertainty: an application to real-time forecasting in the river Rhine. Journal ofHydrology 356 (1–2), 56–69.

Romanowicz, R., Beven, K.J., Tawn, J., 1994. Evaluation of predictive uncertainty innonlinear hydrological models using a Bayesian approach. Statistics for theEnvironment 2, 297–317.

Thiemann, M., Trosser, M., Gupta, H., Sorooshian, S., 2001. Bayesian recursiveparameter estimation for hydrologic models. Water Resources Research 37 (10),2521–2536.

Todini, E., 2007. Hydrological catchment modelling: past, present and future.Hydrology and Earth System Sciences 11 (1), 468–482.

Todini, E., 2008. A model conditional processor to assess predictive uncertainty inflood forecasting. International Journal of River Basin Management 6 (2), 123–138.

Todini, E., 2011. History and perspectives of hydrological catchment modeling.Hydrology Research 42 (2–3), 73–85.

Van der Waerden, B.L., 1952. Order tests for the two-sample problem and theirpower. Indagationes Mathematicae 14 (253), 453–458.

Van der Waerden, B.L., 1953a. Order tests for the two-sample problem and theirpower 2. Indagationes Mathematicae 15, 303–310.

Van der Waerden, B.L., 1953b. Order tests for the two-sample problem and theirpower 3. Indagationes Mathematicae 15, 311–316.

Vogel, R.M., Stedinger, J.R., Batchelder, R., Lee, S.U., 2008. Appraisal of theGeneralized Likelihood Uncertainty Estimation (GLUE) Method. AmericanSociety of Civil Engineers, 1801 Alexander Bell Drive, Reston, VA, 20191-4400,USA.

Vrugt, J.A., Gupta, H.V., Bouten, W., Sorooshian, S., 2003. A Shuffled ComplexEvolution Metropolis algorithm for optimization and uncertainty assessment ofhydrologic model parameters. Water Resources Research 39 (8), 1201.

Wang, G., Xia, J., Chen, J., 2009. Quantification of effects of climate variations andhuman activities on runoff by a monthly water balance model: a case study ofthe Chaobai River basin in northern China. Water Resources Research 45 (7),W00A11.

Widen-Nilsson, E., Gong, L., Halldin, S., Xu, C.-Y., 2009. Model performance andparameter behavior for varying time aggregations and evaluation criteria in theWASMOD-M global water balance model. Water Resources Research 45 (5),W05418.

Widen-Nilsson, E., Halldin, S., Xu, C.-Y., 2007. Global water-balance modelling withWASMOD-M: Parameter estimation and regionalisation. Journal of Hydrology340 (1–2), 105–118.

Xiong, L., O’Connor, K.M., 2008. An empirical method to improve the predictionlimits of the GLUE methodology in rainfall–runoff modeling. Journal ofHydrology 349 (1–2), 115–124.

Xiong, L., Wan, M., Wei, X., O’Connor, K.M., 2009. Indices for assessing the predictionbounds of hydrological models and application by generalised likelihooduncertainty estimation. Hydrological Sciences Journal/Journal des SciencesHydrologiques 54 (5), 852–871.

Xu, C.-Y., 2002. WASMOD – the water and snow balance modelling system. In:Singh, V.P., Frevert, D.K. (Eds.), Mathematical Models of Small WatershedHydrology and Applications. Water Resources Publications, LLC, Chelsea,Michigan, USA, pp. 555–590.

Xu, C.Y., Vandewiele, G.L., 1995. Parsimonious monthly rainfall–runoff models forhumid basins with different input requirements. Advances in Water Resources18 (1), 39–48.

Yang, J., Reichert, P., Abbaspour, K.C., 2007a. Bayesian uncertainty analysis indistributed hydrologic modeling: a case study in the Thur River basin(Switzerland). Water Resources Research 43 (10), W10401.

Yang, J., Reichert, P., Abbaspour, K.C., Xia, J., Yang, H., 2008. Comparing uncertaintyanalysis techniques for a SWAT application to the Chaohe Basin in China.Journal of Hydrology 358 (1–2), 1–23.

Yang, J., Reichert, P., Abbaspour, K.C., Yang, H., 2007b. Hydrological modelling of theChaohe Basin in China: statistical model formulation and Bayesian inference.Journal of Hydrology 340 (3–4), 167–182.

Yapo, P.O., Gupta, H.V., Sorooshian, S., 1998. Multi-objective global optimization forhydrologic models. Journal of Hydrology 204 (1–4), 83–97.