ewater modelling guidelines rrm (v1 mar 2012)

Towards best practice model application

Vaze, J., Jordan, P., Beecham, R., Frost, A., Summerell, G.

Guidelines for rainfall-runoff modelling


2

Copyright Notice

2012 eWater Ltd

Legal Information

This work is copyright. You are permitted to copy and reproduce the information, in an unaltered form, for

non-commercial use, provided you acknowledge the source as per the citation guide below. You must not

use the information for any other purpose or in any other manner unless you have obtained the prior written

consent of eWater Ltd.

While every precaution has been taken in the preparation of this document, the publisher and the authors

assume no responsibility for errors or omissions, or for damages resulting from the use of information

contained in this document. In no event shall the publisher and the author be liable for any loss of profit or

any other commercial damage caused or alleged to have been caused directly or indirectly by this

document.

Citing this document

Vaze, J., Jordan, P., Beecham, R., Frost, A., Summerell, G. (eWater Cooperative Research Centre 2011)

Guidelines for rainfall-runoff modelling: Towards best practice model application.

Publication date: March 2012 (Version 1.0)

ISBN 978-1-921543-51-7

Acknowledgments

eWater CRC acknowledges and thanks all partners to the CRC and individuals who have contributed to the

research and development of this publication.

We acknowledge the inputs from the hydrology group in DERM, Queensland, and Mark Alcorn from SA

Department for Water. We thank Matthew Bethune, Peter Wallbrink, Dugald Black, Jin Teng, Jean-Michel

Perraud, Melanie Ryan, Bill Wang, David Waters, Richard Silberstein, Geoff Podger, David Post, Cuan

Petheram, Francis Chiew and Andrew Davidson for useful discussions.

eWater CRC gratefully acknowledges the Australian Governments financial contribution to this project

through its agencies, the Department of Innovation, Industry, Science and Research, the Department of

Sustainability, Environment, Water, Population and Communities and the National Water Commission

For more information:

Innovation Centre, Building 22

University Drive South

Bruce, ACT, 2617, Australia

T: +61 2 6201 5834 (outside Australia)

Support: 1300 5 WATER (1300 592 937)

E: [email protected]

www.ewater.com.au


3

Contents

1 Introduction ................................................................. 5

1.1 Background ........................................................................................................................... 5

1.2 Definition of Best Practice ..................................................................................................... 5

1.3 Scope .................................................................................................................................... 6

2 Overview of procedure for rainfall-runoff modelling .... 8

2.1 Problem definition ................................................................................................................. 8

Problem statement and setting objectives ............................................................................ 8

Understanding the problem domain ...................................................................................... 8

Metrics and criteria and decision variables ........................................................................... 9

Performance across multiple catchments and subcatchments ............................................. 9

2.2 Option modelling ................................................................................................................... 9

Methodology development .................................................................................................... 9

Collate and review data ...................................................................................................... 10

Setting up and building a model ......................................................................................... 10

Calibration and Validation ................................................................................................... 10

Sensitivity/uncertainty analysis ........................................................................................... 12

Documentation and Provenance ........................................................................................ 12

Model acceptance and accreditation .................................................................................. 13

Use of accepted/accredited model...................................................................................... 13

3 Model choice ............................................................. 14

3.1 Model selection ................................................................................................................... 14

3.2 Available models ................................................................................................................. 15

Empirical methods .............................................................................................................. 15

Large scale energy-water balance equations ..................................................................... 16

Conceptual Rainfall-Runoff Models .................................................................................... 16

Landscape daily hydrological models ................................................................................. 17

Fully distributed physically based hydrological models which explicitly model hillslope and

catchment processes .......................................................................................................... 17

4 Collate and Review Data........................................... 20

4.1 Catchment details ............................................................................................................... 21

Location of gauges (streamflow, rainfall and evaporation) ................................................. 21

Topography and Catchment Areas ..................................................................................... 21

Soil types ............................................................................................................................ 21

Vegetation ........................................................................................................................... 21


4

Water Management Infrastructure ...................................................................................... 22

4.2 Flow data ............................................................................................................................ 22

4.3 Rainfall ................................................................................................................................ 23

4.4 Evapotranspiration .............................................................................................................. 24

5 Statistical Metrics for Testing Performance .............. 25

6 Calibration and validation .......................................... 27

6.1 Calibration ........................................................................................................................... 27

6.2 Validation ............................................................................................................................ 27

6.3 Calibration and Validation of Models to Single Gauge Sites, Multiple Gauge Sites and Regionalisation of Model Parameter Sets ......................................................................... 29

6.4 Automated, Manual and Hybrid Calibration Strategies ....................................................... 30

Manual Calibration .............................................................................................................. 30

Automated Calibration ........................................................................................................ 31

Hybrid Calibration Strategies .............................................................................................. 32

Selection of Objective Functions for Automated and Hybrid Calibration ............................ 33

6.5 Calibration of Regression Models ....................................................................................... 37

7 Uncertainty and Sensitivity Analysis ......................... 38

7.1 Sensitivity Analysis ............................................................................................................. 39

7.2 Application of Multiple Parameter Sets ............................................................................... 39

7.3 More Advanced Quantitative Uncertainty Analysis ............................................................. 40

7.4 Consideration of Uncertainty in Practical Applications of Rainfall Runoff Models .............. 40

8 Concluding remarks .................................................. 42

9 References ................................................................ 43


5

1 Introduction

1.1 Background

Reliable estimates of stream flow generated from catchments are required as part of the

information sets that help policy makers make informed decisions on water planning and

management. The characteristics of the streamflow time series that influence water resources

system modelling and planning can include the sequencing of flows on daily and longer time

steps, spatial and temporal variability of flows, seasonal distribution and characteristics of high

and low flows.

The best available estimate of streamflow would be expected to come from water level

observations made at a gauging station, converted to flow estimates using a well defined and

stable rating curve. However, these observations are only available for limited number of

gauging locations and for limited time span. Estimates for ungauged locations and for a much

longer time period are needed for contemporary water management, and ways to make

estimates for future possible conditions are also required.

A range of methods are available to estimate streamflow from catchments, using observed

data wherever possible, or estimating by empirical and statistical techniques, and more

commonly using rainfall-runoff models. The modelling approach used to estimate streamflow

by different water agencies and consultants varies across Australia and depends on the

purpose of the modelling, time and money available, and the tools and skills available within

the organisation. With increasing levels of inter-agency collaboration in water planning and

management, development of a best practice approach in rainfall runoff modelling is desirable

to provide a consistent process, and improve interpretation and acceptability of the modelling

results.

The purpose of this document is to provide guidance on the best practice for implementing fit

for purpose rainfall-runoff models, covering topics such as setting modelling objectives,

identifying data sources, quality assuring data and understanding its limitations, model

selection, calibration approaches, and performance criteria for assessing fitness for purpose

1.2 Definition of Best Practice

Best Practice Modelling can be defined as a series of quality assurance principles and actions

to ensure that model development, implementation and application are the best achievable,

commensurate with the intended purpose (Black et al., 2011).

What is in practice best achievable, commensurate with the intended purpose may be

subject to data availability, time, budget and other resourcing constraints. Hence, what is

meant by the term Best Practice Modelling can vary. Not only does it depend on the

circumstances of the project, particularly providing results that are fit for the intended purpose,

but it also depends to a great degree on interpretation in peer review. This, in turn, will be

influenced by the general state of knowledge and technology in the modelling field, which can


6

be expected to progressively develop over time (such as new remote sensing data sources

coming on line, and new computing hardware and software), as well as data, time, budget and

resourcing constraints. Best Practice Modelling provides for a strategic approach to

modelling which enables the trade-offs that may be imposed by these constraints to be better

managed, and assists in identifying priorities for addressing model and data limitations.

1.3 Scope

The eWater CRC has prepared generic Best Practice Modelling guidelines (Black et al., 2011).

They aim to provide for an integrated approach that enables interactions and feedbacks

between all domains relevant to water management (e.g. hydrological, ecological,

engineering, social, economic and environmental) to be considered.

The procedure in that guidance is intended to be flexible enough to accommodate variations in

the meaning of the term Best Practice Modelling and also allow for continuous improvement

as the state of knowledge and technology in the modelling field develops and improves.

The eWater CRC will also provide guidelines to support the BPM guidelines in specific areas of

hydrological modelling that relate to the products that they are developing. This guideline is

intended to address rainfall-runoff model application with the objectives being to provide water

managers, consultants and research scientists with information on rainfall-runoff models and

how to choose one that is fit for purpose, the data available to develop them, and the

calibration and validation methodologies.

There are a number of different purposes that a rainfall runoff model may be applied within an

overall water resources or catchment modelling framework, such as eWater Source. Most of

these purposes relate to providing information to support decision making for some water

management policy. In particular, this can involve:

Understanding the catchment yield, and how this varies in time and space, particularly

in response to climate variability: seasonally, inter-annually, and inter-decadally.

Estimating the relative contributions of individual catchments to water availability over a

much larger region, e.g. valley or basin scale.

Estimating how this catchment yield and water availability might change over time in

response to changes in the catchment, such as increasing development of farm dams,

or changes in land-use and land management.

In some instances with a high quality network of long term stream gauges, most of this type of

information can be estimated from the observations. However, the more common case is

where there is some combination of short term stations, variable quality data, and gaps in

spatial coverage. In these cases, spatial and temporal gaps in the information can be

estimated using rainfall runoff models to:

Infill gaps caused by missing or poor quality data in an observed data series for a

gauged catchment.

Estimate flows for a gauged catchment for periods before the observed flow record

started or after when the observed flow record ends.

Estimate flows for an ungauged catchment.

Estimating flows from ungauged subcatchments within an overall gauged catchment.


7

Forecast flows for some immediate future period (typically for a period of between a few

days and a few months), conditioned on current (or recent) observations of the

catchment state.

Assess the impacts of human influences within a gauged catchment (for example

landuse or vegetation cover change) and simulating the flows that would have occurred

for the historical climate sequence with modified catchment conditions. This may

include assessment of catchment conditions that may be non-stationary in either the

observed record or for the simulation.

Assess the potential impact of climate variability and/or climate change on flows from a

gauged catchment.

In some cases, several of the above purposes may be satisfied by rainfall runoff modelling for

the same catchment. There are similarities in the approach that is taken to rainfall runoff

modelling, even though the purpose may differ.


8

2 Overview of procedure for

rainfall-runoff modelling The generic guidelines (Black et al., 2011) outline a procedure for applying a hydrological

model. This can be summarised as occurring in 4 phases:

1 Project management,

2 Problem definition,

3 Option modelling,

4 Compare Options and select the best.

This guidance deals only with problem definition and option modelling because the first and

last phases are discussed sufficiently for the purpose of rainfall-runoff modelling in the generic

guidelines. A further reason is that rainfall-runoff modelling is usually only a part of a larger

hydrological modelling project and these phases would be most appropriately considered in

the context of that larger project. Specific aspects of project management and option

comparison that are directly applicable to the development of a rainfall-runoff model, such as

accreditation, are dealt with at appropriate points in this guidance.

2.1 Problem definition

Problem statement and setting objectives

The problem to be addressed must be clearly articulated to minimise the risk that the wrong

tool will be used for the job. The problem statement will give direction on what objectives will be

considered in developing the rainfall-runoff model. As many water management decisions will

often have more than one goal it will be important to ensure these are all identified.

Sometimes it can be useful to express objectives in a hierarchy that shows primary objectives,

secondary objectives and so on. In this regard, consideration should also be given to

possible additional future objectives and goals that could be met based on the current project

or on future projects that build upon the model established in the current project. The decision

on which option offers the best solution should be based upon whether, or how well, each

option meets the agreed objectives (also see section 2.2.1 and 2.2.2 in the generic

guidelines).

Understanding the problem domain

The choice of the rainfall-runoff model will vary based on the purpose the modelling is being

done for, e.g., to understand seasonal low flow characteristics for an in-stream environmental

need; or to assess over-bank flow frequency; or to estimate overall catchment yield on an

average annual basis. The model selected, data required, and calibration approach adopted


9

should reflect this requirement. Where the same model may be used for two or more different

purposes, there may also be a need to focus the calibration on a number of different flow

regimes. If rough flow estimates are required over large areas and the runoff generation

methodology should be consistent then the data and modelling process will differ again.

Metrics and criteria and decision variables

Model calibration is a process of systematically adjusting model parameter values to get a set

of parameters which provides the best estimate of the observed streamflow (in the case of

rainfall-runoff models). The process of determining which particular set of parameter values

are best for the intended purpose depends on what comparison metrics are used. Metrics

should be used to quantify the acceptability of the developed model. In all cases graphical

assessment and statistical results of some sort will be assessed to identify the ability of the

calibrated model to reproduce the flows calibrated against.

Different metrics will be more effective in determining model appropriateness to meet different

objectives. What these are should be considered when the problem is being defined.

Understanding appropriate metrics allows model acceptance criteria to be identified.

Performance across multiple catchments and subcatchments

In some situations, the purpose of rainfall runoff modelling is to produce an estimate of the

runoff at a single location where there is a streamflow gauge. If this is the case, the calibration

and validation process may be performed for the single gauged catchment. This approach is

justifiable in situations where gauged data is available for most of the period that flow results

are required for and the purpose of the rainfall runoff model is to infill missing data during the

period of record. It may also be justifiable where there is a requirement to extend the period of

record at the single gauge.

A much more common situation is that flow time series estimates are required at several

locations and that gauged streamflow data is also available at several locations. The locations

where flow estimates are required may or may not overlap with the locations where the flow

data is also available. An objective of any project that involves the application of rainfall runoff

models to multiple catchments or subcatchments should be to demonstrate consistency in the

rainfall runoff model response between those catchments and to explain systematic

differences in the hydrological response between catchments and subcatchments in an

appropriate and logical manner.

2.2 Option modelling

This section describes the process of developing a rainfall-runoff model, further details on key

components are provided in later sections.

Methodology development

The models and methodology employed should be appropriate for the purpose that the model

will be used for. The choices made will be directed by the problem definition developed and

any other information at hand to the modeller. Detail on the models available and guidance on

selecting models and methodology that is fit for purpose is provided in Section 3.


10

Collate and review data

The amount and quality of data available to develop a model should be determined at the

outset of the project. This can influence the selection of models, the performance criteria, and

the approach to calibrate models. A bare minimum data set sufficient to make an approximate

estimate of mean annual catchment yield would include catchment area along with spatial and

temporal characteristics of rainfall and potential evapotranspiration (PET). A comprehensive

data set would include long-term streamflow measurements and rainfall and PET data

collected at one or more locations within the catchment along with land use coverage,

vegetation cover and impervious area information, including changes over time.

The quality of the data should be reviewed prior to using to detect errors, non-stationarity if

any, and understand uncertainties that may influence estimates. Some methods are

discussed in section 4.

Setting up and building a model

The catchment characteristics are considered along with the knowledge on data available and

any other information available to the modeller. The rainfall-runoff model chosen is

conceptualised and an initial parameter set is identified.

When the model is first set up consideration should be given to all constraints which are

limiting and their effects on the modelling. Section 5 provides more details associated with this

step.

Calibration and Validation

Model calibration is a process of systematically adjusting model parameter values to get a set

of parameters which provides the best estimate of the observed streamflow (in the case of

rainfall-runoff models).

The term validation, as applied to models, typically means confirmation to some degree that

the calibration of the model is acceptable for the intended purpose (Refsgaard and Henriksen,

2004). In the context of rainfall runoff modelling, validation is a process of using the calibrated

model parameters to simulate runoff over an independent period outside the calibration period

(if enough data is available) to determine the suitability of the calibrated model for predicting

runoff over any period outside the calibration period. If there is not enough data available, the

validation may be performed by testing shorter periods within the full record.

It is normal in research studies to split the observed data sets into calibration and validation

period prior to the study, to demonstrate the performance of the model under both sets of

conditions. Use of this approach can cause problems in practical applications if a model

demonstrates acceptable performance for the calibration data set but produces unsatisfactory

results for the validation data set. An alternative approach in this situation is to calibrate the

rainfall runoff model to all available data but to demonstrate that the performance of the model

is satisfactory over different sub-sets of the period that observed data is available.

Further discussion of model calibration and validation is provided in Section 6.

It is a very common situation in a project that involves rainfall runoff modelling for flow time

series to be required for several catchments or subcatchments within the model domain and

for data to be available from two or more stream flow gauges to facilitate calibration and


11

validation. At locations where gauged flows are available and flow estimates are required, two

options are available to the modeller:

The rainfall runoff models may be calibrated independently for each gauged catchment.

In this case, independent parameter sets will be derived for the rainfall runoff models of

each catchment; or

A joint calibration may be performed, with rainfall runoff models calibrated with

consistent parameters to fit to the gauge records from two or more gauges. In this case,

a single set of rainfall runoff model parameters will be produced for all of the catchments

that represent a compromise to fit the flows at all of the gauges within that group.

Consideration should be given at the outset of modelling to the approach that will be used for

dealing with flows from multiple catchments and subcatchments and from multiple gauges.

The strategy for dealing with this issue should be documented at this point and revised, if

necessary, during the process of calibrating and validating the models.

Calibration of a rainfall runoff model normally involves running the model may times, trialling

different values of parameters, with the aim of improving the fit of the model to the calibration

data. Calibration can be facilitated:

Manually, with the modeller setting the parameter values, running the model to inspect

the results and then repeating this process many times;

Using automated optimisation, with an optimiser algorithm running the model hundreds

or thousands of times with different parameter values; or

Using a hybrid approach of automated optimisation phases, interspersed with manually

implemented trials of parameter sets.

Defining the calibration and validation approach before commencing a modelling project can

maximise the efficiency of the calibration process, whilst avoiding the temptation to overfit

the model to noise in the observational data. A calibration strategy should therefore outline

the:

gauge locations where model calibration and validation will be performed;

viable or allowable ranges for each model parameter value;

known constraints, dependencies or relationships between parameter values (for

example, the total of the three partial area parameters in AWBM, A1, A2 and A3 must

sum to 1);

period for calibration at each gauge location;

period for validation at each gauge location;

expected level of uncertainty in observations introduced by measurement uncertainty;

metrics to be used to test calibration and validation performance;

whether manual or automated calibration strategy will be adopted, or how a hybrid

strategy of progressive manual and automated calibration will be implemented.

If an automated or hybrid optimisation strategy is to be used, further details should be

defined at the outset of the calibration process on:

algorithms to be used for optimisation of parameter values;

objective function(s) that will be used to test the calibration performance;


12

weightings that may be applied in computation of objective functions, to encourage

fitting to different parts of the flow regime (typically the relative weightings to high,

medium and low flows); and

the set of model parameters that will be optimised during calibration and constraints on

the allowable range of values for each parameter.

Ideally, calibration strategy should be documented prior to the commencement of the

calibration process. It may be appropriate for the calibration strategy to be reviewed during the

calibration.

Sensitivity/uncertainty analysis

Relevant sources of uncertainty in typical order of importance include:

1 Model input data including parameters, constants and driving data sets,

2 Model assumptions and simplifications of what the model is representing,

3 The science underlying the model,

4 Stochastic uncertainty (this is addressed under variability below),

5 Code uncertainty such as numerical approximations and undetected software bugs.

The potential impacts of the above sources of uncertainty on the decisions that will be made

using the model should be considered early in the modelling process and then re-examined

once the model has been calibrated, validated and applied for scenario runs. Uncertainty

becomes more important for estimation of events in the tails of the probability distribution,

floods and droughts, than it is for consideration of events that are closer to the centre of the

probability distribution (such as estimation of the mean annual runoff from a catchment).

Documentation and Provenance

Documentation is an important requirement for model acceptance. Its role is:

1 To keep a record of what was done so that it can be reviewed and reproduced,

2 To provide source or background material for further work and research,

3 To effectively communicate the results from models, and

4 To effectively communicate the assumptions made during the modelling process and

the decisions made by the modeller during implementation of the model.

Good documentation supports a study and it will also enable someone coming along later to

see what decisions were made, what was done to underpin the decisions and why, particularly

if aspects of the project are revisited at some later time.

Provenance, as it might relate to hydrological modelling studies simply means the ability to

trace the source/lineage of the data, model and modelling results. Reasons for providing

provenance in rainfall runoff modelling include:

1 Accountability and a full audit trail for all modelled results.

2 Repeatability; ability to re-create a results data set using current data or better

understanding.


13

3 Reproducibility; ability to re-create a results data set exactly using all original data,

workflow ordering, assumptions and parameters.

4 Versioning of both entire workflow and systems implementation. Versioning of the

subcomponents and data sets will be the responsibility of those who govern them but

must be captured by the system.

The degree of provenance required depends on the application and/or how the modelling

system is intended to be used by the individual or organisation in future. Current best practise

provenance is to save all input data and model/parameters version and workflow history such

that the outputs can be reproduced in future if required. In the future the ability to register and

resolve the type and identity of objects within the modelling process should reduce the

requirement to capture and archive these objects, especially as modellers take greater

advantage of services based point of truth data streams and modelling systems, and rely less

on ad hoc locally managed resources.

Model acceptance and accreditation

The aim of model acceptance is to gain agreement that the model is fit for purpose.

Information available from the model accreditation process (Reporting, QA documentation,

Peer review) provides model development details and review results which will enhance

model acceptance.

Peer review plays an important part, especially with stakeholders that are external to the

organisation undertaking the model development. It is important for establishing the

credibility, reliability and robustness of results and the methodology used to obtain the results.

It is undertaken by people with specialist understanding in fields relevant to the project.

Use of accepted/accredited model

Once a calibrated model is evaluated against good quality data and has undergone thorough

review process (model acceptance and accreditation), it can be used for modelling to support

water management planning and policy decisions (provided that the model was accredited for

similar purpose).


14

3 Model choice

3.1 Model selection

Model selection is made based on an understanding of the objectives and the system being

modelled (http://www.toolkit.net.au/Tools/Category-Model_development; CRCCH 2005a, b).

The WMO (2008, 2009) report include the following factors and criteria as being relevant when

selecting a model:

1 The general modelling objective; e.g. hydrological forecasting, assessing human

influences on the natural hydrological regime or climate change impact assessment.

2 The type of system to be modelled; e.g. small catchment, river reach, reservoir or large

river basin.

3 The hydrological element(s) to be modelled; e.g. floods, daily average discharges,

monthly average discharges, water quality, amongst others.

4 The climatic and physiographic characteristics of the system to be modelled.

5 Data availability with regard to type, length and quality of data versus data requirements

for model calibration and operation.

6 Model simplicity, as far as hydrological complexity and ease of application are

concerned.

7 The possible transposition of model parameter values from smaller sub catchments of

the overall catchment or from neighbouring catchments.

8 The ability of the model to be updated conveniently on the basis of current

hydrometeorological conditions.

Other things that should be considered are:

1 The level of modelling expertise available.

2 Whether the model is going to be used on its own, or if it is going to be used in

conjunction with other models.

3 Freedom of choice may be limited by a desire to minimise problems of different models

for much the same purpose in the same project area, or to avoid problems of different

models in adjoining project areas, particularly where the models are linked in some way

in the future or results compared in some way.

4 Whether uncertainty will be explicitly modelled. If uncertainty is to be explicitly included,

what types of uncertainty are to be modelled (e.g. climatic uncertainty, uncertainty in

climate change projections, uncertainty in rainfall runoff model parameter values); what

approaches will be used to generate the replicates to represent uncertainty and how

many replicates will be required to adequately quantify uncertainty.

5 Whether simulation or optimisation, or a combination of both, is adopted.


15

6 Whether the model is to be used for hindcasting or forecasting when being applied in

predictive mode.

In essence the governing principle in choosing a model should be that it should not have more

parameters requiring calibration or a greater level of detail than the available data can support,

to minimise problems of spurious results and false calibrations.

3.2 Available models

Rainfall runoff models can be represented by a range of approaches, in order of increasing

complexity as:

simple empirical methods (e.g., curve number and regression equations);

large scale energy-water balance equations (e.g., Budyko curve);

conceptual rainfall-runoff models (e.g. SIMHYD, Sacramento, AWBM)

landscape daily hydrological models (e.g., VIC, WaterDyn);

fully distributed physically based hydrological models which explicitly model hillslope

and catchment processes (e.g., SHE, TOPOG).

These categories have been used for ease of description, and there is overlap between these

model types. Although these approaches vary in terms of the complexity with which they

represent the rainfall-runoff transformation processes, all of them conceptualise the real

processes using some sets of mathematical equations (and hence are all conceptual models

of the physical environment). Similarly, conceptual rainfall-runoff models run in distributed

mode can be classed as being landscape daily hydrological models. This section provides a

discussion of the characteristics of each of these model types, along with a broad assessment

of the strengths and weaknesses of each approach for rainfall runoff modelling (Table 3-1).

Empirical methods

Empirical methods to rainfall runoff modelling typically involve the fitting and application of

simple equation(s) that relate drivers of runoff response to flow at the catchment outlet.

Empirical equations are most often derived using regression relationships.

Common predictor variables may include rainfall for the catchment, flow observed at another

gauge in the vicinity, evapotranspiration, groundwater levels, vegetation cover and the

impervious area within the catchment. Where rainfall is used as a predictor variable,

regression relationships derived almost always include a non-linear relationship between

rainfall and runoff.

All catchments incorporate storage elements, including interception by vegetation, storage

within the soil column, groundwater storage and storage within stream channels. Catchment

storage typically results in runoff from the catchment being an integrated function of the

climatic conditions for the catchment over some period prior to the period for which runoff is to

be calculated by the model. Therefore, empirical models that produce acceptably accurate

simulations of runoff are either applied at sufficiently long time steps that changes in internal

water storage within the catchment can be ignored (e.g. annual time step) or applied to

represent an integration of the climatic conditions that occurred over some time period prior.

As a practical example, for most catchments a regression model that only includes daily

rainfall on the current day is likely to produce a very poor estimation of daily runoff but a model


16

for predicting daily runoff that used individual values of daily rainfall for several days prior may

produce acceptable runoff estimates.

Empirical regression relationships are often developed using spreadsheets. They can also be

fitted using more sophisticated statistical analysis packages, which may more easily facilitate

the investigation of predictor variables. For general information on the development of

regression relationships, the modeller is referred to NIST/SEMATECH e-Handbook of

Statistical Methods (NIST and SEMATECH, 2010) or to a University Level statistics text book.

Empirical regression equations are best suited to situations where there are two flow gauges

on the same stream with partially overlapping periods of record, which are therefore subject to

similar climatic drivers, and the regression equation is used to extend the simulated flow to the

combined period of record from both sites. They can also produce adequate simulations for

neighbouring gauged catchments with overlapping periods of record in situations where the

two catchments are subject to similar rainfall timeseries and are relatively similar

hydrologically.

Large scale energy-water balance equations

The large scale energy-water balance methods are based on the hypothesis of available

energy and water governing large scale water balance (precipitation, evaporation and runoff).

These are usually developed using large scale observed data sets, eg. the Budyko curve

(Budyko, 1958) was developed using mainly European data, and numerous other forms have

been proposed to improve estimates in local regions and to account for different land cover

types (Arora, 2002). One of the more popular forms of the Budyko method is the Fu (1981)

rational function equation (Zhang et al., 2004) where a single parameter, , in the equation can

be calibrated against local data to tune the method for the local conditions. The inputs to these

equations are rainfall and potential evapotranspiration (PET) and the output is runoff at mean

annual time step.

Conceptual Rainfall-Runoff Models

Conceptual rainfall runoff models represent the conversion of rainfall to runoff,

evapotranspiration, movement of water to and from groundwater systems and change in the

volume of water within the catchment using a series of mathematical relationships. Conceptual

rainfall runoff models almost always represent storage of water within the catchment using

several conceptual stores (or buckets) that can notionally represent water held within the soil

moisture, vegetation, groundwater or within stream channels within the catchment. Fluxes of

water between these stores and in and out of the model are controlled by mathematical

equations.

Most applications of conceptual rainfall runoff models treat the model in a spatially lumped

manner, assuming that the time series of climatic conditions (notably rainfall and potential

evapotranspiration) and the model parameter values are consistent across the catchment.

There have been implementations in more recent times of conceptual rainfall runoff models in

semi-spatially distributed and spatially distributed frameworks. In distributed application, the

catchment is defined by grid cells or subcatchments within the catchment that are assigned

the same rainfall runoff parameter values but different time series of climatic inputs so that

different grid cells or subcatchments within the catchment produce different contributions to

the overall runoff. This is effectively a series of lumped rainfall runoff models, with lumped sets

of model parameters that are applied with spatially distributed rainfall.


17

Conceptual rainfall-runoff models have been widely used in Australia for water resources

planning and operational management because they are relatively easily calibrated and they

provide good estimates of flows in gauged and ungauged catchments, provided good climate

data is available.

In Australia there are six widely used conceptual rainfall-runoff models; AWBM (Boughton

2004), IHACRES (Croke et al. 2006), Sacramento (Burnash et al. 1973), SIMHYD (Chiew et

al. 2002), SMARG (Vaze et al., 2004) and GR4J (Perrin et al. 2003). The input data into the

models are daily rainfall and PET, and the models simulate daily runoff. The models are typical

of lumped conceptual rainfall-runoff models, with interconnected storages and algorithms that

mimic the hydrological processes used to describe movement of water into and out of

storages. They vary in terms of the complexity of the catchment processes that they try to

simulate and in terms of the number of calibration parameters which vary from four to

eighteen.

Landscape daily hydrological models

These models are based on the concept of landscape processes and they model the typical

landscape processes using simplified physical equations (VIC model, Liang et al., 1994;

2CSALT, Stenson et al., 2011; AWRA-L, Van Dijk, 2010). A catchment is usually

conceptualised as a combination of landscapes which are delineated using some combination

of outputs from digital elevation model analysis, underlying geology, soil types and land use.

Often these models have been designed to reproduce other variables in addition to streamflow

(e.g. distributed evapotranspiration, soil moisture, recharge, salinity), and as a result have a

greater complexity to methods that target streamflow alone.

Fully distributed physically based hydrological models which

explicitly model hillslope and catchment processes

The physically based models are based on our understanding of the physics of the

hydrological processes which control the catchment response and use physically based

equations to describe these processes. A discretisation of spatial and temporal coordinates is

made at a very fine scale for the entire catchment and the physical equations are solved for

each discretised grid to obtain a solution.


18

Table 31 Assessment of Strengths and Weaknesses of Different Rainfall Runoff Model Structures

Criteria Model Type

Empirical Large Scale

Energy-Water

Balance

Conceptual Landscape

Daily

Fully

Distributed

Physically

Based

Typical Run Time Step

Can be daily if daily flow from another gauge is used as a predictor variable. Otherwise typically only applied at annual (or longer) time scale

Typically only applied for mean annual runoff, although pattern of flows from a nearby gauge may be used to disaggregate annual totals to monthly or daily time steps

Daily, although shorter run time steps are possible if sufficient climatic data is available at this shorter time step

Daily, although shorter run time steps are possible if sufficient climatic data is available at this shorter time step

Minutes to hours to maintain numerical stability, although often forced with daily data and assumed patterns used to disaggregate to shorter time steps

Typical Number of Parameters

1 to 5 2 to 4 4 to 20 10 to 100 10 to 1000's

Risk of over-fitting or over-parameterising the model.

Low Very Low Moderate High Very High

Need for high resolution spatial data layers

None to Moderate

Low to Moderate

Low High Very High

Strength of Apparent Connection between Model Parameters and Measurable Physical Catchment Characteristics

None None Weak for most parameters (although impervious area or interception may be exceptions)

Moderately weak

Claimed to be strong by proponents but can be difficult to validate this claim

Run time on typically available computer platforms for 100 years of daily data


19

Criteria Model Type

Ability to implement multiple runs for automated calibration

Not typically required - optimum parameters can be obtained by least squares fitting that does not require multiple runs

Not typically required

Very Good. Run times are typically sufficiently low to facilitate this and tools are available (Rainfall Runoff Library and Source) to facilitate this

Good. Run times likely to be sufficiently low to facilitate this in most circumstances, however tools for calibrating such models using automated routines are not as widely available

Poor. Run times are generally too long to consider automated calibration

Typical Performance in Regionalisation

Moderate at annual time steps. Usually very poor at shorter time steps (e.g. Daily)

Good at annual time steps. Usually very poor at shorter time steps (e.g. Daily)

Moderate at daily time steps

Proponents claim to be superior for regionalisation to conceptual rainfall runoff models

Proponents claim this to be a strength of distributed models but in reality the large number of parameters required may compromise the application of distributed models to regionalisation

Representation of non-stationarity in catchment conditions

Not possible

Often applied to explicitly represent non-stationarity in vegetation cover for mean annual runoff signal

Usually difficult, due to lack of physical meaning for many model parameters

Possible Possible

Typical performance of model when applying to a very different climatic period to that used for calibration

Poor Moderate when used to estimate impact on mean annual flow but flows disaggregated to shorter time steps are likely to be poorly estimated

Variable - can be good in some catchments but poor in others



Typical level of expertise with this type of model within Australian water industry

Strong Moderate Strong Weak Very weak

Likelihood that previously calibrated models are available for catchment to be modelled.

Moderate to Low

Moderate Very High Low Very low


20

4 Collate and Review Data Climatic data is the most important driver of any rainfall runoff modelling process. The

calibration and validation of models also involves comparison to observed streamflow data.

Major causes of difficulty in calibrating rainfall-runoff models are errors and uncertainties in the

input data (see Kavetski et al, 2003). A discussion of these problems can be found in the

collection of papers in Duan et al 2003. Checks should therefore be performed on the input

data and the comparison data set for calibration and validation to be used in rainfall runoff

modelling before any attempt is made to apply or calibrate the models. The intent here is to

investigate the integrity of the data, whether observations are in the first instance plausible

(e.g., is the volume in a hydrograph less than the product of the rainfall and catchment area).

Investigations into data to be used for rainfall runoff modelling should include checks of:

Stationarity of the data time series , i.e. has there been any systematic or step change in

the statistical properties over the time of data collection, and if so why;

Spatial coherence of data, i.e., is the data consistent with regional spatial and temporal

patterns and trends;

Accuracy of the spatial location of the gauging site;

Consistency in the approach used to date and time stamp the data, particularly for data

provided by different agencies;

Procedures use for spatially interpolation of point observations to gridded data

estimates or estimated series across catchment areas

e.g., time series plots at different levels of temporal aggregation, ranked plots, residual mass

curves, double mass curves, etc. This will pick up patterns as well as identify anomalies which

may be potential data QA issues.

Other checks and analysis, including regional consistency of runoff depths, rain-runoff ratios,

rating confidence limits, period of record, whether rainfall and PET is observed or interpolated,

base-flow characteristics, checks for stationarity and variability over time, etc would also be

useful. It is important that prior knowledge is considered.

One major factor which will apply across all types of time series data used is that the time base

must be kept consistent so that the data applies to the same time period. An example is

where flow data time steps should be matched to the rainfall data time step. In Australia, daily

rainfall data is commonly recorded as the depth of rainfall that occurred in the 24 hours

preceding 9 am on the date of the recorded data. In contrast, daily streamflow totals are often

quoted for the 24 hour period commencing on the nominated date, resulting in the recorded

flow data being offset by 1 day forward of the rainfall data. Where possible the flow data should

be extracted at a time step to match the rainfall dataset. HYDSTRA databases allow this where

the records are at short time intervals. In other cases shifting the recorded time series by one

day for either the rainfall or flow time series may be required to produce consistent time series

for modelling.

The remainder of Section 4 outlines the data types, sources, availability, accuracy,

manipulations (such as gap filling) and any other issues.


21

4.1 Catchment details

Location of gauges (streamflow, rainfall and evaporation)

The streamflow recorded at the catchment outlet is a combined response to the spatial

distribution of rainfall and evaporation across the catchment. There are uncertainties

associated with the streamflow measurements due to rating curve errors as well as due to

extrapolation outside the limits of the rating curve. There is spatial variability in rainfall (and to

smaller extent evaporation) across a catchment which is not captured when undertaking

lumped catchment modelling using a single rain gauge. There might be problems with the

location of the rain gauge in terms of capturing a representative rainfall for all the rainfall

events especially for catchments with high rainfall gradients.

Topography and Catchment Areas

The catchment area for a catchment represents the contributing area to the catchment outlet

where the streamflow is measured. The catchment boundaries (and the corresponding

catchment area) can either be derived from topographic map layers or using the catchment

digital elevation model (DEM) and a standard package such as ARCGIS. It is usually easier to

determine catchment area for the catchments located in steeper terrain compared to those

located in very flat areas (especially when using DEM).

Slope and dominant aspect may provide useful explanatory variables for estimating routing

parameters or for regionalisation of rainfall runoff parameters between catchments.

Soil types

A catchments rainfall-runoff response is related to the soil types in the catchment. The surface

soil characteristics determine the infiltration rates and so the contributions from different flow

components (surface runoff, throughflow and base flow). Soils information can be obtained

from any soils field work that has been undertaken in the catchment or from large scale soil

properties maps (e.g. Australian Soils Atlas, Northcote et al., 1960). In most practical

applications of conceptual rainfall-runoff models in Australia, soil information is seldom directly

used as input in the calibration process because the inherent spatial variability in soil

properties within a catchment is typically sufficiently large that it has been difficult to

demonstrate statistically robust relationships between conceptual model parameters and soil

properties.

Vegetation

Land cover/vegetation cover in a catchment can often be correlated with the amount of

interception storage/loss and actual evapotranspiration in a catchment. The land cover across

the catchment can be derived from large scale vegetation mapping based on satellite imagery

or remotely sensed data. Vegetation cover data has not typically been used explicitly in

directly determining rainfall runoff model parameters, although there have been some recent

studies which have demonstrated the importance of catchment land cover in rainfall-runoff

model calibration and for predictions in ungauged basins (Zhang and Chiew, 2008; Vaze et al.,

2011c).


22

Water Management Infrastructure

Water management infrastructure within a catchment can allow humans to make very

substantial modifications to flows within a catchment. Water management infrastructure may

include large dams, farm dams and off stream storages, extractions, man-made canals or

diversion pipelines and discharges from sewage treatment plants. Locations of these

infrastructures should be identified where they exist within the catchment so that their potential

impact on streamflows may be assessed. Recorded flows at the catchment outlet may require

adjustment to allow for the influence of water management infrastructure located upstream of

each of the flow gauging locations.

4.2 Flow data

Reliable measurements of streamflow data are critical for successfully calibrating a

rainfall-runoff model to a catchment. The streamflow data for all the gauged locations can be

obtained from the respective state government water management agencies or from the

Bureau of Meteorology (in Australia). Considerations in checking streamflow data include:

the agency collecting the data and the quality assurance procedures (if any)

implemented by that organisation during data collection;

reliability of the rating of levels to flows for the gauge;

the accuracy, extent and currency of cross sections surveyed at the gauge site.

(Surveyed cross sections may only extent to the top of the stream bank and gauging for

flows extending onto the floodplain may use a cross section that is inaccurate);

vegetation and substrate material for the channel bed, channel banks and floodplain

and the influence of assumptions made about these on gauged flows;

the ratio of the highest flow outputs to the highest flow that the gauge has been rated for;

how hydraulically stable (variable over time) the rating site is;

examination of potential backwater effects for the site from influences that are

downstream of the site, such as stream confluences, bridge crossings, culverts, dams

or weirs;

hysteresis effects leading to different flow rates for the same recorded level on rising

and falling limbs of hydrographs;

how well maintained the gauging site and instrumentation used for measuring water

levels has been;

any changes to the gauging instrumentation over time;

the length of time since the last rating at high flows;

length of record at the site;

availability of quality codes with the flow data record;

proportion of missing data;

trends in when data is missing from the record (i.e. Is there any bias toward high or low

flow periods, particular seasons, or are the gaps just random?) and how this might

influence any infilling procedures; and


23

if there are a number of gauges closely located that basically represent the same

catchment the data sets may be able to be combined to give a longer record for the site.

Assessment of the above factors will inform whether the data is useful in calibration of the

model, independent validation of the model or whether the data should be ignored.

4.3 Rainfall

Rainfall is the main driver of runoff and so reliable measurements of rainfall are critical for

successfully calibrating a rainfall-runoff model to a catchment. There are several sources for

obtaining climatic data for a particular catchment:

Site observations from Bureau of Meteorology climate database.

Site observations taken from monitoring sites collected by other organisations that may

exist outside of the Bureau of Meteorology database. Many jurisdictional databases

contain rainfall records.

Gridded data products, such as the Bureau of Meteorologys Australian Water

Availability Project (AWAP) or Queensland Centre for Climate Applications SILO data

set.

It is important to be aware of how this data has been collected and what data quality control

methods have already been applied to the data prior provision of the data set as this may

influence the modelling results. This is particularly relevant to gridded products, such as SILO

and AWAP (SILO, Jeffrey et al., 2001; AWAP, Jones et al., 2009), as these data sources

generally use different algorithms to convert time-series observations at data points to gridded

data products.

In a small catchment, considerably better results may be obtained from using rainfall station

data from the BOM (http://www.bom.gov.au/climate/) or locally collected data than a gridded

data set that smoothes observations from a smaller number of more sparsely located sites. In

some cases it may be appropriate to adjust the station data, normally by a percentage, if the

mean catchment rainfall can be defined using other sources e.g. isohyetal detail.

In large catchments there is spatial variability in rainfall across a catchment which is not

captured when undertaking rainfall-runoff modelling using rainfall time series from the rain

gauges. If using a single rain gauge, there might be problems with the location of the rain

gauge in terms of capturing a representative rainfall for all the rainfall events. If using a

spatial rainfall product (SILO or AWAP in Australia), there can be uncertainties introduced

because of the method used for interpolating rainfall between rain gauges and changes in the

rain gauge network over time. Interpolation methods currently used are more suited to areas

where rainfall varies less over space and in time. They do not account well for orographic

effects, and rainfall networks in Australia historically have not captured the spatial and

temporal variations in tropical and monsoonal areas well.

Vaze et al., 2010b discusses testing carried out considering the effects of using different

rainfall data sets on the calibration and simulation of conceptual rainfall-runoff models. They

conclude that considerable improvements can be made in the modelled daily runoff and mean

annual runoff with better spatial representation of rainfall. Where a single lumped

catchment-average daily rainfall series is used, care should be taken to obtain a rainfall series

that best represents the spatial rainfall distribution across the catchment. However where only

estimates of runoff at the catchment outlet are required, there is little advantage in driving a


24

rainfall-runoff model with different rainfall inputs from different parts of the catchment

compared to using a single lumped rainfall series for the catchment.

4.4 Evapotranspiration

The measured pan evaporation data can be obtained for all the locations with the evaporation

gauges installed (in Australia from the Bureau of Meteorology (BoM) basic records). In

Australia there are also some spatial climate products which use point evaporation

measurements recorded by the BoM and use an interpolation schemes to produce spatial

evaporation surfaces (SILO, Jeffrey et al., 2001; AWAP, Jones et al., 2009).

The network of pan evaporation recording stations in Australia is sparse in comparison to

stream flow and rainfall networks, although there is some compensation in that typically

potential evapotranspiration exhibits substantially higher spatial correlation than rainfall or

stream flow. This limits the ability to accurately represent the true spatial and temporal

variability in evaporation in models however the spatial variability in evaporation is much

smaller compared to the variability in rainfall.

The BoM network records pan evaporation. Modelling requires potential evapotranspiration

(PET). There are a number of methods to convert pan evaporation to PET including Penman

Monteith, Mortons and accepted pan factors. These use climatic variables in the conversion

calculation including solar radiation, temperature, vapour pressure, and wind speed which are

recorded at some pan recording stations but not all. This further limits the network available to

draw data from.

When all the required data is available the conversion calculations will use the records but

often some variable is missing and estimates of that variable are made and used. Where

there is no data for the climatic variables, calculated pan to PET conversion factors from a

nearby station can be used to derive PET from pan evaporation.

Commonly the spatial products have interpolated layers for a range of climatic factors and the

spatial PET layer is calculated from data in these layers rather than interpolating PET

calculated at recording stations.


25

5 Statistical Metrics for Testing

Performance There are many performance measures used to consider the acceptability of a rainfall-runoff

model. In all cases visual assessment and statistical results of some sort will be assessed to

identify the ability of the model to reproduce the flows it is calibrated or validated against. All

may contribute to best practice and which measures are more appropriate will be directed by

the modelling objective. A number of commonly used visual assessment techniques are

outlined in Table 51. Statistical performance measures and their relevance in various study

types are listed below in Table 52.

Table 51 Plots for assessing model performance

Plot Assessment and Purpose

Daily and monthly plots (linear and log)

Used to check the general size, shape and timing of hydrographs. Linear plots will better show medium and high flows and log plots low flows. Baseflow and recession characteristics can be reviewed. If recessions are frequently too flat then this can indicate that the interflow and baseflow are not represented correctly.

Scatter Plot

Scatter plots show the ability of the model to match flows on actual time steps. They show the flow ranges where the model is more accurate. Linear and log plots will show the variability across the various flow ranges. Often a line of best fit is shown to indicate the bias of the model in estimating flows.

Ranked Plots Commonly referred to as frequency of excedence or flow duration graphs, they show the percentage of time a flow is exceeded over the modelled period. They show whether the modelled output can replicate the observed flow regime. Flow duration curves are effective diagnostics to ensure that both the variability and the seasonal pattern are captured.

Cumulative mass or cumulative residual mass curves

Scatter plots and flow duration curves do not examine the time sequence of events. A model could appear to be replicating the flow regime however the replication of regimes during wet and dry periods may not be adequate. A cumulative residual mass curve is a cumulative plot of residuals (flow value - mean of all values). A residual, and therefore slope of the curve, will be positive during wet periods as flows are higher than average and during dry periods the slope will be negative. If the curves diverge there may be a data issue. If they diverge consistently in all wet or all dry periods it is likely that model parameterisation for wet periods or dry periods may not be appropriate.

Plotting average daily or monthly flows (average of all Days, average of all Januaries)

A simple diagnostic to ensuring that the model can replicate seasonality characteristics.


26

Table 52 Statistical performance measures (metrics) and their relevance in various study types (Y Yes, N No)

Metric Purpose

Runoff Yield

Climate change

Landuse change

Low flow

Water quality

Peak flow / floods

Difference in total runoff Y Y Y N N Y

Difference in total runoff over different seasons of the year*

Y Y Y Y Y Y

Difference in total runoff contained within high, medium and low parts of the flow duration curve

Y Y Y Y Y Y (high flows)

Difference in proportion of time that cease to flow occurs

N Y Y Y Y N

Difference in the slope of logarithm of flow versus time for baseflow recession periods

N N Y Y Y N

Mean square error between observed and modelled runoff

Y Y Y N N Y

Coefficient of determination (often referred to as r)

Y Y Y N N Y

Nash Sutcliffe coefficient of efficiency on daily flows

Y Y Y N N Y

Nash Sutcliffe coefficient of efficiency on monthly accumulated flows

Y Y Y N N N

Nash Sutcliffe coefficient of efficiency calculated using logarithm transformed flows

N Y Y Y Y N

* Definition of seasons to be used will vary depending upon the climatic zone that the catchment is in. For

tropical areas, two seasons (a wet season from December-April and dry season from May-November) may

be appropriate. In Southern Australia, it may be appropriate to consider the four conventional calendar

seasons (Dec-Feb, Mar-May, Jun-Aug and Sep-Nov).

** Definitions of high, medium and low flow ranges will depend upon the purpose of the study and the

catchment. Typical ranges might be High flows: days in observed data in the 0 to 20% probability of

exceedance range; Medium flows: days in observed data in the 20 to 80% probability of exceedance range;

Low flows: days in observed data with greater than 80% probability of exceedance and above the cease to

flow level at the gauge. Adjustment of the low and medium flow ranges may be required particularly in

response to the probability of cease to flow conditions at the gauge.


27

6 Calibration and validation

6.1 Calibration

Model calibration is a process of optimising or systematically adjusting model parameter

values to get a set of parameters which provides the best estimate of the observed streamflow.

Virtually all rainfall runoff models must be calibrated to produce reliable estimates of

streamflow because there has been little evidence identified of strong links between physical

characteristics of catchments and the parameters of rainfall runoff models (Beven, 1989).

Models should always be calibrated to observed data to demonstrate that the model can

produce observed flow time series with an acceptable level of accuracy. The acceptable level

of accuracy will depend upon the statistics of the flow data to be reproduced, which is

determined by the purpose that the model will be applied for.

A model may be available that has been previously calibrated for a catchment as part of

another study. In this situation, the calibration performance of the model should be re-tested

before it is applied because the purpose for developing the model may be different between

the earlier and later applications, which may influence the calibration objectives.

When calibrating a model it should always be kept in mind that there are always going to be

tradeoffs, for example between getting wet, dry, and average conditions correct, and those

tradeoffs will be driven by the purposes the model will be used for.

6.2 Validation

Model validation is a process of using the calibrated model parameters to simulate runoff over

an independent period outside the calibration period (if enough data is available) to determine

the suitability of the calibrated model for predicting runoff over any period outside the

calibration period. If there is not enough data available, the validation may be performed by

testing shorter periods within the full record.

Model validation is one of the most important steps in rainfall-runoff modelling as the

performance of the calibrated model in the validation period provides us confidence in the

modelling results when the calibrated model is used for simulating streamflow outside the

measured streamflow period or when the model is used for predicting streamflow under future

climate change scenarios.

Validation has often been achieved using a split sample process, whereby a period of

observed data (say the first two-thirds of the available record) are used for calibration and the

remaining one-third are used for validation. The model that was calibrated using the calibration

data set is run for the validation period without changing the model parameters and the

goodness of fit statistics are computed for the validation period. The split sample approach

assumes that both the catchment and the climatic conditions that it is subject to are stationary

in nature across the entire period that recorded data is available for. Evidence of stationarity

(or non-stationarity) in catchment conditions that would affect the hydrological response during

the period of recorded data should be checked using independent data sources (such as aerial

photography, satellite imagery, landuse, topographic or other spatial information).


28

A more sophisticated calibration approach can involve multiple calibration and validation

periods. As in the simple split sample approach, the model is calibrated to a calibration period

and then performance is tested over the validation period without changing the model

parameter values. This approach is then repeated multiple times, with each replicate using

different start and end dates for the respective calibration and validation periods. This allows a

range of model performance statistics for calibration and validation periods to be reported.

There will be some instances with this calibration and validation approach where the calibrated

parameters perform well against the calibration data set, but performs poorly against the

validation data set. In research type investigations, where the modeller may be comparing

different rainfall-runoff models, calibration methods, or objective functions, the validation

results can be used directly to help decide the best model or method or objective function.

However, in practical applications, a modeller may have to decide either not to change the

calibrated parameters and report the poor results, or to recalibrate the model because the

performance is unacceptable.

The modeller may choose the latter option, and may then recalibrate and compare against the

validation data set several times until the calibrated parameters perform acceptably against

both data sets. However, as the validation data set has now been used to change the

calibrated parameters, it is no longer an independent data set and has in effect indirectly

become part of the calibration data set.

This risk of having much poorer performance in validation than calibration may be mitigated by

ensuring as far as possible both data sets have similar flow distributions, An arbitrary

approach to splitting the data, e.g., at the midpoint, may result in half of the data being in a

much wetter period. A model calibrated to these conditions would not be expected to perform

well under the drier conditions in the validation data set. More alternate approaches should be

considered on how to split the data set, perhaps into non-contiguous periods, to ensure overall

flow distributions are similar in each period.

Data is a valuable resource, and should be used to greatest effect. In most Australian

conditions, long data sets are needed to adequately represent climatic variability. An

alternative approach to having split samples is to use the complete data set to calibrate the

model, then to report its performance for different sub-periods, e.g., first half and last half, or

decadally, or driest X year period and wettest X year period. The objective would be to have a

comparatively persistent performance across all these periods. This does not necessarily give

you an independent assessment of performance, but does report on performance under

different conditions.

Transposition of model parameter values from gauged to ungauged catchments may be

tested using a spatial variant on split sample validation. Under this approach, component

models from a gauged catchment with the calibrated parameter values for that catchment can

be applied to another gauged catchment to test the uncertainty and bias introduced from

transposition. Uncertainty ranges can be established by testing contributions flow series

produced by model outputs with parameter sets adopted from several different gauged

catchments. Examples of the performance of these transposition approaches are discussed in

Viney et al. (2009) and Chiew (2010).

Generally the same metrics used to assess the performance of the model during calibration

are also used to assess model performance during validation. The model performance during

validation is almost always poorer than during calibration because model parameters are

deliberately not specifically fitted to the data for the validation period.


29

6.3 Calibration and Validation of Models to Single Gauge Sites,

Multiple Gauge Sites and Regionalisation of Model

Parameter Sets

It is a very common situation in a project that involves rainfall runoff modelling for flow time

series to be required for several catchments or subcatchments within the model domain and

for data to be available from two or more stream flow gauges to facilitate calibration and

validation. At locations where gauged flows are available and flow estimates are required, two

options are available to the modeller:

The rainfall runoff models may be calibrated independently for each gauged catchment.

In this case, independent parameter sets will be derived for the rainfall runoff models of

each catchment; or

A joint calibration may be performed, with rainfall runoff models calibrated with

consistent parameters to fit to the gauge records from two or more gauges. In this case,

a single set of rainfall runoff model parameters will be produced for all of the catchments

that represents a compromise to fit the flows at all of the gauges within that group.

The advantage of the joint calibration approach is that, assuming some degree of

homogeneity in the rainfall runoff response of the selected gauged catchments, the parameter

sets produced should be more robust when applied to other catchments with similar response

that were not used for the calibration.

If an automated calibration process is used for joint calibration of multiple catchments, the

objective function used for automated calibration to the gauged catchments will be a weighted

average of the objective function values produced at the individual gauges. Options for

selecting the weighting values are:

All gauged catchments contribute equally to the overall objective function;

Weights are assigned according to the length of available record (e.g. number of days

with data) at each site;

Weights are assigned according to the inverse of the correlation coefficient in gauged

flows between one gauge and one or more of the other gauges in the set (i.e. gauges

with strongly correlated recorded flows are assigned lower weighting factors than

gauges that have weaker correlations with other gauges);

Some combination of the above factors.

There are three main methods of developing flow data sets in residual ungauged catchments

between upstream and downstream gauges:

1 Calibrate a model to the difference in flow between the gauged upstream flows routed to

downstream (adjusted for known transmission losses) and downstream gauges.

2 Adjust a flow data set from a nearby catchment using either recorded or generated data,

3 Apply parameter values from other calibrated models and use catchment appropriate

climate data.

There are two main methods of developing flow data sets in ungauged catchments:

1 Develop a regression equation between flows for the ungauged catchment and gauged

catchments and apply this equation to transpose the flow, or


30

2 Apply parameter values from other calibrated models and use catchment appropriate

climate data.

Generally in the second case parameters for a neighbouring or nearby catchment are used but

climate data and catchment characterises of the catchment of interest are applied in the

model. Many studies have shown that selecting a donor catchment based on spatial proximity

gives similar or better results than selecting a donor catchment based on catchment attributes

(Merz and Bloschl 2004, Oudin et al 2008; Parajka et al. 2005; Zhang and Chiew 2009).

6.4 Automated, Manual and Hybrid Calibration Strategies

Calibration of hydrological models can be conducted using manual or automated methods, or

a combination of the two approaches (see Boyle et al, 2000 and Brdossy, 2007 for

frameworks for combining manual and automated methods of model calibration). Calibration

involves the adjustment of model parameter values to improve the fit of model output data to

observations to a level that is acceptable.

In case of manual calibration, definition of goodness of fit is usually produced as a

combination of statistical indices and visual inspection of the observed and simulated

hydrographs. Whereas in case of automated calibration, definition of goodness of fit is

usually produced using an objective function. The objective function translates the observed

and modelled outputs into a single number, so that the results of successive calibration

iterations can be compared. Automated calibration routines use a defined algorithm that runs

the model multiple times, adjusting model parameter values according to a strategy that is

intended to improve the value of the objective function.

The sections that follow give information on the calibration methods available and their

relevance in various study types (shown in Table 3-1) which dealt with model choice

appropriate for intended purposes.

Manual Calibration

Manual calibration involves the modeller selecting a set of parameters for their model, running

the model once and then examining the output statistics from the model (from the list

discussed in Section 5). The modeller would then revise the values of one or more parameters

and repeat the above process. This may continue many times until the model achieves the

desired level of performance.

The match between simulated and observed streamflow can be visually assessed as either a

time series, or as flow duration curves or residual mass curves. The visual assessment can

identify general deficiencies in the matching of the hydrologic regime, e.g., high flow events

under or over estimated, baseflows under or over-estimated or the seasonal response of the

model not captured appropriately. Software that stores the results of conceptual storages and

fluxes for graphing, and interpretation of these results in the context of model structure is also

useful to identify which parameter values need adjusting and in which direction in order to

improve results.

Guidelines are available from the developers of the Sacramento model that describe how to

estimate key parameter values directly from analysis of recorded hydrographs (Burnash,

1995). A range of realistic parameter values has also been recommended to guide initial

estimates.


31

Strengths:

1 Encourages a deeper understanding of model structure and its applicability to

catchment hydrology, rather than treating as a black box.

2 Allows for hydrologist to consider performance against a broad range of performance

metrics, and make appropriate adjustments.

3 Takes into account understanding of the data and the catchment.

4 Allows a logical checking at each change.

5 Produces a greater appreciation of strengths and limitations of calibrated result.

Weaknesses:

1 Repeatability is limited. Different people may get different parameters and output flow

time series.

2 More effort and time required to complete a calibration.

3 Difficult to m

ewater modelling guidelines rrm (v1 mar 2012)

Documents

use of information

acknowledgments ewater

legal information

document vaze

definition of best practice

department of innovation

innovation centre

national water commission