Modeling the Catchment Via Mixtures: an Uncertainty Framework for Dynamic Hydrologic SystemsDynamic Hydrologic SystemsLucy MarshallA i P f f W h d A l iAssistant Professor of Watershed AnalysisDepartment of Land Resources and Environmental SciencesMontana State UniversityyEmail: lmarshall@montana.edu

Thanks to: Kelsey Jencso, Tyler Smith, Brian McGlynn- MSUAshish Sharma- University of New South WalesDavid Nott- National University of Singapore

Conceptualizing first order watershed processesprocesses

Tenderfoot Cr.E F tTenderfoot Cr.E F t • 7 nested watershedsExp. ForestExp. Forest 7 nested watersheds

• Lodgepole pine vegetation• Melt driven runoff

F i t t

¯̄̄̄̄̄

• Freezing temperatures can occur in every month

• 555 ha• Full range of slope

and topographic convergence

0 500 1,000250 Meters

Well TransectFlume SNOTEL

0 500 1,000250 Meters0 500 1,000250 Meters

Well TransectWell TransectFlumeFlume SNOTELSNOTEL

0 500 1,000250 Meters

Well TransectFlume SNOTEL

0 500 1,000250 Meters0 500 1,000250 Meters

Well TransectWell TransectFlumeFlume SNOTELSNOTEL

convergence, divergence

• Elevation ranges from 1840m to 2420The Tenderfoot Creek

Experimental Forest

2420

noff

m/h

r)0.5RunoffSn

owm

elt/R

ain

mm

/day

02040

SWE

(mm

)0

300Snow MeltRainSWE

(a)100000

ST2WST5W

Ru

(mm

0.0( )

Win

ter

24 transect total -binary connectivity- III

ea (m

2 )

TFT2S

ST2W

TFT4N

area

m2

cum

ulat

ed A

re

10000TFT1NTFT5STFT3S

ST7EST1E

TFT1S

Spr

ing

umul

ated

II

Ups

lope

Ac

ST2EST6E

ST3W

ST3E

ST4ETFT2N

ST6W ST7WTFT3S

slop

e ac

c

TFT4S

TFT5NST1W

ST5E

TFT3N

St Ri i Hill l um

mer

Up

I

10/06 12/06 2/07 4/07 6/07 8/07 10/07

1000

ST4W

Stream-Riparian-HillslopeWater Table Connection No Connection S

Kelsey Jencso

Conceptualizing first order watershed processesprocesses

Unknown Process/Model Implementation

0.9

1

450

500 0

1

2

Snow melt-Temp/energy dependent?-Elevation effects?

0.5

0.6

0.7

0.8

off (

mm

)

250

300

350

400 3

4

5 Soil Moisture Accounting/sub surface flow

effects?-Rain on snow?

-Thresholds?-Seasonal?

0.2

0.3

0.4Run

o

100

150

200

observed runoffSWErainfall Storages/

surface flow Seasonal?

0.7

0.8

0.9 MS runoff mm/hrSun Runoff mm/hr

StringerSun

03/01 04/01 05/01 06/01 07/01 08/01 09/01 10/010

0.1

Date

0

50 Residence times -Slope effects?-Seasonal?

0.2

0.3

0.4

0.5

0.6

runo

ff m

m/h

r

-0.1

0

0.1

0.2

4/1/06 5/1/06 5/31/06 6/30/06 7/30/06 8/29/06

date

Conceptual rainfall-runoff modeling in hydrologyhydrology

Watershed is represented  P E

Qs

pas a variable series of storages.

Model uses rainfall S1 S2 S3 Qr

Model uses rainfall, evapotranspiration etc. time series as inputs to 

l ff

A1 A2 A3

simulate runoff

Conceptual distributed models: discretize

BS Qb

models: discretizecatchment into individual units, or use hydrologic response units BS Qbresponse units

An uncertainty framework – ways of i ti h d l i l it incorporating hydrologic complexity

St d d M lti l Hi hi lStandard Bayesian

Model

Multiple Sources of Data

Ensemble Model

Hierarchical Model

Data

y | θ x

y | θ1, x

y1 | θ1, x y | yA, yBy2 | θ2, x

Processy | θ, x1

yA | θ, x1θ1, θ2, x yB | θ, x1Parameters

y ~ variable of interest Adapted after Clark,

θ1| θ2, x

x ~ input data, climatological variablesθ ~ parameters

Adapted after Clark, Ecology Letters, 2005.

Base conceptual modelsp• Two base structures: a simplebucket model and theprobability distributed model(PDM Moore 1985)(PDM,Moore, 1985).

• Three semi‐distribution sub‐structures: based on aspect,elevation and theirelevation and theircombination to account forspatial variability in inputs.

• Three snowmelt accounting routines: temperature index,radiation index and the combination.

Difficulties in characterizing hydrologic model uncertaintymodel uncertainty

• Hydrological models: often have highly correlated and 

interdependent parameters

Histogram for S1 in AWBM

interdependent parameters

600

700

800

900Histogram for S1 in AWBM

300

400

500 • A solution is provided by an adaptive MCMC algorithm using

th hi t f th l d

128 129 130 131 132 133 134 135 1360

100

200 the history of the sampled 

parameter states

Inference results

B k d l d b d b

(a)

Bucket model semi-distributed by aspect and accounting for snowmelt using the temperature- and radiation index approachradiation-index approach

Assessing model uncertainty via Bayesian Model AveragingBayesian Model Averaging

• Probabilistically weight each model

E bl f d l∑

• Ensemble of models is an increasingly accepted way of representing modelrepresenting model ‘structural’ uncertainty

• The Bayesian

Model 1 Model 2

θθθ dMpMyfMym )|(),|()|( 111 ∫= approach accounts for multiple sources of uncertainty

)|()()|( 111 MymMPyMP •∝

The utility of multi model ensembles

• Models represent competing ‘hypotheses’ about the first order processes• Both models provide information on the processes occurring so that the data is better captured0.45

0.15

0.2

0.25

0.3

0.35

0.490% Conf idence

ObservedSimple Average of Two Models

0.4

0.45

90% Confidence

0

0.05

0.1

1 501 1001 1501 2001 2501 3001

0.450.2

0.25

0.3

0.3590% ConfidenceObserved

0 15

0.2

0.25

0.3

0.35

0.490% ConfidenceObserved

0

0.05

0.1

0.15

1 501 1001 1501 2001 2501 3001

0

0.05

0.1

0.15

1 501 1001 1501 2001 2501 3001

Hierarchical Mixtures of ExpertsHierarchical Mixtures of Experts

E h t l d l b q

Logistic gating

Each conceptual model can be cast as:

)();( 2,, itiittit xfQ σεθ +=

q1

q2

Model 1 Model 2

function

The probability of selecting individual models is based on the gating function, using catchment

xx

A single-level two-component Hierarchical Mixture of Experts model

.

predictors Xt:

),X(G

),X(G

,t t

t

eeg β

β

+=

11 Mixture of Experts model

Models are sampled via a conditional simulation of independent Bernoulli

)z|Q(P),,|z(p),,,Q|z(p i

,tt,tt,t

∑=

=σθβ==σθβ= 2

2

12

121

111

e+1

independent Bernoulli random variables zt, with probability specified as:

)z|Q(P),,|z(pi

i,tti,t∑=

=σθβ=1

2 11

Mixture models- alternative models suitable at diff idifferent times

• Probabilistically split

2.0E-03

2.5E-03 • Probabilistically split the data according to some catchment indicatorsDifferent

1.5E-03

indicators

• Fit separate models to the data and data errors Models may

models selected for parts of the

data

5.0E-04

1.0E-03errors. Models may then ‘specialize’

• Can be likened to B i M d l

data

0.0E+001000 1500 2000

Bayesian Model Averaging, where the weights vary in timetime

Can fit same model structure with different parameterizations: assumes that model uncertainty does not arise solely out of the assumed model structure

Mixture models- alternative parameterizations suitable at different times

0 3

0.35

0.4

0.45

0 8

1

1.2

Probability Model 1Modeled

Fit two parameterizations of the single best model (combined

0 1

0.15

0.2

0.25

0.3

Flow

(mm

)

0.4

0.6

0.8

Prob

abili

ty

Observedg (

temperature/radiation index melt, pdm model)

0

0.05

0.1

1 1001 2001 3001 4001 50010

0.2

Mixture models- alternative parameterizations suitable at different times

0.3

0.35

0.4

0.45

0.8

1

1.2

Probability Model 1ModeledObserved

Model preference changes according to:

•Response to event

0.1

0.15

0.2

0.25

Flow

(mm

)

0.4

0.6

Prob

abili

ty

Observed

•Time of season

Comparison of alternate model simulations can indicate which

0

0.05

1 1001 2001 3001 4001 50010

0.2simulations can indicate which parameters are most sensitive to selected calibration period

HME approach gives good fit to data, but has problems with:•Identifiability•Interpretation•Interpretation•Predictions

Combining multiple model parameterizations: catchment “states” catchment states

Hydrologic model: Topmodel q

Logistic gating

function

q 1 q 2

xx

Model 1 Model 2

Two Component HME

A single-level two-component Hierarchical Mixture of Experts model

91

101

111

121

Tarrawarra Catchment Two Component HME

31

41

51

61

71

81 Contour Map

From Hornberger, 19981 11 21 31 41 51 61 71 81 91 101 111 121

1

11

21

31

Combining multiple model parameterizations: catchment “states” catchment states

0 0025

0.003

Simulations from individual mixture

0.002

0.0025component models

0 001

0.0015Q1Q2

0.0005

0.001

0

Combining Multiple Model Parameterizations Model “States” Parameterizations: Model States

0.002

0.0025

0.8

0.9

1

0.0015

rge

(m)

0 5

0.6

0.7

QobsQmean

0.001Dis

cha

0.3

0.4

0.5 QmeanProbability

0

0.0005

0

0.1

0.2

Hour

What about prediction?p

To use the model for prediction means finding anTo use the model for prediction means finding an appropriate catchment descriptor and a function relating this to the probability switching between relating this to the probability switching betweenmodels

Possible predictorsPossible predictorsAntecedent rainfall

Modelled catchment storageModelled catchment storage

Time of the year

The best predictors are often related to the mostThe best predictors are often related to the most dynamic catchment mechanisms

Model Aggregation as a Predictive Tool-Comparison of predictorsComparison of predictors

Model Predictor -0.5 BICTopmodel N/A 32685op ode / 3 685

Model Aggregation as a Predictive Tool-Comparison of predictorsComparison of predictors

Model Predictor -0.5 BICTopmodel N/A 32685op ode / 3 6852 Component

HME Preceding rainfall 33445Change in storage

deficit 33487Change in unsaturatedChange in unsaturated

zone storage 33428Unsaturated zone

storage 33455

Model Aggregation as a Predictive Tool-Comparison of predictorsComparison of predictors

Model Predictor -0.5 BICTopmodel N/A 32685op ode / 3 6852 Component

HME Preceding rainfall 33445Change in storage

deficit 33487Change in unsaturatedChange in unsaturated

zone storage 33428Unsaturated zone

storage 33455

Benefits of the HME approachpp

HME provides an improved framework for incorporating multiple sources of model uncertainty in p g p yhydrology

The HME approach allows combination of multiple models and parameterizations in a single framework

Using Mixture Modeling as a Method of Comparing Model Structures Parameters and ErrorsModel Structures, Parameters and Errors

• HME can highlight problems in the model structure• For conceptual models: different responses in wet and dry 

periods; different ways to model the catchment storage

• For distributed models: different patterns of soil moisture in wet and dry periods; different assumptions about thewet and dry periods; different assumptions about the recession properties

• A mixture of error distributions can provide better A mixture of error distributions can provide betterprediction limits and better model heteroscedasticity

Alternative approach: hierarchical model

• Most temporally sensitive parameters are conditioned on observed/modeled exogenous data

• Easier to interpret in light of the conceptualized hydrologic processes• Look at extent to which parametric variability informs model Look at extent to which parametric variability informs model

structural uncertainty

0 4

0.45

0 973

0.9735•Storage parameter differentiates alternative HME components

0 2

0.25

0.3

0.35

0.4

w (m

m)

0 9705

0.971

0.9715

0.972

0.9725

0.973

e Pa

ram

eter

p

•Condition this on the watershed melt and temperature

0

0.05

0.1

0.15

0.2

Flow

0 968

0.9685

0.969

0.9695

0.97

0.9705

Stor

ageModeled

Observed

Storage RecessionModel Max log-

likelihoodHierarchical 157910

1 1001 2001 3001 4001 50010.968

HME 18068

Comparison of aggregation and hi hi l hhierarchical approaches

How do these approaches compare for:pp p1. Assessing model structural uncertainty

Ensemble methods span the breadth of model space with varying degrees to give a better assessment of model uncertaintydegrees to give a better assessment of model uncertainty.HME and hierarchical formulations can highlight problems in the assumed model structure.

2 Improving model predictions2. Improving model predictionsThe ensemble approach should give a more consistent performance for the main variable of interest. The hierarchical approach does hold promise in improving model performance.promise in improving model performance.

3. InterpretabilityEnsemble and HME approaches are less useful beyond the variable of interest A more complex hierarchical model may give a model structure interest. A more complex hierarchical model may give a model structure greater flexibility and a simulation more consistent with internal watershed processes.

Can we use multi-model approaches for better model building?better model building?

For improved conceptual model assessment we should consider that parameter variability and model structural uncertainty are linked.The HME approach and multi‐model approaches can be usedThe HME approach and multi model approaches can be used to determine the utility of alternative models under different watershed conditionsThese approaches can be used to improve existing models forThese approaches can be used to improve existing models for better interpretability of internal watershed dynamics and their variability 

Modeling the Catchment Via Mixtures: an Uncertainty Framework for Dynamic Hydrologic SystemsDynamic Hydrologic Systems

Lucy MarshallAssistant Professor of Watershed AnalysisDepartment of Land Resources and Environmental SciencesDepartment of Land Resources and Environmental SciencesMontana State University

Email: lmarshall@montana.edu

References

Rainfall Runoff ModelsMoore, R. J. (1985). "The probability-distributed principle and runoff production at point and basin scales." Hydrological Sciences 30(2): 273-297.Beven, K., et al. (1995), Topmodel, in Computer Models of Watershed Hydrology, dit d b V P Si h 627 668 W t R P bli ti Hi hl d R h edited by V. P. Singh, pp. 627-668, Water Resources Publications, Highlands Ranch,

Colorado.Bayesian Inference and Adaptive MCMC Algorithms

Clark JS (2005) Why environmental scientists are becoming Bayesians. Ecology Letters ( ) y g y gy8(1)Haario, H., et al. (2001), An adaptive Metropolis algorithm, Bernoulli, 7(2), 223-242.Haario, H., M. Laine, et al. (2006). "DRAM: Efficient adaptive MCMC." S d C (4) 339 3 4Statistics and Computing 16(4): 339-354.

Hierarchical mixtures of ExpertsJacobs, R. A., et al. (1997), A Bayesian approach to Model Selection in Hierarchical Mixtures-of-Experts Architectures Neural Networks 10(2) 231-241Mixtures-of-Experts Architectures, Neural Networks, 10(2), 231-241.Marshall, L., et al. (2006), Modeling the catchment via mixtures: Issues of model specification and validation, Water Resour. Res., 42(11), 1-14.

Adaptive Bayesian algorithmsp y g

• The Adaptive Metropolis(AM) algorithm (Haario et al(AM) algorithm (Haario et al.,2001):• The covariance of the proposal

distribution is updated using the

AM Jump Space

Initial DRAM Jump Space distribution is updated using the

information gained from thesimulation thus far.

• Often plagued by initializationbl i h l i h

p p

problems, causing the algorithm tobecome trapped in local optima.

• The Delayed RejectionAdaptive Metropolis (DRAM)Adaptive Metropolis (DRAM)algorithm (Haario et al. 2006):• Reduces the probability that the

algorithm will remain at the current

Figure 2. A theoretical parameter surface, diagramming AM &DRAM algorithms’ ability to explore the parameter surface.Rings represent distance from the current location an algorithmcan explore. These exploration limits illustrate DRAM’s ability algorithm will remain at the current

state.can explore. These exploration limits illustrate DRAM s abilityto search more space and AM’s tendency to falsely convergeto local maxima because of its more constricted search area.