lecture i hydrological modeling requirements for water...

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Lecture I

Hydrological modeling requirements for Water

Resources Applications - Model Calibration and

parameter Estimation Issues Soroosh Sorooshian

Center for Hydrometeorology and Remote Sensing

University of California Irvine

The Abdus Salam ICTP Workshop on: Water Resources in Developing Countries - Planning and

Management in a Climate Change Scenario

Trieste, Italy: May 6th – 17th 2013

Center for Hydrometeorology and Remote Sensing, University of California, Irvine and many more …

S. Sellars

University of California Irvine (UCI) Research Team: Present and Recent Past

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Improve California’s water supply management through:

• Forecast system (CaliForecast) •Improved decision optimization

Prepare the next generation of hydrologists and water resources engineers

Utilizing Information Technology to provide world-wide access to real-time global precipitation products: http://hydis.eng.uci.edu/gwadi/

Develop state-of-the-art systems to estimate rainfall from satellite observations at global scale and high spatial and temporal resolutions

Improve the performance and reliability of hydrologic, flood, and water supply forecasting models, particularly those used by the National Weather Service and other operational agencies.

Center for Hydrometeorology and Remote Sensing

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Two Primary Water Resources/Hydrology

Challenges:

• Hydrologic Hazards ( Floods and

Droughts)

• Water Supply Requirements ( Quantity

and Quality)

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Hydrologic Forecasting Needs: Flash Floods

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Droughts: The Other hydrologic Extreme

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Two Primary Water Resources/Hydrology

Challenges:

• Hydrologic Hazards ( Floods and

Droughts)

• Water Supply Requirements ( Quantity

and Quality)

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Increasing Population: Number of Mega Cities

Global Urban population 1970: ~37%

2010: ~53%

Projected Global Population: 8.3 Billion by 2025

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Distribution of Fresh Water Use

90.8 33.4%

17.1%

49.5%

460

7.0% 6.0%

87.0%

36.47

18.6%

22.0% 59.4%

117

60.0% 17.0%

23.0%

467.34

45.2%

13.1%

41.7%

380

4.0% 3.0%

93.0% Agriculture

Industry

Domestic

Fresh Water Use

(109 Cubic Meters)

Water Source

Water Use

USA China India

Russia Japan Brazil

92% 6%

2%

70.3 Iran

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Global Climate: Past Decade and Prediction of End of 21st Centaury

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Primary Solution To Meet

Hydrologic Extremes and

Water Resources Needs

Engineering Approach: Control, Store, Pump and Transfer

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Hoover Dam

A Century of Water Resources Development: Engineering success

Central Arizona Project Aqueduct

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

1800 1900

1950

Impact on Design and Operation of Global Infrastructure

2000

More than 70,000 Dams in the U.S.

Provided by: C. J. Vörösmarty

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Short Range Long Range

hours days weeks months seasons years decades

Required Hydrometeorologic Predictions

Forecast Requirements

Short-range Mid-range Long-range

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Short Range Long Range

hours days weeks months seasons years decades

Required Hydrometeorologic Predictions

Forecast Requirements

Mid-range Long-range

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

A Key Consideration:

The Link Between

Climate and Hydrology

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Global Warming And Hydrologic Cycle Connection

Heating

Temperature Evaporation

Water

Holding

Capacity

Atmospheric

Moisture

Temperature oF Sa

tura

ted

Va

po

r P

res

su

re

t t+20

Green

House

Effect

Rain

Intensity

Drought Flood

Flood Drought

Created by: Gi-Hyeon Park

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

P

E

Qs

D Ss

D Sg Qg

Ig

Coupled

Ocean-Atmosphere

Models

Mesoscale

Models

SVATs

Hydrologic/Routing

Models

Water Resources

Applications

GEWEX

CLIVAR

Hydrologic Services

Water resources

management agencies

From the Global- to Watershed-Scale

CLiC

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Hydroclimate Science and Hydrologic/Water Resources Engineering

Hydrologic/Hydraulic Routing,

Water Resources Models

SCIENCE ENGINEERING

Hydrologic/Hydraulic and

Water Resources Engineering Hydroclimate Science

P

E

Qs

D Ss

D Sg Qg

Ig

Mesoscale

Models

SVATs

GCMs

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

River Basins and Watersheds

Continental Scale:

Watershed Scale:

Where hydrology happens

Where stakeholders exist

Different Scales

Different Issues

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Short Range Long Range

hours days weeks months seasons years decades

Climate-Scale approaches to addressing hydrologic extremes

Forecast Requirements

Long-range

•Use of climate models:

down-scaling and ensemble

schemes

•Traditional statistical

hydrology methods:

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Pre

cipit

ati

on

pre

dic

tion

Time (years)

Present Future 20 40 60 80

Climate Model Downscaling to Regional/Watershed Scales

Generation of Future Precipitation Scenarios

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Ensemble Approach

Generation of Future Precipitation Scenarios

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Present Future 20 40 60 80 Time (years)

Pre

cipit

ati

on

pre

dic

tion

Present Future

Time (years)

Flo

w

Generation of Future Runoff Scenarios

Downscaled Precipitation to Runoff Generation

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

1 0.5 0.2 0.1 0.05 0.02 0.01 0.005 0.002 0.001

Probability of Exceedance

Return Period (Years) 1 2 5 10 20 50 100 200 500 1000

600

500

400

300

200

100

0

Dis

ch

arg

e

(1000 c

fs)

Alternative Approach to Climate Model Downscaling

traditional statistical hydrology methods

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Probability density function

Time (years)

Presen

t Future

10 20 30 40 50 60 70 80 90 100 -100 -75 -50 -25 0

Flo

w (m

3/s

ec)

Past

0255075

100125150175200225250

1900 1910 1920 1930 1940 1950 1960 1970 1980 1990 2000Flo

w (

m3/s

ec)

Time (years)

Statistical Hydrology: “synthetic” stream flow Generation

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Brief Review of Rainfall

Runoff modeling:

Progress in Hydrologic

Modeling

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Fundamental Law

Input Output

Change In Storage

I O

DS

I – O =DS

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Nature’s Way Terrain

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

The Watershed

Area km2 12.78

Perimeter km 19.344

Min Elevation m 478.00

Max Elevation m 1756.00

Mean Elevation 930.34

Max Flow Length 8.878

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Eva

po

ratio

n

(Oce

an

)

Eva

po

tra

nsp

ira

tion

Pre

cip

ita

tio

n

Pre

cip

ita

tio

n P

recip

ita

tio

n

Su

blim

atio

n

Eva

po

ratio

n

(La

ke

s &

Re

se

rvo

irs)

Va

po

r D

iffu

sio

n

Lake

Infiltra

tio

n

Dee

p P

erc

ola

tio

n

River

Aquifer

Eva

po

ratio

n

(La

nd

Su

rfa

ce

)

Vegetation

Interflow

Ponce, 1989

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Trace The Water Drop

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Distributed

Physically-based

API Model A

C

D

B

Lumped Conceptual

Distributed (Mike SHE)

VIC Model

Evolution of Hydrologic R-R Models

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

MODEL

PARAMETER

ESTIMATION

DATA

If the “World” of

Watershed Hydrology

Was Perfect!

Hydrologic Modeling: 3 Elements!

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Flow in Channels: How far can we go simplifying?

V = n-1 R2/3 S1/2

n – Manning Coefficient

R – Hydraulic Radius

S – Energy Slope

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Hydrologic Modeling

Will be Covered By Professor Aghakouchak

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Model Calibration

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

The Identification Problem

1. Select a model structure (Input-State-Output equations)

2. Estimate values for the parameters

U

U – Universal Set M1()

M2()

Mi() – Selected Model Structure

D

D

O

O

B - Basin

“The Truth”

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

The Concept of Model Calibration

Measured

Outputs

Yt

t

Real World

Measured

Inputs

MODEL () Computed

Outputs

Prior

Info

Computed

Outputs

+ -

Optimization Procedure

“Calibration: constraining the model to be consistent with observations”

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

The Automatic

Calibration Approach

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Calibration components

Objective Function

Search Algorithm

Sensitivity Analysis

Problems with identifiability

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

[General Exponential Power Density] (Posterior Parameter Probability Distribution Function)

N

j

jiN

i

ecp

1

1/2

exp,|

y

Uniform

Normal

Exponential

)1(

0

1

.)|(p

Calibration Criterion

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Objective function Parameter Space

θ1

θ2

F(θ)

θ

Parameter Space Objective Function Space

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Parameter Sensitivity

θ1

θ2

Parmeter Space

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Parameter Sensitivity

θ1

θ2

Parameter Space

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Parameter X

Ob

jecti

ve

Fu

ncti

on

True Parameter Set

The Ideal case: Convex Optimization

Created By G-H Park

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Ob

jecti

ve

Fu

ncti

on

Parameter X

True Parameter Set

Difficulties in Global Optimization

Created By G-H Park

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Parameter X

Ob

jecti

ve F

un

cti

on

Global Optimum

Parameter Estimation (non-convex, multi-optima)

Created By G-H Park

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Parameter X

Ob

jecti

ve F

un

cti

on

Global Optimum

Created By G-H Park

Parameter Estimation (non-convex, multi-optima)

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Data information content

“Bucket Model”

Simple two parameter Model

Cm

ax

K S

P

Q

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Data information content C

ma

x

K S

P

Q

Multiple

spills

Cmax

K

No spills

Cmax

K

Cm

ax

K S

P

Q

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

More than one main convergence region

1.- Regions of

Attraction

Difficulties in Optimization

Duan, Gupta, and Sorooshian, 1992, WRR

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

More than one main convergence region

1.- Regions of

Attraction

2.- Local

Optima

Many small "pits" in each region

Difficulties in Optimization

Duan, Gupta, and Sorooshian, 1992, WRR

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

2.- Local

Optima

Many small "pits" in each region

More than one main convergence region

1.- Regions of

Attraction

3.- Roughness Rough surface with discontinuous derivatives

Difficulties in Optimization

Duan, Gupta, and Sorooshian, 1992, WRR

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

3.- Roughness Rough surface with discontinuous derivatives

2.- Local

Optima

Many small "pits" in each region

More than one main convergence region

1.- Regions of

Attraction

4.- Flatness Flat near optimum with significantly different parameter sensitivities

Difficulties in Optimization

Duan, Gupta, and Sorooshian, 1992, WRR

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

4.- Flatness Flat near optimum with significantly different parameter sensitivities

3.- Roughness Rough surface with discontinuous derivatives

2.- Local

Optima

Many small "pits" in each region

More than one main convergence region

1.- Regions of

Attraction

Difficulties in Optimization

5.- Shape Long and curved ridges

Duan, Gupta, and Sorooshian, 1992, WRR

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Optimization Strategy – Local Direct Search

Calibration of the Sacramento Model

Downhill Simplex Method, Nelder & Mead, 1965

Duan, Gupta, and Sorooshian, 1992, WRR

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

The SCE-UA Algorithm …

(1992)

Duan, Gupta, and Sorooshian, 1992, WRR

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

The SCE-UA Algorithm …

Duan, Sorooshian, and Gupta 1992, WRR

The Shuffled Complex Evolution Algorithm

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

The Concept Behind SCE Method

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

The Concept Behind SCE Method

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

The Concept Behind SCE Method

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

The Concept Behind SCE Method

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

SCE Method – How it works …

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Shuffled Complex Evolution (SCE-UA)

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Global Optimization – The SCE-UA Algorithm

Simplex

Method

Shuffled

Complex

Evolution

(SCE-UA)

Duan, Gupta & Sorooshian, 1992, WRR

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Evolving Directions

Advances in Parameter Estimation

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Land-Surface Model

RHODE ISLAND

CONNECTICUT

FLORIDA

MISSISSIPPI

WEST

VIRGINIA

WASHINGTON

OREGONIDAHO

MONTANA

WYOMING

NORTH

DAKOTA

SOUTH

DAKOTA

NEBRASKA

IOWA

MINNESOTA

WISCONSIN

ILLINOIS

INDIANA OHIO

MISSOURIKANSASCOLORADO

UTAHNEVADA

CALIFORNIA

ARIZONA NEW MEXICO

OKLAHOMA

ARKANSAS

KENTUCKY

VIRGINIA

TEXAS

GEORGIA

ALABAMA

SOUTH

CAROLINA

NORTH CAROLINA

TENNESSEE

PENNSYLVANIA

MARYLAND

NEW JERSEY

NEW YORK

VERMONT

MAINE

DELAWARE

LOUISIANA

MICHIGAN

MASSACHUSETTSNEW HAMPSHIRE

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Multi-Objective Approaches

M()

Model

Radiation

Inputs Outputs

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Multi-Objective Optimization Problem

MOCOM Algorithm:

Does NOT require conversion

to a sequence of single optimization problems

Simultaneously finds

several Pareto Solutions

in a Single Optimization

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

ARM-CART SGP Site

100 km

Grid: ~100,000 km2

Luis A. Bastidas Z. (lucho@hwr.arizona.edu)

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

{H, Tg, Sw} {E} {Tg} {Sw}

Single- & Multi-Flux Calibrations {H}

E

Tg

H

Sw

Computed

Observ

ed

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

AGU Monograph – Now Available

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

End of Lecture I

Thank You For Listening

The Rio Grande River, NM Photo: J. Sorooshian 2005

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Additional Slides

Backup Material

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

UCt

UM

LM

Streamflow

Rainfall

percolation

LCt

UK

LK

PD = f(Z, X)

A look into the “heart” of R-R Models

Percolation Process is the

Core element in Partitioning

the rain between the various

stores

PD = f(Z, X)

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Free water

Tension water

Tension water

Percolation

RATE=PBASE(1+ZPERC*DEFR REXP)

NWS Soil Moisture Accounting Model: SMA-NWSRFS

Nonlinear structure with large number of parameters

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Parameter Uncertainty Methods

(1) First-order approximations near global optimum (Kuczera etal)

Limitations

• Assumes Model is Linear

• Assumes Posterior Dist. Guassian

(2) Generalized Likelihood Uncertainty Estimation (GLUE) method (Beven and co-workers)

(3) Markov Chain Monte Carlo (MCMC) methods

(Vrugt and others)

.)|( tp

.)|1( tp

1t t

1

2

.

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

.)|*(p

SCE-UA only solves for Mode of Distribution

Probability distribution

to be maximized

*

180 190 200 210 220 230 240 250 260 270 2800

5

10

15

20

= observations

= simulated flows

*

Hours

Flo

w

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Shuffled Complex Evolution Metropolis

SCE SCEM Vrugt, Gupta, Bouten & Sorooshian

WRR 2003

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Need estimates of the prediction uncertainty

*

Hours 180 190 200 210 220 230 240 250 260 270 2800

5

10

15

20

Uncertainty

associated

with parameters

Total Uncertainty

including structural

errors

.)|*(pProbability distribution

to be maximized

95%

*

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Flow Ranges instead of point estimates

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Multi-Criteria Calibration Concept

θ1

θ2

F1(θ)

F2(θ)

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

Multi-Criteria Calibration Approach

X1

X2

X3

M(θ)

Y1(θ)

Y2(θ) Y2obs

Y1obs

F2(θ)

F1(θ) +

-

+

-

f1

f2

1

2

f1

f2

Gupta, Sorooshian, Yapo, WRR, 1998

Center for Hydrometeorology and Remote Sensing, University of California, Irvine

1 2

3 4

6

1 2

A

B

4 3

5

Combination

Ro

uti

ng

Runoff

“Semi-distributed” Hydrologic Models