1

Phase 5.3 Calibration

Gary Shenk

3/31/2010

2

Calibration Method

• Calibration method largely unchanged for several years– P5.1 – 8/2008 - first automated calibration– P5.2 – 6/2009 - better constraints on parameters and

regional factors– P5.3 – 2/2010 - few small changes in reaction to new

scenario builder data

• Reviews– WQSC– Modeling Subcommittee– STAC review

3

Watershed Model Inputs

• Phase 5.1– No Scenario Builder

• Phase 5.2– Half-Built Scenario Builder with known issues

• Phase 5.3– Final TMDL Scenario Builder

4

Fixed Issues with Scenario Builder for phase 5.3

• Realistic uptake values

• Realistic nutrient applications

• Low variability between states for uptake and application

• Manure spread logic improved

• Scenarios now possible within Scenario Builder

5

Other P5.3 changes

• Land Use – – Better characterization of ag land location– Better trend in urban land

• Point Source– Addition of “non-significant” sources

• Septic– Tied to land use modeling

6

River Calibration Criteria

• CFD only

• Estimator Loads for Regional Factors

• STAC thought this was good calibration strategy but not a representative way to present the results

• Recommended that results communicated in the outputs of interest (loads)

7

Comparisons

• Statistics– Phase 5 and Estimator

• Total Loads over space• Loads at a point over time

– Phase 5 and USGS unbiased Samples– Phase 5 and Validation

• Calibration Plots– Phase 4 and Phase 5– Phase 5 all station

• Compare Loads to Previous Models

ftp://ftp.chesapeakebay.net/modeling/phase5/calibration_pdfs/p53_2010_02/

8

9

Log of WSM and Estimator TN Loads

5

5.5

6

6.5

7

7.5

8

8.5

5 5.5 6 6.5 7 7.5 8 8.5

Estimator (pounds per year)

WS

M p

5.2

(pou

nds

per

year

)

wsm p5.3

wsm p5.3 PQUAL

wsm p5.2

1:1

10

Log of WSM and Estimator TP Loads

5

5.2

5.4

5.6

5.8

6

6.2

6.4

6.6

6.8

7

5 5.2 5.4 5.6 5.8 6 6.2 6.4 6.6 6.8 7

Estimator (pounds per year)

WS

M p

5.2

(pou

nds

per

year

)

wsm p5.3

wsm p5.3 PQUAL

wsm p5.2

1:1

11

Log of WSM and Estimator TSS Loads

5

6

7

8

9

10

11

5 6 7 8 9 10 11

Estimator (pounds per year)

WS

M p

5.2

(pou

nds

per

year

)

wsm p5.3

wsm p5.3 PQUAL

wsm p5.2

1:1

12

Correlation of Fall Line Stations vs Estimator Annual Loads TN

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Sus

queh

anna

Pat

uxen

t

Pot

omac

Rap

paha

nnoc

k

Mat

tapo

ni

Pam

unke

y

Jam

es

App

omat

tox

Cho

ptan

k

Mod

el e

ffic

ienc

y

wsm p5.3

wsm p5.3 PQUAL

wsm p5.2

13

Correlation of Fall Line Stations vs Estimator Annual Loads TP

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Sus

queh

anna

Pat

uxen

t

Pot

omac

Rap

paha

nnoc

k

Mat

tapo

ni

Pam

unke

y

Jam

es

App

omat

tox

Cho

ptan

k

Mod

el e

ffic

ienc

y

wsm p5.3

wsm p5.3 PQUAL

wsm p5.2

14

Correlation of Fall Line Stations vs Estimator Annual Loads TSS

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Sus

queh

anna

Pat

uxen

t

Pot

omac

Rap

paha

nnoc

k

Mat

tapo

ni

Pam

unke

y

Jam

es

App

omat

tox

Cho

ptan

k

Mod

el e

ffic

ienc

y

wsm p5.3

wsm p5.3 PQUAL

wsm p5.2

15

'Unbiased' USGS samples vs WSM Population TN p5.2 AGCHEM

0.1

1

10JL

7_68

00_7

070

JL7_

7100

_703

0

JA5_

7480

_000

1

YM

4_66

20_0

003

YP

4_67

20_6

750

RU

5_60

30_0

001

PS

2_67

30_6

660

SW

7_16

40_0

003

SU

7_08

50_0

730

SU

8_16

10_1

530

PS

5_52

40_5

200

SL9

_249

0_25

20

SL9

_272

0_00

01

PM

7_48

20_0

001

SJ6

_213

0_00

03

EM

2_39

80_0

001

PS

3_51

00_5

080

PM

2_28

60_3

040

XU

3_46

50_0

001

PM

4_40

40_0

003

PU

3_32

90_3

390

PU

2_30

90_4

050

SL3

_242

0_27

00

TN c

once

ntra

tion

(m

g/l)

WSM 10

WSM 25

WSM 50

WSM 75

WSM 90

GS 10

GS 25

GS 50

GS 75

GS 90

16

'Unbiased' USGS samples vs WSM Population TN p5.3

0.1

1

10JL

7_68

00_7

070

JL7_

7100

_703

0

JA5_

7480

_000

1

YM

4_66

20_0

003

YP

4_67

20_6

750

RU

5_60

30_0

001

PS

2_67

30_6

660

SW

7_16

40_0

003

SU

7_08

50_0

730

SU

8_16

10_1

530

PS

5_52

40_5

200

SL9

_249

0_25

20

SL9

_272

0_00

01

PM

7_48

20_0

001

SJ6

_213

0_00

03

EM

2_39

80_0

001

PS

3_51

00_5

080

PM

2_28

60_3

040

XU

3_46

50_0

001

PM

4_40

40_0

003

PU

3_32

90_3

390

PU

2_30

90_4

050

SL3

_242

0_27

00

TN c

once

ntra

tion

(m

g/l)

WSM 10

WSM 25

WSM 50

WSM 75

WSM 90

GS 10

GS 25

GS 50

GS 75

GS 90

17

'Unbiased' USGS samples vs WSM Population TP p5.2 Agchem2

0.01

0.1

1P

S2_

6730

_666

0

RU

5_60

30_0

001

JA5_

7480

_000

1

SW

7_16

40_0

003

SL9

_272

0_00

01

YM

4_66

20_0

003

EM

2_39

80_0

001

SU

7_08

50_0

730

SL9

_249

0_25

20

PM

7_48

20_0

001

JL7_

6800

_707

0

SJ6

_213

0_00

03

YP

4_67

20_6

750

JL7_

7100

_703

0

PS

5_52

40_5

200

SU

8_16

10_1

530

PM

2_28

60_3

040

PU

3_32

90_3

390

XU

3_46

50_0

001

PS

3_51

00_5

080

PU

2_30

90_4

050

PM

4_40

40_0

003

SL3

_242

0_27

00

TP c

once

ntra

tion

(m

g/l)

WSM 10

WSM 25

WSM 50

WSM 75

WSM 90

GS 10

GS 25

GS 50

GS 75

GS 90

18

'Unbiased' USGS samples vs WSM Population TP p5.3

0.01

0.1

1P

S2_

6730

_666

0

RU

5_60

30_0

001

JA5_

7480

_000

1

SW

7_16

40_0

003

SL9

_272

0_00

01

YM

4_66

20_0

003

EM

2_39

80_0

001

SU

7_08

50_0

730

SL9

_249

0_25

20

PM

7_48

20_0

001

JL7_

6800

_707

0

SJ6

_213

0_00

03

YP

4_67

20_6

750

JL7_

7100

_703

0

PS

5_52

40_5

200

SU

8_16

10_1

530

PM

2_28

60_3

040

PU

3_32

90_3

390

XU

3_46

50_0

001

PS

3_51

00_5

080

PU

2_30

90_4

050

PM

4_40

40_0

003

SL3

_242

0_27

00

TP c

once

ntra

tion

(m

g/l)

WSM 10

WSM 25

WSM 50

WSM 75

WSM 90

GS 10

GS 25

GS 50

GS 75

GS 90

19

'Unbiased' USGS samples vs WSM Population TSS - p5.2 Agchem2

0.1

1

10

100

1000P

S3_

5100

_508

0

PS

2_67

30_6

660

JA5_

7480

_000

1

PS

5_52

40_5

200

EM

2_39

80_0

001

YM

4_66

20_0

003

JL7_

6800

_707

0

RU

5_60

30_0

001

SW

7_16

40_0

003

PU

3_32

90_3

390

JL7_

7100

_703

0

PM

2_28

60_3

040

YP

4_67

20_6

750

SL9

_272

0_00

01

PU

2_30

90_4

050

PM

4_40

40_0

003

PM

7_48

20_0

001

SU

7_08

50_0

730

SJ6

_213

0_00

03

SU

8_16

10_1

530

SL9

_249

0_25

20

XU

3_46

50_0

001

SL3

_242

0_27

00

TSS

con

cent

rati

on (

mg/

l)WSM 10 WSM 25WSM 50 WSM 75WSM 90 GS 10GS 25 GS 50GS 75 GS 90

20

'Unbiased' USGS samples vs WSM Population TSS - p5.3

0.1

1

10

100

1000P

S3_

5100

_508

0

PS

2_67

30_6

660

JA5_

7480

_000

1

PS

5_52

40_5

200

EM

2_39

80_0

001

YM

4_66

20_0

003

JL7_

6800

_707

0

RU

5_60

30_0

001

SW

7_16

40_0

003

PU

3_32

90_3

390

JL7_

7100

_703

0

PM

2_28

60_3

040

YP

4_67

20_6

750

SL9

_272

0_00

01

PU

2_30

90_4

050

PM

4_40

40_0

003

PM

7_48

20_0

001

SU

7_08

50_0

730

SJ6

_213

0_00

03

SU

8_16

10_1

530

SL9

_249

0_25

20

XU

3_46

50_0

001

SL3

_242

0_27

00

TSS

con

cent

rati

on (

mg/

l)WSM 10 WSM 25WSM 50 WSM 75WSM 90 GS 10GS 25 GS 50GS 75 GS 90

21

Calibration vs validation KS statistic Nitrogen - AGCHEM

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

calibration

valid

atio

n

Validation Better

Calibration Better

22

Calibration vs validation upper concentration bias Phosphorus - AGCHEM

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

calibration

valid

atio

n

Calibration Better

Calibration Better

Validation Better Validation Better

23

Calibration vs validation upper concentration bias Sediment - AGCHEM

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

calibration

valid

atio

n

Calibration Better

Calibration Better

Validation Better Validation Better

24

AGCHEM vs PQUAL KS statistic Nitrogen

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

AGCHEM

PQ

UA

L

PQUAL Better

AGCHEM Better

25

AGCHEM vs PQUAL upper concentration bias Phosphorus

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

AGCHEM

PQ

UA

L

AGCHEM Better

AGCHEM Better

PQUAL Better PQUAL Better

26

AGCHEM vs PQUAL upper concentration bias Sediment

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

calibration

valid

atio

n

AGCHEM Better

AGCHEM Better

PQUAL Better PQUAL Better

27

TMDL Allocations Based on

• No Action

• E3

• Riverine Delivery Factors

• Estuarine Delivery Factors

28

TN Delivery

0.00

0.20

0.40

0.60

0.80

1.00

1.20

NY PA MD DE DC WV VA

p5.2

p5.3

29

TP Delivery

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

NY PA MD DE DC WV VA

p5.2

p5.3

30

First Look at Draft Scenarios

31

TN comparison phase 5.2 and 5.3

0

50

100

150

200

250

300

350

400

calib NoAction 1985 2007/8 E3

mill

ion

lbs

per

year

p52

p53

32

TP comparison phase 5.2 and 5.3

0

5

10

15

20

25

30

35

40

calib NoAction 1985 2007/8 E3

mill

ion

lbs

per

year

p52

p53

33

TN, p5.2, goal=200, WWTP = 4.5-8 mg/l, other: max=min+20%,

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0 1 2 3 4 5 6 7 8 9 10

Relative Effectiveness

Per

cent

red

ucti

on f

rom

201

0 no

BM

Ps

to

E3

All Other

WWTP

34

TN, p5.3, goal=200, WWTP = 4.5-8 mg/l, other: max=min+20%,

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0 1 2 3 4 5 6 7 8 9 10

Relative Effectiveness

Per

cent

red

ucti

on f

rom

201

0 no

BM

Ps

to

E3

All Other

WWTP

35

TP, p5.2, goal=15, WWTP = .22 - .54 mg/l, other: max=min+20%,

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0 1 2 3 4 5 6 7 8 9 10

Relative Effectiveness

Per

cent

red

ucti

on f

rom

201

0 no

BM

Ps

to E

3

All Other

WWTP

36

TP, p5.3, goal=15, WWTP = .22 - .54 mg/l, other: max=min+20%,

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0 1 2 3 4 5 6 7 8 9 10

Relative Effectiveness

Per

cent

red

ucti

on f

rom

201

0 no

BM

Ps

to E

3

All Other

WWTP

37

TN Progress 2008/2007 as a fraction of E3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

DE DC MD NY PA VA WV

mill

ion

lbs

per

year

p52

p53

38

TP Progress 2008/2007 as a fraction of E3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

DE DC MD NY PA VA WV

mill

ion

lbs

per

year

p52

p53

39

Percent of Target Reached from No Action

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

DC DE MD NY PA VA WV Total

TN p5.2

TN p5.3

TP p5.2

TP p5.3

40

Additional Analyses before WQGIT

• Investigate changes in progress for NY, DE, and WV

• Verify that WWTP is correct

• 2007 shows no progress for ESVA

• Source contributions

• . . .

41

Summary

• Calibration method has been stable for years.

• Scenario Builder is now producing reasonable input data

• Phase 5.3 calibration similar to phase 5.2– Point source based changes in Potomac and

Patuxent– Coastal Plain changes in unmonitored area

• Delivery Factors similar

42

Scenario Builder: Role, Documentation and Planned Continued

EnhancementsChris Brosch

Chesapeake Bay Program Nonpoint Source AnalystUniversity of Maryland/CBPO

4343

Scenario Builder

A database program that generates inputs for the

Phase 5 Chesapeake Bay Watershed Model

Snapshot:

Land Use AcreageBMPsFertilizerManureAtmospheric DepositionPoint SourcesSeptic Loads

44

45

Scenario Builder Planned Enhancements• Version 2.2a: System Maintenance and Documentation Release

– System documentation updated • Version 2.3: Septic and Atmospheric Deposition

– Add these are two new sub-systems • Version 2.4: BMP Descriptions and Other BMP Files

– Accessory BMP files that the model needs to process BMP data from Scenario Builder.

– Input the Phase 5.3 watershed model outputs • Version 2.5: Improve Animal Waste Management System BMPs and

Dead Birds– Both are being addressed by BMPs now—will be addressed more accurately

• Version 2.6: Wastewater Sub System – Will automate input data generation over 3,000 facilities

• Version 3: NEIEN Exchange– Conversion of NEIEN BMP exchange data into Scenario Builder formats.

• Version 4: Data Products– Developing reports or other data products that will stream-line the process for

states, locals and other partners/stakeholders to request information• Version 5: User Interface

– Evolution of version 2.2 User Interface for running “what if” scenarios

46

Scenario List• We have

– 1985 (1985 and allocation air)– 2007 (2007 and allocation air) (not final)– 2010 No Action– 2010 E3 with N-based NM (not final)– VA EPIL (not final)

• Next Up– 1985 No Action– 1985 E3– 2010 E3 with P-based NM– 2008– Trib Strategy– 2009

WQGIT