geos 5311 lecture notes: model calibration file ... · pdf filegeos 5311 lecture notes: model...

GEOS 5311 Lecture Notes: Model Calibration

Dr. T. Brikowski

Spring 2013

0file: calibration.tex,v (vers. 1.10)1

Why Calibrate?

I numerical model and field data are imperfect representationsof the real world, e.g. well tests cannot sample the full rangeof heterogeneity

I the goal is to obtain a reasonably accurate representation thatis at least internally consistent

I this is done by calibrating or adjusting the model until errorsare minimized, as measured by qualitative or quantitativemeans

2

What is Calibration

I Model calibration is solution of an inverse problemI Forward problem: model parameters → model → prediction of

dataI Inverse problem: data → model → prediction of model

parameters (AKA parameter estimation)

I Calibration: given observed data (usually head), adjust modelparameters (usually K ) until model reproduces observation“closely enough”

I Remember perfect accuracy is impossible, especially sincesome errors cannot be reduced in the model, e.g.measurement errors inherent in well test results

I see Hill and Tiedeman (2007) for entire textbook on this topic

3

Calibration Options

I Check model against analytic solution results (verification,only possible for extremely simple cases)

I utilize global or local water mass balance within code (e.g.Modflow, Harbaugh et al., 2000) or add-ons(ZONEBUDGET, Harbaugh, 1990)

I compare to observations (usually head or flux) and adjustparameters

I use visual error indicators (e.g. “calibration targets”, EMRL,2003b)

I use sensitivities for guidance (e.g. ModflowObservation-Sensitivity Process, Hill et al., 2000)

I use formal parameter-estimation program, e.g. built in toModflow, or universal tools like UCODE (Poeter and Hill,1999) or PEST (Doherty, 1994)

4

Inverse “Protocols”

I a formal, pre-determined inverse modeling procedure is bestHill (1998) or Hill et al. (Modflow2K 2000)

I formal mathematical approaches are available (“automated”,e.g. Menke, 1984)

I Trial-and-error calibration: the typical approach in hydrology,“try things until it works”

5

Calibration Data: Head

I Head observations usually represent incomplete view oftransient variations

I Error sourcesI transient variations, try to get a dataset that represents a

single time (synoptic)I measurement errorI scaling effects: measurements made over different

scale/interval than used in modelI interpolation error: measured heads not located at a model

node

6

Calibration Data: Fluxes

I Flux dataI observed or inferred fluxes are very useful calibration dataI may have moderately large errors, use with cautionI most useful in providing range of possible parameter valuesI also help to distinguish which of many parameters to adjust

during calibration

7

Error Criteria

I Used to control iterative solutions

I Head error criterion: best to pick order of magnitude smallerthan level of accuracy desired

I Water balance errorI “residual” error in Modflow, an absolute error

(L3

T

)I most codes set this as a relative errorI 1% mass error acceptable, 0.1% betterI Modflow doesn’t automatically use this (user must read

output file, rerun if mass balance unacceptable)

I Note: GHB package in Modflow can lead to unintended masserrors if boundary conductance set too high

8

Trial-and-Error

I most common approach, easier now with graphical aids (Fig.1)

I Method: repeatedly solve forward problem, with ad hocadjustment of parameters until results match observationswithin tolerance

I strictly valid only if calibration goal specified in advance

I requires tens to hundreds of runs. In GMS be sure to useDisplay/Plot Wizard/Error vs. Simulation tomonitor calibration progress

9

Graphical Calibration Tool

Figure 1: Graphical calibration target in GMS. Error bar is green if errorless than interval, yellow if error is < 100%, red if error is ≥ 200%. Othermodeling packages such as GroundwaterVistas use similar icons. AfterEMRL (Fig. 11.2, 2003a).

10

Graphical Calibration Example

Figure 2: Example of graphical calibration targets. Results indicatemodel head too low to right, too high in middle, acceptable on the right(Brikowski and Faid, 2006). K might be adjusted in the oppositedirection to produce better results during trial-and-error calibration.

11

Automated Calibration

I theoretically better approach, in practice can be unsatisfactorybecause it ignores hydrologist’s “intuition”

I proper zonation is the key to a successful automated inversesolution

I Method:I user divides model area into a number of homogeneous zones,

characterized by one or more parameters, which may vary fromzone to zone

I program (e.g. Modflow2K Parameter Estimation/Senstivityprocess) repeatedly solves forward problem, adjustingparameters in prescribed fashion to minimize an objectivefunction

I Implementation:I objective function is usually the squared residuals (e.g.

least-squares difference between observed and modeled)

12

Automated Calibration (cont.)

I inverse codes vary mostly in the method used to find theminima in the objective function, and secondarily in the choiceof objective function

I best-fit value or sensitivity (effect on model of small changes inparameter) of parameters is reported

I most accessible code: Modflow2K with ParameterEstimation/Senstivity process enabled (Hill, 1998; Hill et al.,2000)

13

Evaluating the Calibration

I QualitativeI show map(s) of difference between calculated and observed

headI be sure errors are distributed ∼randomly in space

I Quantitative (calibration criterion)I best to include a quantitative measure of calibration. This is

already calculated during automatic calibrationI Error measures:

I Mean error: average difference between measured heads.Negative and positive errors may cancel out, giving misleadingindication of minimum error.

I Mean absolute error: average of absolute value of differencebetween measured heads. Avoids the canceling-out problem.

I Root Mean Squared Error (standard deviation). Square rootof average of of squared difference in heads. Weights largeerrors higher, best measure of error in general.

14

Traditional Sensitivity Analysis

I Quantifies uncertainty in the final calibrated model

I Procedure:I vary calibrated values through plausible range of values

(established before calibration)I plot change in calculated variable (e.g. head) as a function of

calibrated parameterI also examine change in spatial distribution of error

I Highlight or re-examine the most sensitive parameters

15

Traditional Sensitivity Plot

Figure 3: Traditional parameter sensitivity plot showing % change inparameter value vs. % change in head (right) and leakage from stream(left). Computed head is most sensitive (depends most) on Recharge andKh. After Anderson and Woessner (Fig. 8.15a, 1992).

16

MF2K Summary

I invoked by enabling the Sensitivity and/or ParameterEstimation processes in Modflow2K (see Modflow/Global

Options in GMS)

I Observation Process must also be active (i.e. create anobservation coverage in GMS using Map

Module/Coverages). Typically observed head at wells will beused, but flux or other observations are possible.

I plot sensitivity (in GMS using Display/Plot

Wizard/Parameter Sensitivities to determine whichparameters are best constrained (most sensitive) byobservation data (Fig. 4)

I my current approach:I trial-and-error calibration to “get used to system”I sensitivity run to determine which parameters are realistically

constrained by observations

17

MF2K Summary (cont.)

I parameter estimation run to get best automatic model (doneoutside of GMS, since I haven’t paid for the inverse module)

I fine-tune using trial-and-error incorporating hydrogeologic“intuition” (soft information)

18

GMS Sensitivity-Calibration Plot

Figure 4: GMS parameter sensitivity/calibration plot showing hydraulic conductivity parameter sensitivites(top) and results of trial-and-error calibration (bottom). Zones HK 100 and HK 300 are best constrained byobserved heads, other parameters may be controlled by other model features (e.g. streamflow for HK 200).

19

Sensitivity/Estimation Without GMS

I sensitivity information is stored in the Global file (*.glo)I find and plot the “COMPOSITE SCALED SENSITIVITY”

(same information that is plotted by GMS “ParameterSensitivity” plot, top of Fig. 4)

I peruse the “ONE-PERCENT SCALED SENSITIVITES”, whichgive the

I contour the 1% sensitivity arrays to determine where newobservations would be most likely to improve the model (notpossible in GMS?)

I best-fit parameter valuesI these are tabulated at the bottom of the global file (“FINAL

PARAMETER VALUES”)I Modflow2K also makes a forward run using these values,

which are tabulated at the top of the output file (*.out)I use these with caution, since they may be hydrologically

absurd if observations contain significant error, wereinsufficient in number or distribution, or you’re just unlucky.

20

Sensitivity/Estimation Without GMS (cont.)

I Follow the procedures in Hill (1998) for further evaluation ofparameter estimates/sensitivity

21

References

Anderson, M.P., Woessner, W.W.: Applied Groundwater Modeling.Academic Press, San Diego (1992)

Brikowski, T.H., Faid, A.M.: Pathline-calibrated groundwater flowmodels of Nile Valley aquifers, Esna, Upper Egypt. J. Hydrology324(1-4), 195–209 (2006)

Doherty, J.: PEST. Watermark Computing, Corinda, Australia (1994)

EMRL: GMS 4.0 Tutorials. Environmental Modeling ResearchLaboratory, Brigham Young Univesity, Provo, UT, 4.0 edn. (2003a)

EMRL: GMS Reference Manual. Environmental Modeling ResearchLaboratory, Brigham Young Univesity, Provo, UT, 4.0 edn. (2003b),http://www.bossintl.com/online_help/gms/

Harbaugh, A.W.: A computer program for calculating subregional waterbudgets using results from the u.s. geological survey modularthree-dimensional finite-difference ground-water flow model. Open-filereport ofr 90-0392, U. S. Geol. Survey, Reston, VA (1990)

22

References (cont.)Harbaugh, A.W., Banta, E.R., Hill, M.C., McDonald, M.G.:

MODFLOW-2000, the U.S. Geological Survey modular ground-watermodel – user guide to modularization concepts and the ground-waterflow process. Open File Rept. OFR00-92, U. S. Geol. Survey, Denver,CO (2000), http://water.usgs.gov/nrp/gwsoftware/modflow2000/ofr00-92.pdf, 121 p

Hill, M.C.: Methods and guidelines for effective model calibration. WaterResour. Investig. 98-4005, U.S. Geol. Survey (1998), http://water.usgs.gov/nrp/gwsoftware/modflow2000/WRIR98-4005.pdf, 90pp.

Hill, M.C., Banta, E.R., Harbaugh, A.W., Anderman, E.R.: User guide tothe observation, sensitivity, and parameter-estimation processes andthree postprocessing programs. Open File Report OFR 00-184, U. S.Geol. Survey, Denver, CO (2000), http://water.usgs.gov/nrp/gwsoftware/modflow2000/ofr00-184.pdf

Hill, M.C., Tiedeman, C.R.: Effective Groundwater Model Calibration:With Analysis of Data, Sensitivities, Predictions, and Uncertainty.Wiley, New York (2007), http://www.wiley.com/WileyCDA/WileyTitle/productCd-047177636X,descCd-reviews.html

23

References (cont.)

Menke, W.: Geophysical Data Analysis: Discrete Inverse Theory.Academic Press, Inc., Orlando, FL (1984)

Poeter, E.P., Hill, M.C.: Ucode, a computer code for universal inversemodeling. Computers & Geosciences 25(4), 457–62 (May 1999),http://dx.doi.org/10.1016/S0098-3004(98)00149-6

24