1

An overview of the use of reforecasts for improving

probabilistic weather forecasts

Tom Hamill

NOAA / ESRL, Physical Sciences Div.tom.hamill@noaa.gov

NOAA Earth SystemResearch Laboratory

2

What do we want from ensemble forecasts?

O

O

BAD(“unreliable”)

GOOD(“reliable”)

O

BEST

“sharp” and “reliable”

33

Dealing with ensemble errors: problems we’d like to correct through “calibration”

44

bias (drizzle over-forecast)

ensemble members toosimilar to each other.

Probabilities too smooth;downscaling needed.

5

Calibration and reforecasting

• Problems with probabilities directly estimated from raw ensemble: so then what?

• Would like f(O|F), that is, the probability distribution of the expected observed state given the forecast (much like your thought process as a forecaster).

today’s ensemble mean forecast

6

Calibration and reforecasting

• Problems with probabilities from raw ensemble: so then what?

• Would like f(O|F), that is, the probability distribution of the expected observed state given the forecast. (much like your thought process as a forecaster).

today’s ensemble mean forecast

lots of other forecaststhat are like today’s forecast

7

Calibration and reforecasting

• Problems with probabilities from raw ensemble: so then what?

• Would like f(O|F), that is, the probability distribution of the expected observed state given the forecast. (much like your thought process as a forecaster).

today’s ensemble mean forecast

lots of other forecaststhat are like today’s forecast

form ensemble fromobserved weatheron days of thosepast forecasts

8

The concept of reforecasting

• Approach: use FIXED model and data set of many past forecasts from this model. Correct current forecast using knowledge about the forecast errors of this model for several decades in the past (MOS on steroids)

• “Calibration” should implicitly:– adjust for model bias– adjust for any spread deficiency– downscale (coarse prediction grid --> predictable local

detail in observations).

9

NOAA’s reforecast data set

• “Reforecast” definition: a data set of retrospective numerical forecasts using the same model as is used to generate real-time forecasts.

• Model: T62L28 NCEP GFS, circa 1998

• Initial States: NCEP-NCAR Reanalysis II plus 7 +/- bred modes.

• Duration: 15 days runs every day at 00Z from 19781101 to now. (http://www.cdc.noaa.gov/people/jeffrey.s.whitaker/refcst/week2).

• Data: Selected fields (winds, hgt, temp on 5 press levels, precip, t2m, u10m, v10m, pwat, prmsl, rh700, heating). NCEP/NCAR reanalysis verifying fields included (Web form to download at http://www.cdc.noaa.gov/reforecast).

• Real-time probabilistic precipitation forecasts: http://www.cdc.noaa.gov/reforecast/narr

10

Can’t we calibrate with only a past few forecasts?

Consider training with a short sample in a climatologically dryregion. How could you calibrate this latest forecast?

you’d like enough training datato have somesimilar eventsat a similartime of yearto this one.

11

Analog high-resolution precipitation forecast calibration technique

(actually run with 10 to 75 analogs)

12

Analog high-resolution precipitation forecast calibration technique

(actually run with 10 to 75 analogs)

ApproximateO | F

13

Example: probability of greater than 25 mm/day (downscaled to 5 km)

Downscaling using PRISM / Mountain Mapper technology (C. Daly. Oregon St., NOAA RFC’s, OHD)

14

Verified over 25 years of forecasts; skill scores use conventional method of calculation which mayoverestimate skill(Hamill and Juras 2006).

15

Comparison against NCEP medium-range T126 ensemble, ca. 2002

the improvement is a little bitof increased reliability, a lotof increased resolution.

BS S =

res o lution - re l iabi l i tyuncerta inty

16

Effect of training sample size

colors of dots indicate which size analog ensembleprovided the largest amount of skill.

17

Real-time products

18

Conclusions

• Large improvement in probabilistic forecast skill and reliability by calibrating using large, stable data set of NWP forecasts / obs.

• Precipitation products are out there for you to use (www.cdc.noaa.gov/reforecast/narr)

• The NWS expects to produce more reforecasts and calibrated products in the coming years. We’re working with Zoltan Toth’s group at NCEP on this.

19

ReferencesHamill, T. M., J. S. Whitaker, and X. Wei, 2003: Ensemble re-forecasting: improving medium-range forecast skill using retrospective forecasts. Mon. Wea. Rev., 132, 1434-1447. http://www.cdc.noaa.gov/people/tom.hamill/reforecast_mwr.pdf

Hamill, T. M., J. S. Whitaker, and S. L. Mullen, 2005: Reforecasts, an important dataset for improving weather predictions. Bull. Amer. Meteor. Soc., 87, 33-46.http://www.cdc.noaa.gov/people/tom.hamill/refcst_bams.pdf

Whitaker, J. S, F. Vitart, and X. Wei, 2006: Improving week two forecasts with multi-model re-forecast ensembles. Mon. Wea. Rev., 134, 2279-2284. http://www.cdc.noaa.gov/people/jeffrey.s.whitaker/Manuscripts/multimodel.pdf

Hamill, T. M., and J. S. Whitaker, 2006: Probabilistic quantitative precipitation forecasts based on reforecast

analogs: theory and application. Mon. Wea. Rev., 134, 3209-3229 http://www.cdc.noaa.gov/people/tom.hamill/reforecast_analog_v2.pdf

Wilks, D. S., and T. M. Hamill, 2006: Comparison of ensemble-MOS methods using GFS reforecasts. Mon. Wea. Rev., 135, 2379-2390.. http://www.cdc.noaa.gov/people/tom.hamill/WilksHamill_emos.pdf

Hagedorn, R, T. M. Hamill, and J. S. Whitaker, 2008: Probabilistic forecast calibration using ECMWF and GFS ensemble forecasts. Part I: 2-meter temperature. Mon. Wea. Rev., 136, 2608-2619. http://www.cdc.noaa.gov/people/tom.hamill/ecmwf_refcst_temp.pdf

Hamill, T. M., R. Hagedorn, and J. S. Whitaker, 2008: Probabilistic forecast calibration using ECMWF and GFS ensemble forecasts. Part II: precipitation. Mon. Wea. Rev., 136, 2620-2632.http://www.cdc.noaa.gov/people/tom.hamill/ecmwf_refcst_ppn.pdf