National Hurricane Center 2009 Forecast Verification

James L. FranklinBranch Chief, Hurricane Specialist Unit

National Hurricane Center

2009 NOAA Hurricane Conference

1

Verification Rules Verification rules unchanged for 2009. Results

presented here in both basins are preliminary.

System must be a tropical or subtropical cyclone at both forecast initial time and verification time. All verifications include depression stage except for GPRA track goal verification.

Special advisories ignored (original advisory is verified.

Skill baselines are recomputed after the season from operational compute data. Decay-SHIFOR5 is the intensity skill benchmark.

2009 Atlantic Verification

Values in green exceed all-time records.

• 48 h track error for TS and H only (GPRA goal) was 69.9 n mi, well below previous record of 86.2.

• Sample is very small (last year 346 forecasts, with 149 verifying at 5 days). Five-day sample is smallest ever.

VT NT TRACK INT(h) (n mi) (kt)============================ 000 144 9.3 1.2012 120 30.4 6.0024 96 44.9 11.1036 75 62.1 14.8048 61 73.0 17.1072 49 119.2 20.2096 38 197.9 19.3120 22 292.3 16.6

Four- and five day track error was almost exclusively along-track (slow).

Atlantic Track Errors vs. 5-Year Mean

Official forecast was better than the 5-year mean, even though the season’s storms were “harder” than normal.

OFCL errors were mostly below the 5-yr means, even though CLIPER5 errors were above their 5-yr means.

Atlantic Track Error Trends

Errors have been cut in half over the past 15 years. 2008 was best year ever. Smaller samples give more erratic trends at days 4-5.

Atlantic Track Skill Trends

2009 set skill records at 24-72 h. Sharp increase over past two years due to greater availability of the ECMWF?

2009 Track Guidance

Official forecast performance was very close to the consensus models. Good year for FSSE.

First year of availability for CMCI. A good start indeed.

Best dynamical models were ECMWF and GFS. UKMET and NOGAPS appear to be serious drags on the consensus. Will need to reevaluate TCON/TVCN members for 2010.

BAMD performed poorly (strong shear).

Atlantic Intensity Errors vs. 5-Year Mean

OFCL errors in 2009 were mostly at or above the 5-yr means, but the 2009 Decay-SHIFOR errors were above their 5-yr means, indicating storms with unusual behaviors.

Atlantic Intensity Error Trends

No progress with intensity.

Atlantic Intensity Skill Trends

Little net change in skill over the past several years.

2009 Intensity Guidance

Best model at every time period was statistical. Very good year for LGEM, which handles changes in the environment better than SHIPS.

2009 East Pacific Verification

VT NT TRACK INT(h) (n mi) (kt)============================000 270 9.4 1.6012 236 29.6 7.1024 202 50.7 12.8036 169 71.6 17.0048 136 88.9 18.8072 88 119.7 18.6096 59 163.2 19.1120 35 243.5 18.9

Values in green tied or exceeded all-time lows.

EPAC Track Error Trends

Since 1990, track errors have decreased by 30%-50%.

EPAC Track Skill Trends

Although errors were higher in 2009, skill was mixed.

EPAC Intensity Error Trends

Errors look pretty flat.

EPAC Intensity Skill Trends

Skill also seems flat in this decade.

2007-09 Genesis Forecast Verification

Relatively steady increase of verifying percentage with forecast percentage indicates good reliability. Forecasters can distinguish likelihood of genesis in 10% increments. NHC will provide public genesis forecasts to the nearest 10% in 2010. Fairly systematic low bias in the east Pacific.

2008-09 Experimental Rapid Intensification Forecast Verification

In house probabilistic forecast of the chance of 30 kt or more of intensification over the next 24 hours (climatology = 5%). EPAC curve (where low probs have high bias while high probs have a low bias) reflects underconfidence on the part of the forecaster (i.e., we know more than we think we do).

Summary Very small Atlantic sample. OFCL track errors

set records for accuracy from 24 to 72 h. Errors continue their downward trends, skill was also up.

Good years for FSSE, GFSI, EMXI, CMCI (first year of availability) in the Atlantic.

Not much new with intensity. Best model was statistical (LGEM).

Genesis forecasts show good reliability. NHC will provide these forecasts in 10% increments in 2010.

Forecast Error by Parameter

Radii errors look good (especially 64 kt radii), but this is misleading...

Forecast Skill by Parameter

Track forecast skill is very high (and higher at 5 days than it is at 24 h). Intensity skill is low but fairly steady with lead time. Wind radii skill falls off significantly with lead time (to near zero by 72 h).

30% skill of the 36 h hurricane radii forecasts represents accuracy improvement of 3-4 n mi over DRCL. This is meaningless in the context of the ~75 n mi mean track error.

Forecast ChangeVariance Explained at Longest Lead

Much more information in the intensity forecasts. Intensity forecasts have a very large “dynamic range”, while the vast majority of hurricane radii forecasts are within 20 n mi of initial radii (such that the forecast radii change is much smaller than typical track error.)

Forecast Parameter Scale Analysis

Tropical Cyclone Forecast Parameter Scale Analysis(Based on Atlantic Basin statistics, 2004-8)

Parameter Analysis Uncertainty

Verification Uncertainty

(at 36 h)

Forecast Skill at longest

lead

Variance Explained at longest lead

Track <1%30° ± 0.3°

20%75 nm ± 15

nm45% Very high

Intensity 10%100 kt ± 10 kt

~50%13 kt ± 7 kt* 10% 60%

Wind Radii

25-40%100 nm ± 25

nm25 nm ± 10 nm

~100%30 nm ± 25

nm10 nm ± 10

nm

0% (34kt/72h)30%

(64kt/36h)

17-34% (34kt)

8-12% (64 kt)

* Assumes 10% error applied to a 70 kt “typical” TC.

Considerations for Forecast Extension

Track (location) is very easy to determine, easy to verify, forecasts have high skill and high information content. Forecasts have high utility. Excellent candidate to extend lead time.

Intensity is readily determined, and can be meaningfully verified most of the time. Forecasts have little skill (relative to climatology and persistence), but have high information content. Forecasts have high utility. Decent candidate for extension.

Wind radii are difficult to determine, virtually impossible to verify quantitatively; 34 kt radii have no skill at extended leads and low information content. Forecasts have low utility due to the much larger track errors. Lousy candidate for extension.