2006 NHC Verification Report

Interdepartmental Hurricane Conference

5 March 2007

James L. Franklin

NHC/TPC

Verification RulesVerification RulesSystem must be a tropical (or subtropical) cyclone at System must be a tropical (or subtropical) cyclone at both the forecast time and verification time, includes both the forecast time and verification time, includes depression stage (except as noted).depression stage (except as noted).Verification results are final (until we change something).Verification results are final (until we change something).Special advisories ignored; regular advisories verified.Special advisories ignored; regular advisories verified.Skill baselines for track is revised CLIPER5 (updated Skill baselines for track is revised CLIPER5 (updated developmental data to 1931-2004 [ATL] and 1949-2004 developmental data to 1931-2004 [ATL] and 1949-2004 [EPAC]), run post-storm on operational compute data.[EPAC]), run post-storm on operational compute data. Skill baseline for intensity is the new decay-SHIFOR5 Skill baseline for intensity is the new decay-SHIFOR5 model, run post-storm on operational compute data model, run post-storm on operational compute data (OCS5). Minimum D-SHIFOR5 forecast is 15 kt.(OCS5). Minimum D-SHIFOR5 forecast is 15 kt.New interpolated version of the GFDL: GHMI. Previous New interpolated version of the GFDL: GHMI. Previous GFDL intensity forecast is lagged 6 h as always, but the GFDL intensity forecast is lagged 6 h as always, but the offset is not applied at or beyond 30 h. Half the offset is offset is not applied at or beyond 30 h. Half the offset is applied at 24 h. Full offset applied at 6-18 h. ICON now applied at 24 h. Full offset applied at 6-18 h. ICON now uses GHMI.uses GHMI.

Decay SHIFOR5 ModelDecay SHIFOR5 Model

Begin by running regular Begin by running regular SHIFOR5 model.SHIFOR5 model.Apply the DeMaria module to Apply the DeMaria module to adjust intensity of tropical adjust intensity of tropical cyclones for decay over land. cyclones for decay over land. This includes recent This includes recent adjustments for less decay adjustments for less decay over skinny landmasses over skinny landmasses (estimates fraction of the (estimates fraction of the circulation over land).circulation over land).Algorithm requires a forecast Algorithm requires a forecast track. For a skill baseline, track. For a skill baseline, CLIPER5 track is used. (OFCI CLIPER5 track is used. (OFCI could be used if the intent was could be used if the intent was to provide guidance).to provide guidance).

2006 Atlantic Verification2006 Atlantic Verification

VT NT TRACK INTVT NT TRACK INT(h) (n mi) (kt)(h) (n mi) (kt)============================ ============================ 000 241 9.5 2.1000 241 9.5 2.1012 223 012 223 29.729.7 6.5 6.5 024 205 024 205 50.850.8 10.0 10.0036 187 036 187 71.971.9 12.4 12.4048 169 048 169 97.097.0 14.3 14.3072 132 072 132 148.7148.7 18.1 18.1096 100 205.5 19.6096 100 205.5 19.6120 78 265.3 120 78 265.3 19.019.0

Values in green meet or exceed all-time records.

* 48 h track error for TS and H only was 96.6 n mi.

Track Errors by Storm

Track Errors by Storm

2006 vs. 5-Year Mean

New 5-Year Mean

55 n mi/day

OFCL Error Distributions

Errors cut in half since 1990

Mixed Bag of Skill

2006 Track Guidance (Top Tier)

2nd Tier Early Models

2006 Late Models

Guidance Trends

Goerss Corrected Consensus

CCON 120 h FSP: 36%CGUN 120 h FSP: 33%

Small improvements of 1-3%, but benefit lost by 5 days.

FSU Superensemble vs Goerss Corrected Consensus

FSU Superensemble vs Other Consensus Models

2006 vs 5-Year Mean

No progress with intensity

Skill sinking faster than dry air over the Atlantic

Intensity Guidance

Dynamical Intensity Guidance Finally Surpasses Statistical Guidance

Intensity Error Distribution

When there are few rapid-intensifiers, OFCL forecasts have a substantial high bias.

GHMI had larger positive biases, but higher skill (i.e., smaller but one-sided errors).

FSU Superensemble vs Other Consensus Models

2006 East Pacific Verification2006 East Pacific VerificationVT N Trk IntVT N Trk Int

(h) (n mi) (kt)(h) (n mi) (kt)

================================================

000 379 8.8 1.7000 379 8.8 1.7

012 341 012 341 30.230.2 6.8 6.8

024 302 54.5 11.2024 302 54.5 11.2

036 264 77.4 14.6036 264 77.4 14.6

048 228 99.7 16.1048 228 99.7 16.1

072 159 142.3 17.8072 159 142.3 17.8

096 107 186.1 19.3096 107 186.1 19.3

120 71 227.5 120 71 227.5 18.318.3

Values in green represent all-time lows.

2006 vs 5-Year Mean

Errors cut by 1/3 since 1990

OFCL Error Distributions

Skill trend noisy but generally upward

2006 Track Guidance (1st tier)

Larger separation between dynamical and consensus models (model errors more random, less systematic).

FSU Superensemble vs Other Consensus Models

Relative Power of Multi-model Consensus

ne = 1.65 ne = 2.4

Same as it ever was…

…same as it ever was.

2006 Intensity Guidance

FSU Superensemble vs Other Consensus Models

SummarySummaryAtlantic Basin - TrackAtlantic Basin - Track OFCL track errors set records for accuracy from 12-OFCL track errors set records for accuracy from 12-

72 h. Mid-range skill appears to be trending upwards.72 h. Mid-range skill appears to be trending upwards. OFCL track forecasts were better than all the OFCL track forecasts were better than all the

dynamical guidance models, but trailed the dynamical guidance models, but trailed the consensus models slightly.consensus models slightly.

GFDL, GFS, and NOGAPS provided best dynamical GFDL, GFS, and NOGAPS provided best dynamical track guidance at various times. UKMET trailed track guidance at various times. UKMET trailed badly. No (early) dynamical model had skill at 5 days!badly. No (early) dynamical model had skill at 5 days!

ECMWF performed extremely well, when it was ECMWF performed extremely well, when it was available, especially at longer times. Small available, especially at longer times. Small improvement in arrival time would result in many more improvement in arrival time would result in many more EMXI forecasts.EMXI forecasts.

FSU super-ensemble not as good as Goerss FSU super-ensemble not as good as Goerss corrected consensus, and no better than GUNA in a corrected consensus, and no better than GUNA in a three-year sample.three-year sample.

Summary (2)Summary (2)

Atlantic Basin - IntensityAtlantic Basin - Intensity OFCL intensity errors were very close to the long-OFCL intensity errors were very close to the long-

term mean, but skill levels dropped very sharply (i.e., term mean, but skill levels dropped very sharply (i.e., even though Decay-SHIFOR errors were very low, even though Decay-SHIFOR errors were very low, OFCL errors did not decrease). The OFCL errors OFCL errors did not decrease). The OFCL errors also trailed the GFDL and ICON guidance.also trailed the GFDL and ICON guidance.

For the first time, dynamical intensity guidance beat For the first time, dynamical intensity guidance beat statistical guidance.statistical guidance.

OFCL forecasts had a substantial high bias. Even OFCL forecasts had a substantial high bias. Even though the GFDL had smaller errors than OFCL, its though the GFDL had smaller errors than OFCL, its bias was larger.bias was larger.

FSU super-ensemble no better than a simple average FSU super-ensemble no better than a simple average of GFDL and DSHP (three-year sample). of GFDL and DSHP (three-year sample).

Summary (3)Summary (3)

East Pacific Basin - TrackEast Pacific Basin - Track OFCL track errors up, skill down in 2006, OFCL track errors up, skill down in 2006,

although errors were slightly better than the although errors were slightly better than the long-term mean.long-term mean.

OFCL beat dynamical models, but not the OFCL beat dynamical models, but not the consensus models. consensus models. Much larger difference Much larger difference between dynamical models and the between dynamical models and the consensus in the EPAC (same as 2005).consensus in the EPAC (same as 2005).

FSU super-ensemble no better than GUNA FSU super-ensemble no better than GUNA (two-year sample).(two-year sample).

Summary (4)Summary (4)

East Pacific Basin - IntensityEast Pacific Basin - Intensity OFCL intensity errors/skill show little OFCL intensity errors/skill show little

improvement. improvement. GFDL beat DSHP after 36 h, but ICON GFDL beat DSHP after 36 h, but ICON

generally beat both.generally beat both. FSU super-ensemble slightly better than FSU super-ensemble slightly better than

ICON at 24-48 h, but worse than ICON after ICON at 24-48 h, but worse than ICON after that.that.