Use of Tropical Cyclone Wind Speed Probabilities in Public and Marine

Forecasts: An Update

Pablo SantosWFO Miami, FL

David Sharp

WFO Melbourne, FL

Mark DeMariaNOAA/NESDIS

NOAA Hurricane Conference 2008PS/DS

“HURRICANE CONDITIONS EXPECTED.”

2

Contents of Presentation

• Brief Background Review

• Methodology

• 2008 Season Cases

• Verification of Incremental Probabilities– The probability of sustained winds reaching or exceeding the specified

threshold during the 12 hour forecast period ending at the specified forecast hour.

• Future Plans

3

Text Formatters: Logic Hierarchy INTENT (Item 42-05): to establish a set of rules for triggering enhanced wording which

responsibly conveys wind speed uncertainties within text forecasts

Priority Sensitivity Grid Sets

1. Official Warnings,

Official Watches

Hazard Grids

2. Official Forecast Wind Grids

3. Forecast Error

Incremental Probabilities

64-knot Probability Grids,

34-knot Probability Grids

Note: Forecast Error implies appreciation for the inherent uncertainties regarding the track, intensity, and size of the tropical cyclone.

Message Sensitivity(Fostering One Forecast Message)

4

(Text Formatters) • HURRICANE CONDITIONS *• HURRICANE CONDITIONS EXPECTED• HURRICANE CONDITIONS POSSIBLE

• TROPICAL STORM CONDITIONS *• TROPICAL STORM CONDITIONS * WITH HURRICANE CONDITIONS POSSIBLE• TROPICAL STORM CONDITIONS EXPECTED• TROPICAL STORM CONDITIONS EXPECTED WITH HURRICANE CONDITIONS POSSIBLE• TROPICAL STORM CONDITIONS POSSIBLE• TROPICAL STORM CONDITIONS POSSIBLE WITH HURRICANE CONDITIONS ALSO POSSIBLE

Note:• The phrases with * above are equivalent to imminent or ongoing conditions as reflected in the 00-12 hour period simultaneously by the hazard, wind, and probability grids. • The word EXPECTED is used in situations mainly during the Warning period, but is also selectively used in the Watch period (e.g., Day 1 & 2, or out to period 4).• The word POSSIBLE is used during the extended period (e.g., Day 3 - 5), but is also used in specific situations within Watch/Warning period. • Also, certain situations require compound phrases (e.g., Tropical Storm Warning with a Hurricane Watch).

Baseline Phraseology

5

Methodology

• Load Wind Grids with TCMWindTool and take care of any local editing issues to address background blending (gradient), wind gradients within the coastal zone, other local value-added adjustments, etc.

• Next, compute Wind Gust Grids (use GFE SmartTools).

• Then populate the Incremental Probability Grids (PWS_Procedure).

• If warranted, be sure to populate any Hazard Grids by using a combination of PlotTPCEvents (TCV) procedure and Make Hazard Tools.

• Finally, publish your grids and then run the ZFP and CWF text formatters. QC and post-edit before sending. Publishing the grids automatically activates the point and click version too.

2008 Season ExamplesFeedback

7

Fay – WFOs MFL and MLB

8

Ike – WFOs BRO/CRP/HGX/LCH

9

2008 Summary of Issues

• TPCWindSpeed Probabilities mapping problem. DR 20206.• Sites not updating probabilities every 6 hours (PWS_Procedure) or

running the procedure too early.• Timeliness of Probability files (Around 20 minutes after advisory time).• Internet communication issues with grids and Internet support.• Forecasters not applying procedure with every advisory resulting in

packages that employed the probabilistic wording and some that did not.

• Forecasters freelancing with post editing outside the bounds of what the training discussed they should be doing.

• Impression from field forecasters that Incremental Probability thresholds for hurricane conditions might be low.

• Otherwise, feedback from field offices very positive in general. One field survey by WFO Brownsville showed 8 out of 11 respondents specifically used the terms of uncertainty from our forecast products.

10

Special Probability Verification for WFO Application

• Rerun 2008 MC model for all cases 2004-2008 with continental U.S. hurricane warnings – Include runs up to 3 days prior to 1st warning

• 401 runs from 19 storms• Calculate 12 hr incremental probabilities at 343

coastal breakpoints locations • Use standard probabilistic skill metrics to

optimize probability thresholds for WFO application – Threat score and ROC diagrams

• Preliminary results for 64 kt probabilities

11

Pt Optimization Procedures

• Threat Score– Threat Score useful for low probability events since it

does not include No-No cases– TS ranges from 0 to 1 (1 is best)– Pick Pt that maximizes TS

• Hit Rate/False Alarm Rate– Plot HR vs. FR in 2-D plane for range of Pt

• Relative Operating Characteristic (ROC) diagram

– Pick Pt for point closest to upper left corner in ROC diagram

• Upper left corner has Hit Rate=1, False Alarm Rate=0

12

Optimal 64 kt Probability Thresholds

based on Threat Score and ROC Diagram 64 Knots Incremental Probability Verification

0

5

10

15

20

25

30

35

40

0-1

2 h

r

12

-24

hr

24

-36

hr

36

-48

hr

48

-60

hr

60

-72

hr

72

-84

hr

84

-96

hr

96

-10

8 h

r

10

8-1

20

hr

Time Interval (hr)

Op

tim

al P

rob

ab

ility

Th

res

ho

ld

(%)

Threat Score-based

ROC Diagram-based

Hist-Based

13

Future - 2009 Season

• Every site will have OB9. In OB9 formatters were re-designed for better integration with baseline ZFP and CWF formatters. In order for this last base lining step to be successful, FOCAL POINTS MUST follow the procedures outlined here as a post install after they upgrade to OB9:

• http://www.nws.noaa.gov/os/vtec/GFEOB9Changes.html#Trop

• So the question is where do we go from here?

14

Future – 2009 Season

• Complete Incremental Probabilities verification for 34 knots and verify using other metrics. Adjust thresholds currently used accordingly and run tests with old and new thresholds.

• Propose going official with this capability with ZFP/CWF and point and click pages in 2009. This will require:

– WSH look into status of DR 20206.– Coordinate any needed changes to the public and marine directives with those

programs to support transition to official. Preliminary conference call on this topic was held prior to even the 2007 season.

– Conduct formal training with all coastal offices and associated inland offices using modules already developed by Dave and Pablo to support transition.

– Provision for proper support for point and click in case something goes wrong. If this capability is adopted officially this will need to be communicated to Bungee’s group.

• Expand capability to tabular products, specifically, the PFM. Enlist help of Tracy Hansen if needed to more effectively accomplish this. I believe this is doable for next season.

• Propose Wind Team reconvene to discuss all these.

Supplemental

16

Gustav – WFO LIX

Zones

17

Conversion of Probabilities into Yes-No Forecast

• Pick probability threshold (Pt)

• If P ≥ Pt, forecast YES

• If P < Pt, forecast NO

• Best choice of Pt depends on particular application

• Use appropriate validation metric to select Pt objectively

• Metrics determined from contingency table

18

Contingency Table for Validation of Yes-No Forecasts

ObservedYes No

Yes

Forecast

No

a b

c d

Threat Score (TS) = a/(a + b + c)Hit Rate (HR) = a/(a + c)False Alarm Rate (FR) = b/(b + d)

19

Storm Cases For 401 MC Model Runs

0

5

10

15

20

25

30

35

Alex 2

004

Charle

y 20

04

Franc

es 2

004

Gasto

n 20

04

Ivan

2004

Jean

ne 2

004

Arlene

200

5

Denni

s 200

5

Emily

200

5

Katrin

a 20

05

Ophel

ia 20

05

Rita 2

005

Wilm

a 20

05

Alberto

200

6

Humbe

rto 2

007

Dolly 2

008

Fay 2

008

Gusta

v 200

8

Ike 2

008

Storm Name

No

. M

C F

ore

cast

s

20

Threat Score vs. Probability Threshold

24-36 hr Incremental Probabilities

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0 10 20 30 40 50 60 70 80 90 100

Probability Threshold (%)

Th

reat

Sco

re

Optimal Probability Threshold = 19%

21

ROC Diagram for 24-36 hr Incremental Probabilities

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

False Alarm Rate

Hit

Rat

e

Optimal Probability Threshold Pt = 4%

Pt=0%Pt=1%

Pt=100%