verification of tropical cyclone forecasts
DESCRIPTION
Verification of Tropical Cyclone Forecasts. Beth Ebert (BOM) Barb Brown (NCAR) Laurie Wilson (RPN) Tony Eckel (ERT) 8 th TIGGE Working Group Meeting 22-24 February 2010, Geneva. What kind of TC forecasts?. Ensemble Track distribution strike prob., cone of uncertainty - PowerPoint PPT PresentationTRANSCRIPT
Verification of Tropical Cyclone Forecasts
Beth Ebert (BOM)
Barb Brown (NCAR)
Laurie Wilson (RPN)
Tony Eckel (ERT)
8th TIGGE Working Group Meeting
22-24 February 2010, Geneva
What kind of TC forecasts?
Deterministic• TC track
• Intensity – maximum wind– central pressure– temporal trend (rapid
intensification)
• Wind field / radii• Precipitation• Storm surge• Temporal consistency
Ensemble• Track distribution
– strike prob., cone of uncertainty
• Intensity distribution– mean / median– spread– 90th percentile
• Prob (wind > threshold)• Prob (precip > threshold)• Prob (surge > threshold)• Temporal consistency
Different users need different kinds of verification information
• Public and emergency managers– Simple, graphical– Focus on impact
• Forecasters– Information on how to interpret forecasts– Timing errors
• Modellers– Systematic errors
• How to improve the model
• How to improve ensemble distribution / spread
Quality of deterministic forecasts
• What are the track errors (along-track, cross-track)?
• What are the intensity errors?• Are temporal intensity trends correctly predicted?• What is the error in timing of landfall?• What is the error in forecast maximum wind
(rain)?– Multi-day total precipitation
• Is the spatial distribution of wind (rain) correct?
Quality of ensemble forecasts
• Does the ensemble enclose the observed track?• Are the ensemble probabilities skillful and reliable
predictions for – strike probability (track)– intensity (max wind, central pressure)– wind– precipitation
• Does the ensemble produce an appropriate spread for these variables?
Data issues
• Verification data– Data sources
• Best track, Dvorak, surface instruments, radar, …– Problems measuring in extreme conditions
• Forecast data– Size of ensemble– Model grid resolution– Details of cyclone tracker
• Reference forecasts– Statistical forecast (e.g., CLIPER)
Verification methods for deterministic TC forecasts
• Example 1 – visual comparison
Courtesy Noel Davidson, BOM
Analysis Track/intensity verificationModel
Verification methods for deterministic TC forecasts
• Example 2 – along-track and cross-track errors
Courtesy James Franklin, NHC
Verification methods for deterministic TC forecasts
• Example 3 – cumulative distribution of track errors
Courtesy James Franklin, NHC
Verification methods for deterministic TC forecasts
• Example 4 – distribution of intensity errors
HFIP High-Resolution Hurricane Test – DTC Final Report Sept 2009
Verification methods for deterministic TC forecasts
• Example 5 – rapid intensification
Count
Hours since onset of observed RI
Observed
High resolution model
Low resolution model
AOML / WRF – 69 cases
HFIP High-Resolution Hurricane Test – DTC Final Report Sept 2009
Verification methods for deterministic TC forecasts
• Example 6 – rain intensity distribution
Marchok et al. 2008
Verification methods for ensemble TC forecasts
• Example 1 – visual comparison
Verification methods for ensemble TC forecasts
• Example 2 – probabilistic scores and methods
MOGREPS 120 h forecast forstrike probability (within 120 km)
TC Anja, November 2009
fake
^
Verification methods for ensemble TC forecasts
• Example 3 – spread of track and intensity forecasts
20-member FIM ensemble
track intensitytrack intensity
New approaches for verifying TCs
• Spatial verification methods
Precipitation andwind fields
Storm characteristics • location• size• intensity• shape, etc.
New approaches for verifying ensemble TC forecasts
• Minimum spanning tree – multi-variate rank histogram
• Ensemble of object properties– ensemble mean object properties– distributions – use standard
methods for ensembles and probability forecasts
– relationship of TC genesis to the number of ensemble members predicting the TC at day 1+?
– correspondence ratio
observation
ensembleforecastensembleforecast
Reporting guidelines
• Provide all relevant information– Model(s), grid, range of dates, forecast lead times,
verification data source, etc.
• Aggregation and distribution of results
• Confidence intervals / uncertainty
Verification tools for TCs
• US Developmental Testbed Centre (DTC) – Tracker (Marchok)– Code for verification of TC track and intensity
• Other international tool sets??
Document on TC verification – commented literature review to be written this year
Contents
1. Introduction
2. Verification strategy
3. Reference data
4. Verification methods
5. Reporting guidelines
6. Summary
References
Appendices:
a. Brief description of scores
b. Guidelines for computing aggregate statistics
c. Confidence intervals for verification scores
d. Examples of graphical verification products