ensemble forecasting
DESCRIPTION
Ensemble Forecasting. Yuejian Zhu Environmental Modeling Center December 6 th 2011 Acknowledgment for: Members of Ensemble & Probabilistic Guidance Team. Outlines. Responsibility of ensemble team Include all ensemble systems Current available data and products - PowerPoint PPT PresentationTRANSCRIPT
1
Ensemble Forecasting
Yuejian ZhuEnvironmental Modeling Center
December 6th 2011
Acknowledgment for:Members of Ensemble & Probabilistic Guidance Team
2
Outlines• Responsibility of ensemble team
– Include all ensemble systems• Current available data and products
– Data access through all available resource– Digital probabilistic products– Web-based products
• Implementations for past year– Include all major and minor – Update information from other centers
• On going implementations– Include all major and minor– Reforecasting for future ensemble post process
• Future plans– Development of major system– Post process and calibration– How to satisfy user requests?
3
Responsibilities of Ensemble Team- Assess, model, communicate uncertainty in numerical forecasts
• Present uncertainty in numerical forecasting– Tasks
• Design, implement, maintain, and continuously improve ensemble systems– Topics
• Initial value related uncertainty• Model related forecast uncertainty
– Ensemble systems• Global – GEFS / NAEFS / NUOPC• Regional – SREF / HREF / NARRE-TL / HWAF ensemble• Climate – Contributions to future coupling CFS configuration• NAEFS/GEFS downscaled• Ocean wave ensemble (MMA/EMC)
• Statistical correction of ensemble forecasts– Tasks
• Correct for systematic errors on model grid• Downscale information to fine resolution grid (NDFD)• Combine all forecast info into single ensemble/probabilistic guidance
• Probabilistic product generation / user applications– Contribute to design of probabilistic products– Support use of ensembles by
• Internal users (NCEP Service Center, WFOs, OHD/RFC forecasters and et al.)• External users (research, development, and applications)
4
NAEFS Products DistributionSystem Current available products
Config. 1.deg 0-384h, every 6 hours, 20 members (NCEP) and 20 members (CMC), ens. control (NCEP and CMC)
Format GRIB1 (and GRIB2, GIF images for web display)
CCSNCEP: pgrba, pgrbb, pgrba_bc, pgrba_an, pgrba_wt, ensstat, ndgdCMC: pgrba, pgrba_bc, pgrba_an, pgrba_wt, ensstatNAEFS: ndgd, pgrba_an, pgrba_bc
NCEPFTPPRD
ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/gens/prod cd gefs.${yyyymmdd} for NCEP ensemble1. pgrb2a (00, 06, 12 and 18UTC) (1.0 degree, all lead times, 1(c) + 20 (p))2. pgrb2alr (00, 06, 12 and 18UTC (2.5 degree, all lead times, 1(c) +20 (p))2. pgrb2b (00, 06, 12 and 18UTC) (1.0 degree, all lead times, 1(c) + 20 (p))4. pgrb2blr (00 and 12UTC) (2.5 degree, all lead times, 1(c) + 20 (p))5. ensstat (00UTC) (prcp_bc, pqpf and pqpf_bc files)6. wafs (00 and 12UTC)7. ndgd_gb2 (00, 06, 12, 18UTC) (CONUS-5km, all lead times and all probability forecasts)ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/gens/prod cd cmce.${yyyymmdd} for CMC ensemble1. pgrba (00 and 12UTC) (1.0 degree, all lead times,1 control + 20 members)ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/gens/prod cd naefs.${yyyymmdd} for NAEFS products1. pgrb2a_an (00, 12UTC) (1.0 degree, all lead times, anomaly for ensemble mean)2. pgrb2a_bc (00,12UTC) (1.0 degree, all lead times, probabilistic forecasts)3. ndgd_gb2 (00,12UTC) (CONUS-5km, all lead times, probabilistic forecasts)
TOC
ftp://tgftp.nws.noaa.gov/SL.us008001/ST.opnl/ cd MT.ensg_CY.${cyc}/RD.${yyyymmdd} for NCEP only1. PT.grid_DF.gr1_RE.high (00 and 12UTC) (Pgrba: 1.0 and 2.5 degree, 0-384 hrs, c + 10 (p))2. PT.grid_DF.gr1_RE.low (00 and 12UTC) (Pgrbb: 1.0 degree, 0-84 hrs, 2.5 d, 90-384 hrs, c + 10 (p))3. PT_grid_DF.bb
NOMADShttp://nomad5.ncep.noaa.gov/ncep_data/ for ftp: combined pgrba and pgrbb at 1 degree resolution, for all ensemble members (c+20(p)) and all lead time (0-384 hours)http://nomad5.ncep.noaa.gov/pub/gens/archive/ for http: combined pgrba and pgrbb at 1 degree resolution
Web-based Probabilistic Products• NCEP web-site
– NCO supported all ensemble probabilistic products (main page)– EMC (experimental) developed probabilistic products
• PQPF for various thresholds (include precipitation types)• RMOP for 500hPa height• Tropical storm track forecast• Extra-tropical storm track forecast (global tracking)
• NAEFS web-site– CPC developed extended probabilistic forecast
• Temperature, 500hPa height and precipitation– NCO supported NAEFS products
• Spaghetti, – EMC (experimental) NAEFS products
• PQPF for various thresholds• Anomaly forecast
– CMC supported NAEFS products• Metagrams for NA major cities (example)
• NUOPC web-site– EMC (experimental) NUOPC products
• PQPF for various thresholds (include precipitation types)• TS track forecast for multi-model ensembles
– FNMOC’s NUE probabilistic products (example) 5
Implementations for past year• Alaska downscaling - Dec. 7th 2010
– 6km probabilistic guidance for Alaska region (improvement for Max/Min temperature, wind speed/direction)
• NUOPC – IOC Jan. 18 2011– Getting FNMOC ensemble data (both raw and bias corrected) in NCEP and
NOMADS for public– Ceremony of NUOPC IOC
• NAEFS products upgrade – March 1st 2011– Adding more variables for exchange, more variables for bias correction
• 5th Ensemble User Workshop – May 9-11 2011 – In Laurel Maryland (highlight)
• NAEFS data upgrade – May 24th 2011– Receive CMC’s grib2 bias corrected forecast directly;
• CCPA upgrade – July 26th 2011– Adding 3-hrly analysis for regional application
• CMC’s GEFS upgrade – August 17th 2011 (highlight)• FNMOC’s GEFS upgrade – September 14th 2011 (highlight, stats)• NMME (August) and IMME (December) implementation in CPC • CMC’s GEFS data on NOMADS – November 2011
6
On going implementations• Major GEFS upgrade – Jan. 2012
– Highlight the changes– What do we expect from this change?
• Overall skills, TC tracks, bias correction• More stats: http://www.emc.ncep.noaa.gov/gmb/yzhu/html/imp/201109_imp.html
• NAEFS product upgrade – April. 2012– Extend variables for CONUS downscaling
• Temperature, winds, humidity and dew point– Precipitation calibrations (CONUS, RFC)– Anomaly forecast from CFSRR climatology
• 6th NAEFS workshop – May 1-3 2012– At Monterey CA – FNMOC will be a localhost
• GEFS reforecast – August 2010 –– Leading by Tom Hamill (ESRL)– Bias correction for TS forecast
• CCPA update– Use more historical data for calibration
• Additional supports– CSTAR (the Collaborative Science, Technology, and Applied
Research) program - ensemble sensitivity analysis7
Future plans• GEFS
– Plan for FY2013-2015 (EMC-ESRL collaboration)• NAEFS
– Increasing products resolution and output frequencies– Improving calibration method– Apply GEFS reforecast for calibration– EMC-MDL collaboration for high moment adjustment– EMC-OHD collaboration for improving temperature and precipitation uncertainty forecast
• NUOPC – adding FNMOC ensemble to NAEFS– Project information and highlights– Evaluation metrics– Tentative Schedule
• Extended range forecast– Coupling ensemble system– Extended to 45 days to cover week-3 and week-4
• Seamless forecast system– Exchange extended forecast data with CMC– Support IMME and NMME to improve MJO prediction
• User requests– Mean sea level pressure – more useful for users– Relative humidity or surface dew-point – more useful for users– Anomaly forecast – extreme weather index– Clustering (questionable for single model ensemble?) 8
9
http://mag.ncep.noaa.gov/NCOMAGWEB/appcontroller
Global ensemble
Regional ensemble
NAEFS
Return
10
http://www.cpc.ncep.noaa.gov/products/predictions/short_range/NAEFS/Outlook_D264.00.php
Example of temperature forecast
Upper tercile
Lower tercile Return
Map of PQPF and Precipitation Types: every 6 hours, 4 different thresholds
Return
Example: The tracker puts out 4 time a day for all cyclones (Northern Hemisphere)
Return
13
NAEFS products – Metagram (examples)
Return
Return
15
Application for Alaska region and HPC Alaska desk
Max Temp
Solid – RMS error
Dash - spread
CRPS – small is better
Max Temp
10-m wind speed
10-m U
Bias (absolute value)
Bias (absolute value)Return
16
5th NCEP Ensemble User Workshop• Logistics
– Workshop organized by EMC/NCEP and DTC/NCAR (co-organizer)– May 10-12 2011, Laurel, MD, 90+ participants
• NWS Regions (6), Headquarters (17), NCEP (44)• OAR (5), other government agencies (4), private (2), academic (5) & international (11)
– For further info, see: http://www.dtcenter.org/events/workshops11/det_11/ • Main Theme
– How to support NWS in its transition from single value to probabilistic forecasting• Goal is to convey forecast uncertainty in user relevant form
• 46 presentations– Covering all ensemble forecast systems
• SREF, GEFS/NAEFS, Wave ensemble, CFS and NMME
– Reports from NCEP Service Centres and Regions (WFOs)• E.g., first numerical ensemble-based 2-day tornado, week 3-4, monthly MJO outlook
• Working groups– Ensemble configurations - Ensemble forecasting– Statistic post processing - Reforecast/hindcast generation– Probabilistic product generation - Forecaster’s role and training– Ensemble data depository / access - Database interrogation /
forecaster tools• Outcome / Recommendations
– Prepared report for NWS roadmap reference• Plan for immediate steps (interim solution to be implemented in 2-3 years)• Outline for long term solution and resource requirements (5-10 years)
– All activities to be coordinated under NWS Forecast Uncertainty Program (NFUSE)
Return
CMC’s GEFS Implementation
• Modification to EnKF analysis configuration– Use 192 ensemble members instead of 96– Has more satellite data
• Upgrade GEM version– Use 4.2 version (vertical staggering) instead of
version 3.0• Increase model top to 2 hPa from 10 hPa• Resolutions
– Horizontal: 600x300 (66km) from 400x200 (100km)
– Vertical: 40 levels from 28 levels
18Return
FNOMC’s GEFS Implementation and Plan
• 9 latitude band Ensemble Transform initialization instead of 5-banded
• T159L42 instead of T119L30 in horizontal and vertical resolutions
• Plan for T239L42 in operations for June 2012
• Implement the bias correction for selected variables (NAEFS algorithm)
• Implement forecast vs observation verification system
• NUOPC (FNMOC+NCEP+CMC) productsReturn
Tropical Cyclone Track Error T119, T159, T239
0
100
200
300
400
500
600
12 24 36 48 72 96 120Forecast Hour
Err
or
(km
)
CTL
G119
G159
G239
# fcsts: 356 320 281 245 182 129 86
Homogenous NHTC track forecast error (km), for G119, G159, and G239 ensemble mean tracks as denoted in key. Also shown is the average forecast error of the T239L30 NOGAPS operational deterministic forecast (CTL). The numbers of verifying forecasts are shown below the x axis. The differences between G119 and G159 are statistically significant at the 95% level out to 96 h. The differences between G159 and G239 are statistically significant at 48 and 72 h. All used global ET.(Fig 3 from Impact of Resolution and Design on the U.S. Navy Global EnsemblePerformance in the Tropics, Reynolds, et al., MWR, July 2011, p 2145-2155.) Return
Proposal Changes• Model and initialization
– Using GFS V9.01 (current operational GFS) instead of GFS V8.00– Improved Ensemble Transform with Rescaling (ETR) initialization– Improved Stochastic Total Tendency Perturbation (STTP)
• Configurations– T254 (55km) horizontal resolution for 0-192 hours (from T190 – 70km)– T190 (70km horizontal resolution for 192-384 hours (same as current opr)– L42 vertical levels for 0-384 hours (from L28)
• Add Sunshine duration for TIGGE data exchange• Part of products will be delayed by approximately 20 minutes
– Due to limit CCS resources– 40-42 nodes for 70 minutes (start +4:35 end: +5:45)
• Unchanged:– 20+1 members per cycle, 4 cycles per day– pgrb file output at 1*1 degree every 6 hours– GEFS and NAEFS post process output data format
• Why do we make this configurations?– Considering the limited resources and resolution makes difference
• What do we expect from this implementation?– Improve general probabilistic forecast skill overall– Significant improvement of tropical storm tracks (especially for Atlantic basin)
21
Return
Winter 2 months
Skillful line10.25d
11.00d
NH 500hPa height
SH 500hPa height
NH 850hPa temperature SH 850hPa temperature
Anomaly Correlation
GFS V8.0 .vs V9.0
22
0
50
100
150
200
250
0 12 24 36 48 72 96 120
GEFSo GEFSx GFS
Forecast hours
Atlantic, AL01~19 (06/01~11/30/2011)
#CASES 309 279 251 227 202 162 125 88
GEFSo---GEFS T190 (operational run) GEFSx---GEFS T254 (parallel run)GFS ------GFS T574 (operational run)
Tra
ck e
rror
(NM
)
11%
12%
22%
20%Improvement
Return
GEFSx runs once per day before Oct.
24
CRPS for NH 850hPa Temperature
CRPS for NH 10m U-wind
CRPS for NH 2-meter TemperatureCRPS for NH 10m V-wind
Tmax Tmin
Temperature
Latest evaluation for CONUS temperature forecast by apply :1.Bias correction at 1*1 degree for NCEP GFS/GEFS, CMC/GEFS2.Bybrid bias corrected NCEP GFS and GEFS3.Apply statistical downscaling for all bias corrected forecast4.Combined all forecasts at 5*5 km (NDGD) grid with adjustment - NAEFS
CRPS CRPS
CRPS
25Return
CRPS CRPS
CRPSCRPS
U 10mV 10m
Wind speed
Wind direction
26Return
Dew point T RH
RMS & SpreadRMS & Spread
Dew Point T RH
CRPSCRPS
27Return
28
Precipitation calibration for 2009-2010 winter season (CONUS only)
Comparison for GFS and ensemble control (raw and bias corrected)
ETS for 0-6hr fcst
BIAS for 0-6hr fcst
ETS for all lead-time
BIAS for all lead-time
Perfect bias = 1.0
The probabilistic scores (CRPS -not show here) is much improved as well. We are still working on the different weights, different RFC regions, downscaledto 5km as well. More results will come in soon. Plan for implementation: Q4FY11
Courtesy of Yan LuoReturn
Significantly reduced bias for CONUS and each RFC
Bias=1.0
•More effective on lower amount precip
•Consistently effective along with leading time
NWRFCMBRFC
CNRFCCBRFC
ABRFC
WGRFC
NCRFC
OHRFC
LMRFC
NERFC
MARFC
SERFC
1 * 1 deg
1 * 1 deg 1 * 1 deg
CONUS
CBRFC OHRFC
before bias correction before bias correction
after bias correction after bias correction
30
Experimental maps to support CSTAR program for winter season
31
GEFS Implementation Plan for FY13-15• Hybrid data assimilation based GEFS initializations
– Using 6-hr EnKF forecast to combine improved ETR (without cycling) (Schematic diagram)
– Improving ETR (still in discussion and investigation)• Adaptive modification of initial and stochastic model perturbation variances • Based on recursive average monitoring of forecast errors and ensemble spread• Avoid having to tune perturbation size after each analysis/model/ensemble
changes• Improving performance and easy maintenance
• Real time generation of hind-casts (pending on resource)– Make control forecast once every ~5th day (6 runs for each cycle)
• T254L42 (0-192) and T190L42 (192-384), and use new reanalysis (~30y)– Increasing sample of analysis – forecast pairs for statistic corrections– Improving bias correction beyond 5-d– Potential for regime/situation dependent bias correction
• Coupled ocean-land-atmosphere ensemble– Couple MOM4/HYCOM with land-atmosphere component using ESMF
• Depending on skill, extend integration to 35 days• Merge forecasts with CFS ensemble for seamless weather climate interface• Land perturbation and surface perturbation (later)
– Explore predictability in intra-seasonal time scale– Potential skill beyond 15 days
• Hydro-meteorological (river flow) ensemble forecasting– Pending on operational LDAS/GLDAS, and RFC application Return
Development of Statistical Post-Processing for NAEFS
• Opportunities for improving the post-processor – Utilization of additional input information
• More ensemble, high resolution control forecasts (hybrid?)• Using reforecast information to improve week-2 forecast and precipitation• Analysis field (such as RTMA and etc..)
– Improving calibration technique• Calibration of higher moments (especially spread)• Use of objective weighting in input fields combination• Processing of additional variables with non-Gaussian distribution
– Improve downscaling methods
Future Configuration of EMC Ensemble Post-Processor
Return
Project Information and Highlights• Evaluate the value added for current NAEFS
inclusion of FNMOC ensembles– Current NAEFS products attached
• Period: December 1st 2011 – May 31st 2012– Cover winter and spring seasons
• Available data for each participating centers– NCEP
• Raw and bias corrected NCEP, CMC and FNMOC ensembles– CMC
• Raw and bias corrected NCEP, CMC and FNMOC ensembles– FNMOC
• Raw NCEP, CMC and FNMOC ensemble data only– AFWA
• Raw NCEP, CMC and FNMOC ensemble data onlyReturn
Evaluation metrics • NUOPC evaluation metrics
– RMSE (MAE) and spread of ensemble mean, CRPS, Brier score for selected thresholds
– Targeting evaluation parameters:• 2m T, 10m winds, 250hPa winds, 700hPa RH, 500hPa Z (targeting for
next NCEP implementation)• TS tracks, precipitation (potential for future consideration)• Significant wave high, total cloud cover
– NCEP is expected to have most evaluations against own analysis (GSI and RTMA) and observations (to connect with NCEP users)
• NAEFS evaluation metrics– RMSE and spread of ensemble mean, CRPS (resolution and
reliability) and etc…– Targeting evaluation parameters:
• 2m T, 10m winds, 250 and 850 winds, 500hPa, 1000hPa Z, 850 T. (Targeting for next NCEP implementation)
• Total precipitation (raw)Return
Scheduling• Current – November 30: preparation• December 1st 2011 – May 31st 2012: Data collection and
evaluation for all participate centers– Mid-term performance review through NAEFS and UEO meetings– Another performance review by NAEFS workshop (May 1-3,
Monterey CA) • Late of June
– Fully evaluations from all participated centers– Possibility to have one day meeting at Silver Springs/Camp Springs
or NUOPC workshop in June (???)• Mid of July
– Decision will be made to recommend for NCEP implementation (or not)
• July 1st – deadline for EMC RFCs for implementation• August – September 2012: NCO test and real time parallel for
NCEP users evaluations (see additional slide for details)• September 25 2012: targeting for implementation Return
36
week-1 week-2 one month
GEFS/NAEFSservice CFS
service
Weather/Climate linkage
NCEP/GEFS will plan for T254L42 (2011 GFS version) resolution with tuned ETR initial perturbations and adjusted STTP schemefor 21 ensemble members, forecast out to 16 days and 4 cycles per day. Extended to 45 days at T126L28/42 resolution, 00UTC only (coupling is still a issue?)NAEFS will include FNMOC ensemble in 2011, with improving post process which includebias correction, dual resolution and down scaling
Main eventMJO
Future seamless forecast system
Operational CFS has been implemented inQ2FY2011 with T126L64 atmospheric modelresolution (CFSv2, 2010version) which is fully coupled with land, ocean and atmosphere(GFS+MOM4+NOAH), 4 members per day (using CFS reanalysis as initial conditions, one day older?), integrate out to 9 months.
Future: initial perturbed CFS
Main products:1. Probabilistic forecasts for every 6-hr
out to 16 days, 4 times per day: 10%, 50%, 90%, ensemble mean, mode and spread.
2. D6-10, week-2 temperature and precipitation probabilistic mean forecasts for above, below normal and normal forecast
3. MJO forecast (week 3 & 4 … )
Main products:
ENSO predictions???Seasonal forecast???
SEAMLESS
Ensemble fcst (1)t=j-1 j
Ensemble fcstt=j, j+1
EnKF assimilation
t=j
EnKFassimilation
t=j+1
GSI/3DVARt=j
GSI/3DVARt=j+1
Estimated Background Error Covariance from
Ensemble Forecast(6 hours)
Estimated Background Error Covariance from
Ensemble Forecast(6 hours)
HybridAnalysis?
Replace Ensemble Mean
Flow Chart for Hybrid Variation and Ensemble Data Assimilation System (HVEDAS) - concept
Lower resolution
Higher resolution
Two-wayhybrid
Ensemble fcst (2)t=j 16 days
Ensemble initialization
37
38
BACKGROUND
0
50
100
150
200
250
300
0 12 24 36 48 72 96 120
GEFSo GEFSx GFS
Forecast hours
Atlantic, AL01~17 (06/01~09/30/2011)
#CASES 235 213 194 178 159 133 103 75
GEFSo --- GEFS T190 (operational run) GEFSx --- GEFS T254 (parallel run)GFS ------ GFS T574 (operational run)
Tra
ck e
rro
r(N
M)
17%
13%
26%
24%Improvement
40
Variables Domains Resolutions Total 4/8
Surface Pressure CONUS/Alaska 5km/6km 1/1
2-m temperature CONUS/Alaska 5km/6km 1/1
10-m U component CONUS/Alaska 5km/6km 1/1
10-m V component CONUS/Alaska 5km/6km 1/1
2-m maximum T Alaska 6km 0/1
2-m minimum T Alaska 6km 0/1
10-m wind speed Alaska 6km 0/1
10-m wind direction Alaska 6km 0/1
Note: Alaska products is in real time parallel
Expect implementation: Q1 FY2011
NAEFS downscaling parameters and productsLast update: May 1st 2010
(NDGD resolutions)
All products at 1*1 (lat/lon) degree globallyEnsemble mean, spread, 10%, 50%, 90% and mode
back
41
Variables pgrba_bc file Total 49 (14)
GHT 10, 50, 100, 200, 250, 500, 700, 850, 925, 1000hPa 10 (3)
TMP 2m, 2mMax, 2mMin, 10, 50, 100, 200, 250, 500, 700, 850, 925, 1000hPa
13 (3)
UGRD 10m, 10, 50, 100, 200, 250, 500, 700, 850, 925, 1000hPa 11 (3)
VGRD 10m, 10, 50, 100, 200, 250, 500, 700, 850, 925, 1000hPa 11 (3)
VVEL 850hPa 1(1)
PRES Surface, PRMSL 2(0)
FLUX (top) ULWRF (toa - OLR) 1 (1)
14 new vars
Notes
NEXT NAEFS pgrba_bc files(bias correction)
back
42
0
10
20
30
40
50
60
70
80
90
100
24 48 72 96 120 144 168 192 216 240 264 288 312 336 360 384
Lead time (hours)
Prob
abilit
ies (p
erce
nt)
P10-expect P10-ncepraw P10-naefs
P90-expect P90-ncepraw P90-naefs
2-m temp 10/90 probability forecast verificationNorthern Hem, period of Dec. 2007 – Feb. 2008
3-month verifications
Top: 2-m temperature probabilisticforecast (10% and 90%) verificationred: perfect, blue: raw, green: NAEFS
Left: example of probabilistic forecasts(meteogram) for Washington DC, every6-hr out to 16 days from 2008042300
90%
10%
back
43
Critical Event: sfc winds > 50kt
Cost (of protecting): $150K
Loss (if damage ): $1M
Hit
FalseAlarm
Miss
CorrectRejection
YES NO
YES
NO
Forecast?
Obs
erve
d?
Decision Theory Example
Deterministic Observation ProbabilisticCase Forecast (kt) (kt) Cost ($K) Forecast 0% 20% 40% 60% 80% 100%
1 65 54 150 42% 150 150 150 1000 1000 10002 58 63 150 71% 150 150 150 150 1000 10003 73 57 150 95% 150 150 150 150 150 10004 55 37 150 13% 150 0 0 0 0 05 39 31 0 3% 150 0 0 0 0 06 31 55 1000 36% 150 150 1000 1000 1000 10007 62 71 150 85% 150 150 150 150 150 10008 53 42 150 22% 150 150 0 0 0 09 21 27 0 51% 150 150 150 0 0 0
10 52 39 150 77% 150 150 150 150 0 0Total Cost: 2,050$ 1,500$ 1,200$ 1,900$ 2,600$ 3,300$ 5,000$
$150K $1000K
$150K $0K
Deterministic Observation ProbabilisticCase Forecast (kt) (kt) Cost ($K) Forecast 0% 20% 40% 60% 80% 100%
1 65 54 150 42% 150 150 150 1000 1000 10002 58 63 150 71% 150 150 150 150 1000 10003 73 57 150 95% 150 150 150 150 150 10004 55 37 150 13% 150 0 0 0 0 05 39 31 0 3% 150 0 0 0 0 06 31 55 1000 36% 150 150 1000 1000 1000 10007 62 71 150 85% 150 150 150 150 150 10008 53 42 150 22% 150 150 0 0 0 09 21 27 0 51% 150 150 150 0 0 0
10 52 39 150 77% 150 150 150 150 0 0Total Cost: 2,050$ 1,500$ 1,200$ 1,900$ 2,600$ 3,300$ 5,000$
Cost ($K) by Threshold for Protective Action
Optimal Threshold = 15%back
44
Overall temperature forecasts: Average over past 30 days: (20080929-20081028) MAE Bias >10 err <3 err off. rank Best G. 2nd G. Worst G. 1 12-hr 2.44 0.7 0.1% 67.3% 1 out of 7 NAM40 65.4% NAM12 60.1% NGM80 44.4% 2 24-hr 2.84 1.0 0.3% 59.1% 2 out of 7 NAM40 60.3% NAM12 56.9% SREF 47.0% 3 36-hr 2.94 0.8 0.3% 57.8% 1 out of 7 NAM40 55.9% NAM12 52.6% NGM80 44.0% 4 48-hr 3.36 1.6 2.1% 52.8% 1 out of 7 MOSGd 48.9% NAM40 48.3% NGM80 12.9% 5 60-hr 3.26 1.0 1.7% 54.8% 1 out of 6 MOSGd 50.1% NAM12 48.8% NAM40 6.2% 6 72-hr 3.35 1.3 2.1% 53.1% 1 out of 5 MOSGd 49.9% NAM12 49.5% SREF 44.0% 7 84-hr 3.80 0.6 4.7% 49.0% 1 out of 5 NAEFS 48.6% SREF 44.5% NAM12 2.6% 8 96-hr 3.96 0.7 4.0% 44.4% 2 out of 4 NAEFS 46.2% HPCGd 42.6% MOSGd 40.6% 9 108-hr 4.43 0.9 5.5% 38.5% 2 out of 3 NAEFS 41.7% MOSGd 37.7% MOSGd 37.7%10 120-hr 4.57 1.0 5.9% 36.6% 2 out of 4 NAEFS 40.9% HPCGd 36.5% MOSGd 36.3%11 132-hr 4.83 0.7 7.8% 34.7% 1 out of 3 NAEFS 34.5% MOSGd 34.4% MOSGd 34.4%12 144-hr 4.83 0.5 7.4% 34.7% 3 out of 4 HPCGd 36.4% NAEFS 35.5% MOSGd 33.3%13 156-hr 5.43 0.1 11.9% 30.3% 3 out of 3 NAEFS 32.1% MOSGd 30.8% MOSGd 30.8%14 168-hr 5.74 0.3 14.4% 27.7% 2 out of 4 HPCGd 27.7% MOSGd 26.9% NAEFS 26.1
Minimum temperature forecast: Average over past 30 days: (20080929-20081028) 1 12-hr 3.17 -1.2 1.0% 53.4% 3 out of 7 NAEFS 59.7% SREF 57.1% NGM80 21.8% 2 24-hr 3.03 -0.9 0.6% 55.5% 2 out of 7 SREF 57.2% NAEFS 54.2% NGM80 24.9% 3 36-hr 3.25 -0.8 0.9% 51.6% 3 out of 7 NAEFS 54.2% SREF 53.9% NGM80 23.2% 4 48-hr 3.94 -1.1 2.9% 43.2% 3 out of 7 NAEFS 51.9% SREF 45.8% NGM80 6.2% 5 60-hr 4.30 -0.4 4.4% 39.1% 4 out of 6 NAEFS 49.2% SREF 43.0% NAM40 8.9% 6 72-hr 4.76 0.1 6.4% 33.7% 5 out of 5 NAEFS 42.9% SREF 40.1% NAM12 35.2% 7 84-hr 4.85 0.3 7.5% 34.7% 2 out of 6 NAEFS 40.0% MOSGd 33.4% NAM12 8.9% 8 96-hr 5.24 0.4 13.0% 33.1% 1 out of 3 NAEFS 32.7% MOSGd 29.9% MOSGd 29.9% 9 108-hr 5.11 0.8 12.8% 35.4% 1 out of 4 HPCGd 34.5% NAEFS 32.1% MOSGd 30.5%10 120-hr 5.31 0.7 12.0% 31.9% 1 out of 3 MOSGd 31.6% NAEFS 24.8% NAEFS 24.8%11 132-hr 4.97 0.7 9.9% 35.1% 2 out of 4 HPCGd 38.0% MOSGd 30.9% NAEFS 27.2%12 144-hr 5.42 0.6 15.0% 35.0% 1 out of 3 MOSGd 31.3% NAEFS 29.0% NAEFS 29.0%13 156-hr 5.40 0.5 14.9% 35.7% 1 out of 4 HPCGd 32.9% MOSGd 32.7% NAEFS 23.4%14 168-hr 5.46 1.1 17.7% 38.1% 1 out of 3 MOSGd 35.6% NAEFS 28.4% NAEFS 28.4%
Official Guidance: NGM80, NAM40, SREF, NAM12, MOSGd, HPCGd, NAEFSContributed by Richard Grumm (WFO)
Ocean Wave Ensemble System - Hendrik Tolman
• Configuration of ocean wave ensemble system Wave ensemble has been running since 2008
– Running on 1°×1° wave model grid as the control.– 20 wave members generated through GEFS using ETR method – Cycling initial conditions for individual members to introduce
uncertainty in swell results. – 10 day forecast using the GEFS bias corrected 10m wind (future
operation)• Improving forecast uncertainties through
– Introducing ensemble initial perturbations from previous model cycle
– Introducing bias corrected ensembles as external forcing.– Example of comparison (wave heights)
• Plans– Work towards a combined NCEP-FNMOC ensemble– Analyze the role of swell played in the wave ensemble
Comparison of the ensemble systems
old cycle
cycle, BCOld ensemble setup, ensemble with cycling of initial conditions and wind bias correction (BC).
Mean wave height (contours) and spread (shading)
2008/03/28 t06z 120h forecastback
47Courtesy of Dave Novak
50th (median) and mean are best
back
48
EMC-MDL COLLABORATION• Compare quality of current operational / experimental products
– Gridded MOS vs. Downscaled NAEFS• Ongoing
– Kathy Gilbert, Val Dragostano – Zoltan Toth, Bo Cui, Yuejian Zhu• Proxy for truth issue unresolved
– Need observations independent of MOS– MDL experimental ensemble guidance vs. Downscaled NAEFS
• 10/50/90 percentiles to be evaluated– Matt Peroutka & Zoltan Toth
• Proxy for truth issue
• Proxy for truth?– Agree on best proxy for truth
• Collaborate on– Improving RTMA, including bias correction for FG– Creating best CONUS precipitation analysis & archive
• Joint research into best downscaling methods?– Climate, regime, case dependent methods– Addition of fine temporal/spatial variability into ensemble
49
SREF Probability of STP Ingredients: Time Trends
24 hr SREF Forecast Valid 21 UTC 7 April 2006
Prob (MLCAPE > 1000 Jkg-1)
X
Prob (6 km Shear > 40 kt)
X
Prob (0-1 km SRH > 100 m2s-2)
X
Prob (MLLCL < 1000 m)
X
Prob (3h conv. Pcpn > 0.01 in)
Shaded Area Prob > 5%
Max 50%
50
BACKGROUND
51
User Requirements - Simplified
• Applications affected by (extreme/high impact) weather– Must consider information on weather to
• Minimize losses due to adverse weather• Optimal user decision threshold equals
– Probability of adverse weather exceeding• Cost / loss ratio of decision situation (simplified decision theory)
• Probability of weather events must be provided– Only option in past, based on error statistics of single value forecasts
• e.g., MOS POP• Now can be based on ensemble statistical information (e.g., RMOP)
– Users act when forecast probability exceeds their cost/loss ratio (example)• Advantages
– A set of products (e.g., 10 / 50 / 90 percentile forecast , metagram, mean and mode)
• Advanced - Problems???– Proliferation of number of products
• For different variables, probability / weather element thresholds, joint probabilities– Limited usage
• Downstream applications severely limited (e. g., wave, streamflow, etc, ensembles not possible)
52
User Requirements - Advanced
• Advanced information– Statistical reliable ensemble forecast products– Ensemble statistical data – historical information– 6D-cube – space (3D) + time + variables + ensemble
• Expanded NDFD – future official NWS weather / climate / water forecast database
• Joint probabilities– Many variables, different probabilities / critical value decision thresholds– Some (or many) of forecast events are related joint probabilities.
• Probability of significant convection• Fire weather
• User application model (UAM)– Must be easy operating with quick information access– Simulating optimal weather related operations– Simulating different user procedures for multiple plausible weather scenarios– Able to tell – what are actions / costs / benefits? - assuming weather is known
• Application– Run UAM n x n times with multiple weather scenarios from each ensemble member (n) and user procedures (n)– Weather scenario from each ensemble - generated from optimized user procedures– Take ensemble mean of economic outcome (costs + losses) for each set of user procedures– Choose operating procedures to minimize costs and losses in expected sense.– Make optimizing weather related decisions
• Challenge– Requires - storage / telecom bandwidth– Requires - smart sub-setting & interrogation tools – can derive any weather related information include joint probabilities
53
NCEP/GEFS raw forecast
NAEFS final products
4+ days gain from NAEFS
From Bias correction (NCEP, CMC)Dual-resolution (NCEP only)Down-scaling (NCEP, CMC)Combination of NCEP and CMC
back
54
From Bias correction (NCEP, CMC)Dual-resolution (NCEP only)Down-scaling (NCEP, CMC)Combination of NCEP and CMC
NAEFS final products
NCEP/GEFS raw forecast
8+ days gain
back
Data on NOMADS
• SREF grid221 (North America, 30km) for individual members
• SREF bias corrected grid212 (CONUS, 40km) for individual members
• GEFS raw forecast (1*1 degree globally) for individual members
• GEFS bias corrected (1*1 degree globally) for individual members and products
• NAEFS probabilistic forecast (1*1 degree globally)• NAEFS downscaled probabilistic forecast (5*5 km
CONUS only) • NAEFS downscaled probabilistic forecast (6*6 km
Alaska)
56
Post Process - Derived Variables (Plan)
– Objective• Generate variables not carried in NWP models• Or variables can not be easy calibrated
– E. g. relative humidity
– Input data• Bias corrected and downscaled ensemble data (NWP
model output)
– Methods• Model “post-processing” algorithms
– Apply after downscaling for variables affected by surface processes
• SMARTINIT for global forecast – Geoff Manikin et al.
– NDFD weather element generator
• Other tools?– Text generation, etc?
GEFS operational extratropical cyclone tracking
The system includes two components currently:
• A global cyclone tracking system.
• A graphic display web site.
Planned GEFS cyclone tracking and verification:
• An unified storm ID for all members will be created during the tracking process;
• This will allow us to obtain a mean track among the 21 members;
• Probabilistic errors will be computed using the mean track, member tracks and an analysis track (GFS).
• All tracks will be processed and stored in MySQL Database.
Example: A Nor’easter forms at Gulf of Mexico. 3 days before impacting the mid Atlantic states, all single models predicted the storm would move
out to sea; The GEFS had several members that showed significant impact (tracks close to the BEST track)