1

March 2007 NAEFS Upgrade

Yuejian Zhu,

Richard Wobus, Mozheng Wei, Bo Cui and Zoltan Toth

Environmental Modeling Center

NOAA/NWS/NCEP

Acknowledgements:

Qingfu Liu, DingChen Hou, Mark Iredell and Stephen Lord EMC

Luke Lin, David Michaud, Joey Carr and Brent Gordon NCO

2

Planned Changes - Summary1. Increasing NAEFS (NCEP/GEFS)

membership• 80 perturbations in cycling (see schematic plot)

• Replaced operational 56 perturbations in cycling

• 20 perturbed long forecasts (16-d) in each cycle• Replaced operational 14 long forecasts in each cycle

2. NAEFS (NCEP/GEFS) post-product• Post-process 20 ensemble forecasts instead of

operational 14 ensemble forecasts for:• Bias correction

• Anomaly forecast

3

T00Z

56m

T06Z

56m

T18Z56m

T12Z56m

6hrs Next T00Z

Re-scaling

Re-scaling

Re-scaling

Up to 16-d

Up to 16-d

Up to 16-d

Up to 16-d

T00Z

80m

6hrs

T06Z

80m

T12Z

80m

T18Z

80m

Up to 16-d

Up to 16-d

Up to 16-d

Up to 16-d

Re-scaling

Re-scaling

Re-scaling

Re-scaling

6 hours breeding cycle Production

6 hours breeding cycle Planned Change

Next T00Z

Re-scaling

4

Current Plan

Model GFS GFS (current)

Initial uncertainty ETR ETR

Model uncertainty None None

Tropical storm Relocation same

Daily frequency 00,06,12 and 18UTC same

Hi-re control

(GFS)

T382L64 (d0-d7.5)

T190L64 (d7.5-d16)

same

Low-re control (ensemble control)

T126L28 (d0-d16)

00,0612 and 18UTC

same

Perturbed members 14 for each cycle 20 for each cycle

Forecast length 16 days (384 hours) same

Implementation May 30th 2006 March 27th 2007

GEFS configurations

5

Early study indicated:

(by Mozheng Wei)

ET20/80 give most skillful probabilstic forecast when

compared to 10m operation and 10m ET

6

Planned Changes 1 • Increasing NAEFS (NCEP/GEFS) membership.

– From current 14 increasing to 20 members per cycle.• Tuning initial perturbations.

• Using 80 cold start initial perturbations (on Jan. 25) – From 24-h forecasts and many dates.

– To have large spread of sampling

– This change is intended to improve ensemble based probabilistic forecast over all and to support NAEFS (North American Ensemble Forecast System) project.

– Results:• Improving probabilistic skills.

• Not much improvement for ensemble mean (expected).

7

ENS_s – operationENS_b – large sprdENS_d – small sprd

Doted lines – 1-day spreadDay-5 PAC scores

Doted lines-spread

Cold start from 00UTC Nov. 17 2006

Large initial spread has more skill

8

Different period-more runs for large initial spread (cold start)

ENS_s – operationENS_c – large sprd

Cold start from 00UTC Nov. 7th 2007

NH 500hPa ROC

Tropical 500hPa ROC

SH 500hPa ROC

9

20m has more prob skills than 14m

ENS_s – operationENS_b – large sprdENS_d – small sprd

NH 500hPa RPSS

SH 500hPa ROC

Tropical 500hPa ROC

10

Retrospective experiments/NCO real time parallel

• Retrospective experiments.– Summer period: August 22- September 24 2006 (Hurricane season).

– Winter period: November 7-December 7 2006

– 20 ensemble members.

– Using current GFS.

• NCO real time parallel.– Starting since January 25 2007.

• Full evaluations (follow up above experiments).– Summer: http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/html/exp/ens_20m_2006_exp_h.html

– Winter: http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/html/exp/ens_20m_2006_exp_c.html

– NCO real time: http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/html/opr/prx_2007.html

• Expect to implement (March 27 2007).

11

Summer 2006Retrospective runs

Winter 2006Retrospective runs

12

Summer 2006Retrospective runs

Winter 2006Retrospective runs

13

Summer 2006Retrospective runs

Winter 2006Retrospective runs

14

0

50

100

150

200

250

300

12 24 36 48 72 96 120

ENSs ENSx

Hurricane Track Errors (Period: 08/22-09/24/2006)

Cases 157 146 133 121 99 80 58

Hours

ENSs-operational ensembleENSx-retrospective runs

Atlantic Basin

0

100

200

300

400

500

12 24 36 48 72 96 120

ENSs ENSx

Cases 98 86 74 64 40 19 10

East Pacific Basin

15

NCO real time parallel

NH 500hPa geopotential height

NCO real time parallel

NH 2 meter temperature

16

Conclusions of the 20m experiments• Ensemble mean:

– No significant different (do not expect).– No degradation (not worse – very important)

• Ensemble distribution (probabilistic forecast):– Significant improvement from 20 ensemble members.

• For ROC, EV, Reliability.• Not for RPSS BSS.

• Initial perturbation for cold start: – Large cold start initial spread is (JIFed)

• From backup perturbation based on 24h forecasts in many dates.• Giving sufficient spread in ET.• Quickly scaling down to the expected level (1-2 days).• Able to catch up large uncertainty cases.• Degrading the skills a little for short lead-time in the first few cycles.

– Small cold start initial spread is• From backup perturbation based on 6h forecasts in one day.• No degradation at early cycling.• Not able to catch up large uncertainty system for longer lead-time.• Because error grows very slowly.

17

Planned Changes 2Details for Slide 2

• NAEFS (NCEP/GEFS) Post Products– Bias corrected members of joint MSC-NCEP ensemble

• NCEP – Current: 14 (perturbed forecasts) + 1 (control forecast) = 15 forecasts at each cycle– Plan: 20 (perturbed forecasts) + 1 (control forecast) = 21 forecasts at each cycle

• MSC – Same: 16 (multi-model, multi-physics forecasts) + 1 (control forecast) = 17 forecasts at 00UTC and

12UTC cycle only

• 35 of NAEFS variables• Bias correction against each center’s own operational analysis

– Weights for each member for creating joint ensemble (equal weights right now)• Weights don’t depend on the variables• Weights depend on geographical location (low precision packing)• Weights depend on the lead time

– Climate anomaly percentiles for each member• NCEP

– Current: 14 (perturbed forecasts) + 1 (control forecast) = 15 forecasts at each cycle– Plan: 20 (perturbed forecasts) + 1 (control forecast) = 21 forecasts at each cycle

• MSC– Same: 16 (multi-model, multi-physics forecasts) + 1 (control forecast) = 17 forecast at 00UTC and

12UTC cycle only

• 19 of NAEFS variables

18

NH Mean Sea Level Pressure

NH 2 Meter Temperature

NH 500hPa Height

NCO real time parallel:

Accumulated bias (absolute values) for past one month

Black – raw forecasts

Red – bias corrected forecasts

19

Summary of Subjective Analysis• CPC

– We do not see any cause for concern that an increase in the ensemble will have any negative impact on the CPC operational products.  Given that, we fully support this implementation to increase the ensemble size.

• HPC– There was little improvement over the standard GFS ensemble forecast. This

subjective evaluation was noted on both mass fields and model QPF. – HOC recommends implementation

• OPC– Little difference noted between operational and parallel data sets.

– OPC recommends implementation

• AWC– While AWC supports the idea to increase the number of members in the

ensemble to represent better the envelope of possibilities for various forecast parameters, we are unable to provide a more rigorous assessment due to the lack of resources to ingest the data and routinely view the available fields