performance of deterministic numerical weather forecasts during the frost-2014 project field...

Performance of deterministic numerical weather

forecasts during the FROST-2014 project field campaign

Anastasia Bundel (1), Anatoly Muraviev (1), Pertti Nurmi (2), Dmitry Kiktev (1), Alexander Smirnov (1), and

Andrea Montani (3)(1) Hydrometeorological Research Centre of Russia, Moscow, Russian

Federation, (2) Finnish Meteorological Institute,

(3) HydroMeteoClimate Service of Emilia-Romagna Region

What’s new since the 4th FROST meeting

• FROST observations were additionally controlled, scores recalculated

• Verification tools on the FROST site are developed: - Bootstrap CIs are added in Monitoring:

- Point forecast and diagnostic data viewer – selection of stations, CSV export, more user-friendly interface

http://frost2014.meteoinfo.ru/forecast/point-forecast-and-diagnostic-data-viewer

CIs are not available yet for station aggregations

Verification setup

Mountain cluster: 25 stations Coastal cluster: only 6 stations

Period: 29 Jan – 15 March 2014 –> homogeneity among all the models (COSMO-Ru1 started 29 Jan 2014)Only nearest point approach (with its shortcomings)!

48 hr / 1 h 2 km

INCA 24 (precip 10 min)

48 h / 1h (precip 10 min)

1 km

ARPA-DET 2 72 / 3h 7 km

Joint 24 78 / 1 h Pointwise

+

Only uncorrected forecasts were verified!

All the models, T2m MAE, Mountain, 00h run

0

1

2

3

4

5

6 COSMO_Ru1COSMO_Ru2COSMO_Ru7GEM-1GEM-2.5GEM-250HARMONIENMMBINCAJointKMA-NIMRARPA-DET

Big errors of KMA-NIMR at the initial steps

Nowcasting effect

highest-resolution GEM-250 best overallHARMONIE best during the daytime (up to ~1500 m)

lowest-resolution 7-km models worst

All the models, T2m MAE, Coastal, 00h run

0

1

2

3

4

5


Big daytime errors of GEM

HARMONIE’s daytime minimum-nighttime maximum persists

T2m MAE, 00h run

0

1

2

3

4

5

6

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48

COSMO_Ru2GEM-2.5KMA-NIMR

0

1

2

3

4

5

6

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48

COSMO_Ru1GEM-1HARMONIENMMBINCA

Mountain: COSMO, INCA, NMMB: min errors during the nighttime GEM, HARMONIE, KMA-NIMR: min errors during the daytime

0

1

2

3

4

5

6

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48


0

1

2

3

4

5

6

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48


Coastal: GEM diurnal cycle is opposite to the Mountain one

T2m MAE, 7km models, 00h runEffect of driving model (GME for COSMO-Ru7 vs. ECMWF-IFS for ARPA-DET)

0

1

2

3

4

5

6

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48

COSMO_Ru7ARPA-DET

0

1

2

3

4

5

6

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48

COSMO_Ru7ARPA-DET

Mountain Coastal

Effect of resolution, T2m MAE, 00h run

0

1

2

3

4

5

6

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48

COSMO_Ru1COSMO_Ru2COSMO_Ru7

0

1

2

3

4

5

6

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48

GEM-1GEM-2.5GEM-250

• Clear effect of resolution – the higher, the better (closer nearest point due to better orography!)• COSMO-Ru1 and GEM-1 have comparable MAEs• COSMO-Ru2 is closer to COSMO-Ru1 than GEM2.5 to GEM-1• Potential for COSMO-Ru1 (increase in domain, etc.)

Mountain

Effect of resolution, T2m MAE, 00h run

• COSMO family: similar scores• GEM-250: flattened diurnal cycle• GEM-1 and GEM-2 – diurnal maximum is opposite to that of the Mountain cluster

0

1

2

3

4

5

6

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48


0

1

2

3

4

5

6

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48

GEM-1GEM-2.5GEM-250

Coastal

All the models, RelHum, MAE, Mountain, 00h run

0

5

10

15

20


Nowcasting effect

GEM-250 best again (as for T2m)Otherwise, no clear dependence on resolution

COSMO-Ru7 worst during the daytimeMAE max at 10-14 UTC for most models

COSMO-Ru1

Effect of resolution, RelHum MAE, Mountain, 00h run

• COSMO family: effect of resolution after initial steps• GEM family: effect of resolution is clear from 10 to 20 UTC

0

5

10

15

20

25

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48


0

5

10

15

20

25

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48

GEM-1GEM-2.5GEM-250

All the models, Wind speed MAE, Mountain, 00h run

0

0.5

1

1.5

2

2.5 COSMO_Ru1COSMO_Ru2COSMO_Ru7GEM-1GEM-2.5GEM-250HARMONIENMMBINCAJointKMA-NIMRARPA-DET

No nowcasting effect!

No clear dependence on resolution

Big INCA and NMMB MAEs

COSMO-Ru7 vs. ARPA-DET: Almost no influence of driving model

Ski Jump-800, 15 Jan-15 Mar 2014

Wind direction ME is negative overall, Wind speed and gusts are mostly overestimated. Joint, HARMONIE and INCA best for gusts

ME

JointCOSMO-RU2GEM-250mHARMONIENMMBINCAMAE

Effect of resolution, Wind speed MAE, Mountain, 00h run

• COSMO family: COSMO-Ru1 advantage for wind speed (confirmed by forecasters!)• GEM family: GEM-250 worst! (unlike for T2m and RelHum) Can it be related to explicit resolving of large eddies?

0

0.5

1

1.5

2

2.5

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48


0

0.5

1

1.5

2

2.5

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48

GEM-1GEM-2.5GEM-250

CONCLUSIONS

• For T2m, the effect of resolution is clear in the mountains (better nearest point due to higher resolution orography), for relative humidity, it is important within model families, for wind speed, there is no clear dependence of errors on resolution

• T2m: - In the mountain cluster, COSMO, INCA, and NMMB have min MAE during the nighttime, while GEM, HARMONIE, and, KMA-NIMR, during the daytime; - In the coastal cluster, for GEM-1 and GEM-2.5, a big daytime error maximum appears- Bigger daytime errors for ARPA-DET vrt. COSMO-Ru7

CONCLUSIONS, cont.

• Relative humidity (in the mountains) :- GEM-0.25 has lowest MAEs, then goes COSMO-Ru1 - ARPA-DET has lower relative humidity MAE vrt. COSMO-Ru7

• Wind speed (in the mountains) : - INCA and NMMB errors are high due to overestimation (INCA is good for gusts) - COSMO-Ru1 has added value for wind (confirmed by forecasters)- GEM-0.25 is worse than GEM-1 and GEM-2.5- Almost no difference between COSMO-Ru7 and ARPA-DET -> no

influence on driving model

PLANS

• To proceed with area aggregation (height levels, etc.)• To add the CIs for the aggregated scores• To add some categorical scores (visibility!)• To gather and analyze the results in a paper

Thank you for your attention!

20COSMO GM Sibiu, 2 - 5 Sept. 2013

T2m Mean errors (BIAS)

• Вставить общие графики ME для Mountain и Coastal!

• Underestimation of temperature during the sunny weather, likely due to insufficient soil warming

T2m MAE for different heights, 15 Jan-15 Mar 2014, 00 runs

HARMONIE’s errors become bigger after ~1500 mJoint and INCA are the best at RKHU-3 during the first hours (nowcasting effect!)

RKHU-3, altitude 2043 m

JointCOSMO-RU2GEM-250mHARMONIENMMBINCA

Models’ parameterizationsModel Source of BC

and ICLarge-scale

dynamicsBoundary

layer param.

ConvectionParam.

Large-scalePrecip.scheme

Cloud scheme Land-surface scheme

Radiation

COSMO GME primitive hydro thermodynamical equations describing compressible non-hydrostatic flow in a moist atmosphere

Surface layer scheme (based on turbulent kinetic energy) including a laminar-turbulent roughness layer.

Tiedtke (1989) mass-flux convection scheme with equilibrium closure based on moisture convergence. Option for a modified closure based on CAPE.

Reduced Tiedtke scheme for shallow convection only.

Precipitation formation by a bulk microphysics parameterization including water vapour, cloud water, cloud ice, rain and snow with 3D transport for the precipitating phases.

Cloud water condensation and evaporation by saturation adjustmen

Multi-layer version of the former two-layer soil model after Jacobsen and Heise (1982) based on the direct numerical solution of the heat conduction equation. Snow and interception storage are included.

δ-two stream radiation scheme after Ritter and Geylen (1992) for short- and longwave fluxes (employing eight spectral intervals); full cloud-radiation feedback.

WRF-NMMB Downscaled from GFS

nonhydrostatic Eulerian gridpoint

Mellor-Yamada-Janjic level 2.5

Betts-Miller-Janjic

Ferrier Ferrier NOAH Rapid Radiative Transfer Model (RRTM)

HARMONIE Downscaled from ECMWF-IFS

GEM GEM global run at 25 km resolution

Semi-Lagrangian Moist TKE PBL scheme

Kain and Fritsch deep convection scheme (10-km only), Kuo-Transient shallow convection scheme (all resolutions)

Precipitation is diagnosed from convective and moist TKE PBL schemes. Grid-scale precipitation and clouds are from Milbrandt-Yau 2-moment bulk microphysics scheme.

Same as large-scale

ISBA land-surface scheme

Li and Barker radiative transfer scheme

WRF-KMA ECMWF T127

Runge-Kutta 3rd order

YSU PBL no CP in 2km not used in 6km and 2km resolution, Kain-Fritsch in 18km only

WDM6 Modified NoahLSM using Penman-Monteith eq. with modified saturation in snow

Band model using RRTM

24

Is it due to model changes, or 2013 warm anomaly, or observation data distribution, amount, and quality?Difficulty for model calibration!

COSMO T2m ME changes its sign from 2011-2012 winter to 2013 winter

1st test period, Dec2011-Feb2012 2nd test period, Jan-Feb 2013COLD winter WARM winter

0

0.5

1

1.5

2

2.5

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48

COSMO_Ru7ARPA-DET

0

0.5

1

1.5

2

2.5

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48


0

0.5

1

1.5

2

2.5

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48


Wind speed MAE, Mountain, 00h run

~1-km models

7-km models

Almost no influence of driving model

~2-km models

Visibility scores, meters, Mountain, 00h run

0

10000

20000

30000

40000

50000

60000

70000

80000

0 2 4 6 8 10 12 14 16 18 20 22 24 26

GEM-1GEM-2.5GEM-250NMMB

0

10000

20000

30000

40000

50000

60000

70000

80000

0 2 4 6 8 10 12 14 16 18 20 22 24 26

GEM-1GEM-2.5GEM-250NMMB

MAE ME

Difference between GEM and NMMB is due to different maximum values (roughly, 10000 m and 70000 m are the same) Categorical scores are needed!

performance of deterministic numerical weather forecasts during the frost-2014 project field...

Documents