Development of a LAM ensemble for the

2014 Winter Olympic Games

Andrea Montani, C. Marsigli, T. Paccagnella

ARPA Emilia-Romagna Servizio IdroMeteoClima, Bologna (I)

ServizioIdroMeteoClima

COSMO meetingLugano, 10-13 September 2012

A.Montani; The COSMO-LEPS system.

Outline

• Introduction to COSMO-LEPS system: methodology and implementation.

• Limited-area ensemble activity for FROST-2014: relocation of COSMO-LEPS: COSMO-FRost-EPS.

• Performance of COSMO-FROST-EPS: time-series verification using SYNOP reports and comparison

against ECMWF;

case-study assessment;

• Conclusions and open issues.

A.Montani; The COSMO-LEPS system.

COSMO-LEPS (developed at ARPA-SIMC)

• What is it?It is a Limited-area Ensemble Prediction System (LEPS),

based on COSMO-model and implemented within COSMO (COnsortium for Small-scale Modelling, including Germany, Greece, Italy, Poland, Romania, Russia, Switzerland).

• Why?It was developed to combine the advantages of global-

model ensembles with the high-resolution details gained by the LAMs, so as to identify the possible occurrence of high-impact and localised weather events (heavy rainfall, strong winds, temperature anomalies, snowfall, …)

generation of COSMO-LEPS to improve the forecast of high-

impact weather in the short and early-medium range.

Dim

2

Initial conditions Dim 1

Dim

2

Possible evolution scenarios

Dim 1 Initial conditions

ensemble size reduction

Cluster members chosen as representative members (RMs)

LAM integrations driven byRMs

LAM scenario

LAM scenario

LAM scenario

COSMO-LEPS methodologyCOSMO-LEPS methodology

A.Montani; The COSMO-LEPS system.

Outline

• Limited-area ensemble activity for FROST-2014: relocation of COSMO-LEPS: COSMO-FRost-EPS.

• As for Sochi-2014 Winter Olympics, Roshydromet is organizing a blended RDP/FDP (Research and Development Project / Forecast Demonstration Project) under the auspices of WMO: FROST-2014 (Forecasting and Research: the Olympic Sochi Testbed; http://frost2014.meteoinfo.ru).

A.Montani; The COSMO-LEPS system.

COSMO activities for FROST-2014COSMO is performing a number of activities related to FROST-2014 WWRP

RDP/FDP, coordinated in the framework of the Priority Project CORSO:

a) deterministic forecasting,

b) probabilistic forecasting:

b1) FDP initiatives (COSMO-FROST-EPS: relocation of COSMO-LEPS over the Sochi area),

b2) RDP initiatives (development of a convective-scale ensemble system for the Sochi area),

c) post-processing and product generation,

d) verification (development of VERSUS software for probabilistic verification).

Relocation of COSMO-LEPS system over the Sochi area so as to provide:

• probabilistic forecasting at high resolution for the Olympic competitions,

• support to the deterministic forecasting,

• initial and boundary conditions for the development of a convective-resolving EPS.

A.Montani; The COSMO-LEPS system.

Main milestones of COSMO-FROST-EPS

• 11/3/2011. Nothing present.

• 2/5/2011. Submission of a new ECMWF Special Project (Title: “Implementation of a limited-area ensemble prediction system for Sochi Olympic Games”; Project investigators: Majewski, Montani, Steiner; duration: 3 years) for provision of computer time to run the system on ECMWF super-computers;

• 5-9/9/2011. Discussion during the COSMO meeting about the system set-up;

• 6/12/2011. Approval of the Special Project by ECMWF Council;

• 6-9/9/2011. Visit of Russian colleagues at ARPA-SIMC to define specifics of the new ensemble system;

• 19/12/2011. Beginning of provision of COSMO-FROST-EPS products on a daily basis.

A.Montani; The COSMO-LEPS system.

COSMO-FROST-EPS @ ECMWF: present status

d+3d+3dd d+2d+2d+1d+1

ECMWF EPS

clustering interval

Cluster Analysis and RM identificationCluster Analysis and RM identification

4 variables

Z U V Q

3 levels

500 700 850 hPa

2 time steps

Cluster Analysis and RM identificationCluster Analysis and RM identification

Black-Sea area

Complete Linkage

10 Representative Members driving the 10

COSMO-model integrations (weighted according to the cluster

populations)

employing either Tiedtke or Kain-Fristch

convection scheme (randomly choosen)

+perturbations in

turbulence scheme and in physical

parameterisations

clustering area

• Δx ~ 7 km; 40 ML; fc+72h;• initial time: 00/12 UTC;• At the moment, computer time

(~ 2 million BUs for 2012) is provided by an ECMWF Special Project;

• suite managed by ARPA-SIMC; • contributions from ECMWF

member states could be needed in the future.

Integration Domain

A.Montani; The COSMO-LEPS system.

Disseminated products

All post-processing is done using with COSMO-software fieldextra:

probability fields for the exceedance of thresholds for surface fields;

ensemble mean and ensemble standard deviation for some fields;

individual ensemble members for the generation of point-forecasts;

hourly boundary conditions (from fc+0h to fc+48h) for convective-resolving ensemble (RDP part).

10 perturbed COSMO-model runs (ICs and BCs from 10 selected EPS members): start at 00UTC and 12UTC; t = 72h;

1 deterministic run (ICs and BCs from the deterministic ECMWF forecast) to “join” deterministic and probabilistic approaches: start at 00UTC and 12UTC; t = 72h;

A.Montani; The COSMO-LEPS system.

Outline

• Performance of COSMO-FROST-EPS:verification using SYNOP reports and comparison against ECMWF-EPS;

case-study assessment;

A.Montani; The COSMO-LEPS system.

COSMO-FROST-EPS vs ECMWF-EPS

Main features of verification:

variable: 12h cumulated precip (18-06, 06-18

UTC);

period : from Jan 2012 to Mar 2012;

region: 40-50N, 35-45 (SOCHIDOM);

method: nearest grid point; no-weighted

fcst;

obs: synop reports (about 60

stations/day);

fcst ranges: 6-18h, 18-30h, 30-42h, 42-54h, 54-66h;

thresholds: 1, 5, 10, 15, 25, 50 mm/12h;

scores: ROC area, BSS, RPSS, Outliers, …

systems:

- COSMO-FROST-EPS (10m, 7 km, 40 ML)

- ECMWF-EPS (51m, 25 km, 62 ML)

“Large-scale” verification tends to smooth out differences between higher and lower resolution systems ECMWF-EPS should be favoured.

A.Montani; The COSMO-LEPS system.

Impact of higher resolution

-40

500010 300020001000500100

ECMWF-EPS orography (25 km)

COSMO-FROST-EPS orography (7 km)

7 km still not enough to resolve some scale-scale features

need of statistical post-processing to downscale the information

A.Montani; The COSMO-LEPS system.

ROC area Area under the curve in the HIT rate vs FAR diagram; the higher, the better … Valuable forecast systems have ROC area values > 0.6. Consider the event: 12-hour precipitation exceeding 10 mmConsider the event: 12-hour precipitation exceeding 10 mm

COSMO-FROST-EPS outperforms ECMWF-EPS for all forecast ranges.

12-hour cycle of the score for both systems, which better predict precipitation occurring during daytime (6-18Z).

ROC area values show little dependence on the threshold (not shown).These results need to be confirmed over higher-resolution observational networks.

A.Montani; The COSMO-LEPS system.

Brier Skill Score BSS is written as 1-BS/BSref. Sample climate is the reference system. Useful forecast systems if

BSS > 0. BS is equivalent to MSE for deterministic forecast. BS measures the mean squared difference between forecast and observation in probability

space.

Lower threshold: COSMO-FROST-EPS wins for all forecast ranges.

Higher thresholds: similar performance of the two systems, but fewer obs are available.

Performance of the systems shown for 2 events:• total precipitation exceeding 1 mm in 12 hours

(dashed lines),• total precipitation exceeding 5 mm in 12 hours (solid

lines).

A.Montani; The COSMO-LEPS system.

Ranked Probability Skill Score

BSS “cumulated” over all thresholds. RPSS is written as 1-RPS/RPSref. Sample climate is the reference system. RPS is the extension of the Brier Score to the multi-event situation.

Useful forecast systems for RPSS > 0. RPSS depends on the ensemble size and penalises small ensemble sizes. Consider debiased RPSS: RPSSD = 1 –(RPS/(RPSref + RPSref /N))

In either case (RPSS or RPSSD) COSMO-FROST-EPS has higher scores than ECMWF-EPS for all forecast ranges, despite the lower ensemble size.

RPSS

RPSS

D

A.Montani; The COSMO-LEPS system.

Outliers How many times the analysis is out of the forecast interval spanned by the ensemble members. … the lower the better … In a “well-constructed” ensemble system of N members, the percentage of outliers should

asymptotically approach its theoretical values (= 2/N+1).

COSMO-FROST-EPS has fewer outliers than ECMWF-EPS, especially for shorter forecast ranges.

COSMO-FROST-EPS approaches quickly the theoretical limit; ECMWF-EPS is away from its limit.

A.Montani; The COSMO-LEPS system.

Case-study assessment

1. Heavy precipitation event: snowfall started on 31 January, at 00UTC in the mountains and at 3UTC on the coast. Snowfall lasted 2 days (31 Jan and 1 Feb). In the first day, 18 cm of snow in Sochi, 5 cm in Adler.

2. Foehn event: started on 3 February in the evening (local time) and lasted until 6 February.

Sochi local time = GMT + 4 hours

GMT = Sochi local time – 4 hours

A.Montani; The COSMO-LEPS system.

Probability maps of tp > 20 mm eq water / 24h

All maps verify on 1 Feb 2012, 12UTC

(4pm local time)

IC: 31/1, 00UTC +12-36h

IC: 30/1, 00UTC +36-60h IC: 30/1, 12UTC +24-48h

A.Montani; The COSMO-LEPS system.

• The new COSMO-based ensemble system over the Sochi-area (COSMO-FROST-EPS) was developed, implemented and runs on a daily basis.

• Preliminary verification results indicate the added value of the system with respect to ECMWF-EPS; positive results are confirmed also for case-study analysis.

• 7 km not enough to resolve some small-scale features:

– statistical post-processing to downscale the information FDP?

– dynamical downscaling to generate convective-resolving EPS RDP?

Conclusions and open issues

• COSMO-FROST-EPS is running throughout 2012 (and not only for 6 months), thanks to the Swiss contribution.

• Improve dissemination:

– develop new products “on demand” (in the future “multi-model” products?);

– produce meteograms and/or xml output files,

– Concentrate on issues related to data format / grid / ...

• Improve probabilistic verification:

– COSMO-software VERSUS is being developed to address probabilistic verification,

– need of good-quality observations at high resolution.

A.Montani; The COSMO-LEPS system.

Important ingredients

1. Provide reasonable “numbers”. addressed

2. Develop experience with probabilities.

3. Feedback on the top-priority products. being addressed

4. Snow analysis.

5. Soil-field initialisation.

6. Observations to assess the quality of the system. . being addressed

7. Computer time. addressed

8. Timeliness in product delivery. addressed

9. …

anything to add/remove?

A.Montani; The COSMO-LEPS system.

Thank you !