x general cosmo meeting cracow,15-19 september 2008

A.Montani; The COSMO-LEPS system: present status and outlookCOSMO meeting, Cracow, 15-19 September 2008

The COSMO-LEPS system:

present status and outlook

Andrea Montani, Chiara Marsigli and Tiziana Paccagnella

ARPA-SIM

Hydrometeorological service of Emilia-Romagna, Italy

X General COSMO meetingCracow,15-19 September 2008


Outline

• Present status and recent updates

• Possible changes and future plans


The new COSMO-LEPS suite @ ECMWFbefore December 2007

d-1d-1 dd d+5d+5d+1d+1 d+2d+2 d+4d+4d+3d+3

older EPSolder EPS

younger EPSyounger EPS

clustering clustering periodperiod

0000

1212

Cluster Analysis and RM identificationCluster Analysis and RM identification

4 variables4 variables

Z U V QZ U V Q

3 levels3 levels

500 700 850 hPa500 700 850 hPa

2 2 time time stepssteps


European European areaarea

Complete Complete LinkageLinkage

COSMO-LEPS

Integration Domain

1616 Representative Representative Members driving the Members driving the

1616 COSMO-model COSMO-model integrations integrations

(weighted according (weighted according to the cluster to the cluster populations)populations)

employing either employing either Tiedtke or Kain-Tiedtke or Kain-

Fristch convection Fristch convection scheme (randomly scheme (randomly

choosen)choosen)

COSMO-LEPS

clustering area

• suite running as a ”time-critical application” managed by ARPA-SIM;

• Δx ~ 10 km; 40 ML;• COSM0-LM 3.20 since Nov06;• fc length: 132h;• Computer time (4.3 million

BU for 2007) provided by the COSMO partners which are ECMWF member states.


Recent updates (1)

Changes to the operational suite on 1 December 2007:Changes to the operational suite on 1 December 2007:• new model version (COSMO-LM 4.0);• new numerics (Runge-Kutta scheme); • use of multi-layer soil model;• new perturbations introduced relative to the maximal turbulent length scale (tur_len) and

to the length scale of thermal surface patterns (pat_len) to increase spread and maintain skill.

Old system and new system ran in parallel for 52 days (October and November 2007);

Larger T2m spread in new system for all forecast ranges;

Comparable skill (in terms of T2m root-mean-square error) of the two systems.


Recent updates (2)

Since 15/6/2008, JRC has got access to COSMO-LEPS data for the use Since 15/6/2008, JRC has got access to COSMO-LEPS data for the use in the experimental European Flood Alert System (EFAS)in the experimental European Flood Alert System (EFAS)

• In the framework PREVIEW Project (WP: Medium-Range Plain Flood), COSMO-LEPS showed added value for flood forecasting, with respect to ECMWF EPS and varEPS.

COSMO-LEPS will be used, with ECMWF EPS and GME DWD deterministic forecasts, as input to the flood forecasting system over the major European river basins.

success for COSMO both from a scientific and a visibility point of view.


COSMO-LEPS suite @ ECMWFsince December 2007

d-1d-1 dd d+5d+5d+1d+1 d+2d+2 d+4d+4d+3d+3

older EPSolder EPS

younger EPSyounger EPS


0000

1212



Z U V QZ U V Q

3 levels3 levels

500 700 850 hPa500 700 850 hPa





COSMO-LEPS

Integration Domain

1616 Representative Representative Members driving the Members driving the 1616

COSMO-model COSMO-model integrations (weighted integrations (weighted according to the cluster according to the cluster

populations)populations)

employing either employing either Tiedtke or Kain-Fristch Tiedtke or Kain-Fristch

convection scheme convection scheme (randomly choosen) (randomly choosen)

++perturbations in perturbations in

turbulence schemeturbulence scheme

COSMO-LEPS

clustering area

• suite runs as a ”time-critical application” managed by ARPA-SIM;

• Δx ~ 10 km; 40 ML; fc+ 132h;• COSM0 v4.0 (with RK + multi-

layer);• computer time (5.8 million

BU for 2008) provided by the COSMO partners which are ECMWF member states.


Operational set-up

Core products:16 perturbed COSMO-model runs (ICs and 3-

hourly BCs from 16 EPS members) to generate, “via weights”, probabilistic output: start at 12UTC; t = 132h;

Additional products: 1 deterministic run (ICs and 3-hourly BCs from the

high-resolution deterministic ECMWF forecast) to “join” deterministic and probabilistic approaches: start at 12UTC; t = 132h;

1 hindcast (or proxy) run (ICs and 3-hourly BCs from ECMWF analyses) to “downscale” ECMWF information: start at 00UTC; t = 36h.


but …


Semi-diurnal cycle in COSMO-LEPS scores

RPSS score for 12-hour cumulated precipitation … the higher the better … Performance of the system assessed for the last 5 summers (JJA) and the last 5 autumns

(SON).

RPSSEvident 12-hour cycle in

RPSS scores (the same holds for BSS, while less evident for ROC area scores).

Better performance of the system for “night-time” precipitation, that is for rainfall predicted between 18Z and 6Z (ranges 30-42h, 54-66h, …).

The amplitude of the cycle is somewhat reduced throughout the years (in summers) and with increasing forecast range.

Bad performance of the system in Summer 2006.


Score dependence on the domain size (1)

Verification of COSMO-LEPS against synop reports over the MAP D-PHASE area (~ 470 stations; MAPDOM) and the full domain (~ 1500 stations; fulldom):

different statistics of the verification samples; up to now, performance of the system over the 2 domains assessed only for 6

months (March-August 2007). difficult to draw general conclusions


Score dependence on the domain size (2)

RPSS

RPSS score… the higher the better… (and positive).

ROC area… the higher the better… (and above 0.6).Smoother transitions from month to month in “fulldom” scores.

Slightly better performance of COSMO-LEPS over the MAPDOM, but the signal varies from month to month.

Higher predictability with orographic forcing?

Need to check individual regions and/or to stratify for type of stations.

OUTL

ROC

Outliers percentage … the lower the better.


Overestimation of Td2m and soil moisture (1)

Verification period: MAM07 and MAM08.

Obs: synop reports (about 470 stations x day).

Region: 43-50N, 2-18E (MAP D-PHASE area).

Larger bias and larger rmse in MAM08 rather than in MAM07 for COSMO-LEPS deterministic run (in 2007, no multi-layer soil model).


Overestimation of Td2m and soil moisture (2)

from Andrè Walser

COSMO-LEPS members have higher soil moisture than COSMO-7 run at Meteoswiss (case of 20/7), especially in the Alps.

For the future, do not simply interpolate from ECMWF IFS to COSMO model with the multi-layer soil model.

generate “home-made” soil moisture analysis (from COSMO-EU, from a proxy-run, …)


every now and then, “Floating-point exceptions” when the model writes output files over small subdomains …

Model explosion for COSMO-LEPS

Temporarily patch (which works, anyway!):the failed task is resubmitted with half time-step.

A more sophisticated investigation of model explosions will be carried out in the near future.


Feasibility study of COSMOLEPS at 7 km (cleps_7)

Motivations:Provide a more detailed description of

mesoscale processes by incresing the horizontal resolution.

Do not lose a “reasonable advantage” against ECMWF EPS, which will go to x=25 km during 2009.

“Keep the pace” with deterministic model (x~ 2-3 km): if the gap in resolutions between deterministic and probabilistic systems is too large, the two systems go for different solutions (that is, they forecast different weather!).

from 10 to 7 km (plus small domain extensions) does not seem a lot


but …


Present systemx = 10 kmz = 40 MLt = 90 sngp = 306x258x40 =

3.157.920fcst range = 132hcost = 640 BU x runelapsed time = 45 min

COSMO-LEPS at 7 km (cleps_7): the answer to forecasters’ dream?

New systemx = 7 kmz = 40 MLt = 72 sngp = 510x405x40 =

8.262.000fcst range = 132hcost = 1925 BU x runelapsed time = 138 min

… cleps_7 is about 3 times more expensive than the present configuration

new computer at ECMWF being installed

Computer resources for each ECMWF member state will increase by a factor of 5 (five) and ….


The dream is possible

COSMO-LEPS 10 kmCOSMO-LEPS 7 km

the grid of cleps_7 would be almost identical to that of COSMO-EU, this making easier and cleaner the use of initial fields provided by DWD (e.g. soil moisture analysis).


• tackle the soil moisture deficiency:

– use the soil moisture analysis fields provided by DWD,

– use the fields provided by the “proxy-run”,

• run cleps_7 for ~ 40 days in autumn 2008 and assess the impact;

• within TIGGE-LAM, develop coding of COSMO-LEPS output files in GRIB2 format;

Future plans (2008 and 2009)

• migration to the new machine at ECMWF;

• use a better snow analysis (possibly provided by DWD or Meteoswiss);

• extend the cluster analysis so as to consider not only ECMWF EPS, but also UKMO MOGREPS as global ensemble providing ic’s and bc’s (first tests);

• implementation of cleps_7;

• gaining from COSMO-SREPS experience, introduce more model perturbations;• test COSMO-LEPS nested on the under-development ECMWF EDA during MAP

D‑PHASE period;

• support calibration and verification.


Thank you !


Main results

• COSMO-LEPS system runs on a daily basis since November 2002 (6 “failures” in almost 6 years of activity) and it has become a “member-state time-critical application” at ECMWF ( ECMWF operators involved in the suite monitoring).

• COSMO-LEPS products used in EC Projects (e.g. PREVIEW), scientific experiments (e.g. COPS, MAP D-PHASE, EFAS) and met-ops rooms across COSMO community.

• Nevertheless, positive trends can be identified:• increase in ROC area scores and reduction in outliers percentages;• positive impact of increasing the population from 5 to 10 members (June

2004);• some deficiency in the skill of the system were identified after the system

upgrades occurred on Feb 2006 (from 10 to 16 members; from 32 to 40 ML + EPS upgrade!!!), but scores are encouraging throughout 2007 and, to a certain extent, 2008.

Time series scores cannot easily disentangle improvements related to COSMO-LEPS itself from those due to better boundaries by

ECMWF EPS.

2 more features:

• marked semi-diurnal cycle in COSMO-LEPS scores (better skill for “night-time” forecasts);

• better scores over the Alpine area rather than over the full domain (to be confirmed).


COSMO-LEPS

16-MEMBER EPS

51-MEMBER EPS

tp > 1mm/24h

tp > 5mm/24h

Average values (boxes 0.5 x 0.5)MAM06

As regards AVERAGE precipitation above these two threshols, the 3 systems have similar performance.


ENSEMBLE SIZE REDUCTIONIMPACT EVALUATED ON CASE STUDIES (1)


2003082512 Friuli

(+72-+96)

20 mm 100 mm

5 RMs

10 RMs

All 51


ENSEMBLE SIZE REDUCTIONIMPACT EVALUATED ON CASE STUDIES (2)

Observed precipitation between 15-11-2002 12UTC and 16-11-2002 12 UTC

Piedmont case


2002111212 Piedmont

20 mm 150 mm

5 RMs

10 RMs

All 51


COSMO-LEPS

Real time products


Why Limited Area Ensemble Prediction?

• Global Ensemble Prediction Systems

– have become extremely important tools to tackle the problem of predictions beyond day 2

– are usually run at a coarser resolution with respect to deterministic global predictions → skill in forecasting intense and localised events is currently still limited.


Why Limited Area Ensemble Prediction?(2)

As regards high resolution deterministic forecast in the short range, where limited-area models play the major role, a “satisfactory” QPF is still one of the major challenges. The same can be said for other local parameters.

This is due, among other reasons, to the inherently low degree of predictability typical of severe and localised events.

Probabilistic/Ensemble approach is so required also for the short range at higher resolution


From Global EPS to LAM EPS

• In the Limited-area ensemble systems, tailored for the short range, perturbations must be already “active” during the first hours of integration

• The characteristic of the LAM ensemble are strongly dependent by the lateral boundaries forcing.

• Due to the “regional” application of these Limited Area Ensembles, methodologies can be different in different geographycal regions.

A pratical consideration:Global EPS ~ Big Centres

Limited Area EPS ~ (also) Relatively Small Centres


In the last period the verification package is being developed keeping into account two measure of precipitation:

the cumulative volume of water deployed over a specific region

the rainfall peaks which occur within this region

OBJECTIVE VERIFICATION OF COSMO-LEPS

COSMO observations


CLEPS EPS

Verification grid

OBS MASK


COSMO-LEPS

16-MEMBER EPS

51-MEMBER EPS

tp > 1mm/24h

tp > 5mm/24h

Average values boxes 0.5x0.5 deg3 sis


COSMO-LEPS

16-MEMBER EPS

tp > 1mm/24h

NOCC=610

NOCC=1195

tp > 5mm/24h

NOCC=2671

tp > 10mm/24h

ave 0.5


Maximum values boxes 0.5x0.5 deg

tp > 1mm/24h

tp > 5mm/24h

tp > 10mm/24h

COSMO-LEPS

16-MEMBER EPS

51-MEMBER EPS

3 sis


COSMO-LEPS

16-MEMBER EPS

tp > 20mm/24hNOCC=227

Average values boxes 0.5x0.5 deg


COSMO-LEPS NW

COSMO-LEPS W

tp > 1mm/24h

COSMO-LEPS weighting procedure

maximum values (boxes 0.5x0.5 deg)


The COSMO-LEPS suite @ ECMWFNovember 2002 – May 2004

d-1d-1 dd d+5d+5d+1d+1 d+2d+2 d+4d+4d+3d+3

oldest EPSoldest EPS

1212

middle EPSmiddle EPS

oldest EPSoldest EPS

youngest EPSyoungest EPS


0000

1212



Z U V QZ U V Q

3 levels3 levels

500 700 850 hPa500 700 850 hPa





5 Representative 5 Representative MembersMembers

Driving the 5Driving the 5COSMO-model COSMO-model integrationsintegrations

COSMO-LEPS

Integration Domain

COSMO-LEPS

clustering area


The COSMO-LEPS suite @ ECMWFJune 2004 – January 2006

d-1d-1 dd d+5d+5d+1d+1 d+2d+2 d+4d+4d+3d+3

middle EPSmiddle EPS

youngest EPSyoungest EPS


0000

1212



Z U V QZ U V Q

3 levels3 levels

500 700 850 hPa500 700 850 hPa





COSMO-LEPS

Integration Domain

10 10 Representative Representative

MembersMembersdriving the 10driving the 10COSMO-model COSMO-model integrationsintegrations

employing either employing either Tiedtke or Kain-Tiedtke or Kain-Fristch scheme Fristch scheme

randomly randomly choosenchoosen

COSMO-LEPS

clustering area

• Suite running in real time at ECMWF managed by ARPA-SIM;

• Δx ~ 10 km• Fc length: 120h


COSMO-LEPS (developed at ARPA-SIM)

• What is it?It is a Limited-area Ensemble Prediction System

(LEPS), based on COSMO-model and implemented within COSMO (COnsortium for Small-scale MOdelling, which includes Germany, Greece, Italy, Poland, Romania, Switzerland).

• Why?Because the horizontal resolution of global-model

ensemble systems is limited by computer time constraints and does not allow a detailed description of mesoscale and orographic-related processes.

The forecast of heavy precipitation events can still be inaccurate (in terms of both locations and intensity) after the short range.


COSMO-LEPS project

combine the advantages of global-model ensembles with the high-resolution details gained by the LAMs, so as to identify the possible occurrence of intense and localised weather events (heavy rainfall, strong winds, temperature anomalies, snowfall, …)

generation of COSMO-LEPS in order to improve the Late-Short (48hr) to Early-Medium (132hr) range forecast of the so-

called “severe weather events”.


Dim 2

Initial conditions Dim 1 Dim 2

Possible evolution scenarios

Dim 1 Initial conditions

ensemble size reduction

Cluster members chosen as representative members (RMs)

LAM integrations driven byRMs

LAM scenario

LAM scenario

LAM scenario

COSMO-LEPS methodologyCOSMO-LEPS methodology


Time series of Brier Skill ScoreBSS is written as 1-BS/BSref. Sample climate is the reference system. Useful forecast systems

if BSS > 0. BS measures the mean squared difference between forecast and observation in probability

space. Equivalent to MSE for deterministic forecast.BSS improvement of

performance detectable for all thresholds along the years;

still problems with high thresholds, but good trend in 2007.

fc step: 30-42h

Jun04: 5m 10m

Feb06: 10m16m; 32ML 40 ML


Time series of ROC area for COSMO-LEPS (2)

Area under the curve in the FAR vs HIT diagram. Valuable forecast systems have ROC area values > 0.6

ROC

the positive impact of increasing the ensemble size in 2004 is evident for all thresholds and for different fcst ranges.

poor performance of the system in Spring and Summer 2006, despite system upgrades.

Jun04: 5m 10m

Feb06: 10m 16m; 32ML 40 ML

fc step: 78-90h


Time series of ROC areaArea under the curve in the HIT rate vs FAR diagram. Valuable forecast systems have ROC area values > 0.6.

ROC

Jun04: 5m 10m

Feb06: 10m 16m; 32ML 40 ML

fc step: 30-42h

the positive impact of increasing the ensemble size in 2004 is evident for all thresholds and for different fcst ranges.

poor performance of the system in Spring and Summer 2006 (both particularly dry), despite upgrades.

fc step: 78-90h

Jun04: 5m 10m

Feb06: 10m 16m; 32ML 40 ML


Outliers: time series + seasonal scoresHow many times the analysis is out of the forecast interval spanned by the ensemble

members. … the lower the better … Performance of the system assessed as time series and for 5 different Summers (JJA).

OUTL Evident seasonal

cycle (more outliers in winter), but overall reduction of outliers in the years.

Jun04: 5m 10m Feb06: 10m 16m; 32ML 40 ML

Reduction of outliers from one Summer to the other, related to the increase of ensemble size (more evident for the 5 to 10 increase).

Need to take into account the different statistics for each season (JJA 2003 less rainy than the others).


Outliers: time series + seasonal scoresHow many times the analysis is out of the forecast interval spanned by the ensemble

members. … the lower the better … Performance of the system assessed for 4 different Autumns.

OUTL

Continuous reduction of outliers from one year to the other, also related to the increase of ensemble size (more evident for the 5 to 10 increase, rather than from 10 to 16).

Need to take into account the different statistics for each season (SON 2006 less rainy than the others).

Jun04: 5m 10m Feb06: 10m 16m; 32ML 40 ML


Semi-diurnal cycle in COSMO-LEPS scores

BSS score … the higher the better … Performance of the system assessed for 5 different Summers (JJA).

BSS Evident 12-hour cycle in

BSS scores (the same holds for RPSS, while less evident for ROC area scores).

Better performance of the system for “night-time” precipitation, that is for rainfall predicted between 18Z and 6Z (ranges 30-42h, 54-66h, …).

The amplitude of the cycle is somewhat reduced throughout the years and with increasing forecast range.

The bad performance in Summer 2006 is confirmed.


Main results

• COSMO-LEPS system runs on a daily basis since November 2002 (6 “failures” in almost 5 years of activity) and it has become a “member-state time-critical application” at ECMWF ( ECMWF operators involved in the suite monitoring).

• COSMO-LEPS products used in EC Projects (e.g. PREVIEW) , field campaigns (e.g. COPS, MAP D-PHASE) and met-ops rooms across COSMO community.

• Nevertheless, positive trends can be identified:• increase in ROC area scores and reduction in outliers percentages;• positive impact of increasing the population from 5 to 10 members (June

2004);• although some deficiency in the skill of the system were identified after the

system upgrades occurred on February 2006 (from 10 to 16 members; from 32 to 40 model levels + EPS upgrade!!!), scores are encouraging throughout 2007.

Time series scores cannot easily disentangle improvements related to COSMO-LEPS itself from those due to better boundaries by

ECMWF EPS.

2 more features:

• marked semi-diurnal cycle in COSMO-LEPS scores (better skill for “night-time” forecasts);

• better scores over the Alpine area rather than over the full domain (to be confirmed).

x general cosmo meeting cracow,15-19 september 2008

Documents

cosmoleps system

cosmoleps data

cosmo partners

cosmomodel integrations

perturbed cosmomodel

new system

new model version cosmolm

flood forecasting system