ix general cosmo meeting athens,18-21 september 2007
DESCRIPTION
The COSMO-LEPS system: getting close to the 5-year milestone Andrea Montani, Chiara Marsigli and Tiziana Paccagnella ARPA-SIM Hydrometeorological service of Emilia-Romagna, Italy. IX General COSMO meeting Athens,18-21 September 2007. Motivations Methodology of COSMO-LEPS - PowerPoint PPT PresentationTRANSCRIPT
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
The COSMO-LEPS system:
getting close to the 5-year milestone
Andrea Montani, Chiara Marsigli and Tiziana Paccagnella
ARPA-SIM
Hydrometeorological service of Emilia-Romagna, Italy
IX General COSMO meetingAthens,18-21 September 2007
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Outline
• Motivations• Methodology of COSMO-LEPS• Verification results:
– SYNOP on the GTS
• Future plans
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
COSMO-LEPS (developed at ARPA-SIM)
• What is it?It is a Limited-area Ensemble Prediction System
(LEPS), based on COSMO-model and implemented within COSMO (COnsortium for Small-scale MOdelling, which includes Germany, Greece, Italy, Poland, Romania, Switzerland).
• Why?Because the horizontal resolution of global-model
ensemble systems is limited by computer time constraints and does not allow a detailed description of mesoscale and orographic-related processes.
The forecast of heavy precipitation events can still be inaccurate (in terms of both locations and intensity) after the short range.
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
COSMO-LEPS project
combine the advantages of global-model ensembles with the high-resolution details gained by the LAMs, so as to identify the possible occurrence of intense and localised weather events (heavy rainfall, strong winds, temperature anomalies, snowfall, …)
generation of COSMO-LEPS in order to improve the Late-Short (48hr) to Early-Medium (132hr) range forecast of the so-
called “severe weather events”.
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Dim 2
Initial conditions Dim 1 Dim 2
Possible evolution scenarios
Dim 1 Initial conditions
ensemble size reduction
Cluster members chosen as representative members (RMs)
LAM integrations driven byRMs
LAM scenario
LAM scenario
LAM scenario
COSMO-LEPS methodologyCOSMO-LEPS methodology
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
The COSMO-LEPS suite @ ECMWFNovember 2002 – May 2004
d-1d-1 dd d+5d+5d+1d+1 d+2d+2 d+4d+4d+3d+3
oldest EPSoldest EPS
1212
middle EPSmiddle EPS
oldest EPSoldest EPS
youngest EPSyoungest EPS
clustering clustering periodperiod
0000
1212
Cluster Analysis and RM identificationCluster Analysis and RM identification
4 variables4 variables
Z U V QZ U V Q
3 levels3 levels
500 700 850 hPa500 700 850 hPa
2 2 time time stepssteps
Cluster Analysis and RM identificationCluster Analysis and RM identification
European European areaarea
Complete Complete LinkageLinkage
5 Representative 5 Representative MembersMembers
Driving the 5Driving the 5COSMO-model COSMO-model integrationsintegrations
COSMO-LEPS
Integration Domain
COSMO-LEPS
clustering area
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
The COSMO-LEPS suite @ ECMWFJune 2004 – January 2006
d-1d-1 dd d+5d+5d+1d+1 d+2d+2 d+4d+4d+3d+3
middle EPSmiddle EPS
youngest EPSyoungest EPS
clustering clustering periodperiod
0000
1212
Cluster Analysis and RM identificationCluster Analysis and RM identification
4 variables4 variables
Z U V QZ U V Q
3 levels3 levels
500 700 850 hPa500 700 850 hPa
2 2 time time stepssteps
Cluster Analysis and RM identificationCluster Analysis and RM identification
European European areaarea
Complete Complete LinkageLinkage
COSMO-LEPS
Integration Domain
10 10 Representative Representative
MembersMembersdriving the 10driving the 10COSMO-model COSMO-model integrationsintegrations
employing either employing either Tiedtke or Kain-Tiedtke or Kain-Fristch scheme Fristch scheme
randomly randomly choosenchoosen
COSMO-LEPS
clustering area
• Suite running in real time at ECMWF managed by ARPA-SIM;
• Δx ~ 10 km• Fc length: 120h
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
The new COSMO-LEPS suite @ ECMWFsince February 2006
d-1d-1 dd d+5d+5d+1d+1 d+2d+2 d+4d+4d+3d+3
older EPSolder EPS
younger EPSyounger EPS
clustering clustering periodperiod
0000
1212
Cluster Analysis and RM identificationCluster Analysis and RM identification
4 variables4 variables
Z U V QZ U V Q
3 levels3 levels
500 700 850 hPa500 700 850 hPa
2 2 time time stepssteps
Cluster Analysis and RM identificationCluster Analysis and RM identification
European European areaarea
Complete Complete LinkageLinkage
COSMO-LEPS
Integration Domain
1616 Representative Representative Members driving the Members driving the
1616 COSMO-model COSMO-model integrations integrations
(weighted according (weighted according to the cluster to the cluster populations)populations)
employing either employing either Tiedtke or Kain-Tiedtke or Kain-
Fristch convection Fristch convection scheme (randomly scheme (randomly
choosen)choosen)
COSMO-LEPS
clustering area
• suite running as a ”time-critical application” managed by ARPA-SIM;
• Δx ~ 10 km; 40 ML;• COSM0-LM 3.20 since Nov06;• fc length: 132h;• Computer time (4.3 million
BU for 2007) provided by the COSMO partners which are ECMWF member states.
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Present methodology
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Operational set-up
Core products:16 perturbed LM runs (ICs and 3-hourly BCs
from 16 EPS members) to generate probabilistic output (start at 12UTC; t = 132h);
Additional products: 1 deterministic run (ICs and 3-hourly BCs from the
high-resolution deterministic ECMWF forecast) to assess the relative merits between deterministic and probabilistic approach (start at 12UTC; t = 132h);
1 “hindcast” run (ICs and 3-hourly BCs from ECMWF analyses) to “downscale” ECMWF information (start at 00UTC; t = 36h).
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Dissemination
probabilistic products deterministic products (individual COSMO-LEPS runs) derived probability products (EM, ES) meteograms over station points
products delivered at about 1UTC to the COSMO weather services, to Hungary (case studies) and to the MAP D-PHASE and COPS communities (field campaign).
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
– SYNOP on the GTS (COSMO-LEPS only);
Objective verification of COSMO-LEPS
Verification package includes the traditional probabilistic scores:
•Brier Skill Score (Wilks, 1995)•ROC area (Mason and Graham, 1999)•Cost-loss Curve (Richardson, 2000)•Percentage of Outliers (Buizza, 1997)•Ranked Probability Skill Scores (Wilks, 1995)
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
– SYNOP on the GTS (COSMO-LEPS only);
Objective verification of COSMO-LEPS
Main features:
variable: 12h cumulated precip (18-06, 06-18
UTC);
period: from Dec 2002 to Aug 2007;
region: 43-50N, 2-18E (MAP D-PHASEPHASE
area);
method: nearest grid point; no-weighted fcst;
obs: synop reports (about 470 stations x
day);
fcst ranges: 6-18h, 18-30h, …, 102-114h, 114-126h;
thresholds: 1, 5, 10, 15, 25, 50 mm/12h;
system: COSMO-LEPS
both (57) monthly and (19) seasonal scores are computed.
work is in progress for verification over the full domain (1500
stations)
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Time series of Brier Skill ScoreBSS is written as 1-BS/BSref. Sample climate is the reference system. Useful forecast systems
if BSS > 0. BS measures the mean squared difference between forecast and observation in probability
space. Equivalent to MSE for deterministic forecast.BSS improvement of
performance detectable for all thresholds along the years;
still problems with high thresholds, but good trend in 2007.
fc step: 30-42h
Jun04: 5m 10m
Feb06: 10m16m; 32ML 40 ML
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Time series of ROC areaArea under the curve in the HIT rate vs FAR diagram. Valuable forecast systems have ROC area values > 0.6.
ROC
Jun04: 5m 10m
Feb06: 10m 16m; 32ML 40 ML
fc step: 30-42h
the positive impact of increasing the ensemble size in 2004 is evident for all thresholds and for different fcst ranges.
poor performance of the system in Spring and Summer 2006 (both particularly dry), despite upgrades.
fc step: 78-90h
Jun04: 5m 10m
Feb06: 10m 16m; 32ML 40 ML
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Outliers: time series + seasonal scoresHow many times the analysis is out of the forecast interval spanned by the ensemble
members. … the lower the better … Performance of the system assessed as time series and for 5 different Summers (JJA).
OUTL Evident seasonal
cycle (more outliers in winter), but overall reduction of outliers in the years.
Jun04: 5m 10m Feb06: 10m 16m; 32ML 40 ML
Reduction of outliers from one Summer to the other, related to the increase of ensemble size (more evident for the 5 to 10 increase).
Need to take into account the different statistics for each season (JJA 2003 less rainy than the others).
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Semi-diurnal cycle in COSMO-LEPS scores
BSS score … the higher the better … Performance of the system assessed for 5 different Summers (JJA).
BSS Evident 12-hour cycle in
BSS scores (the same holds for RPSS, while less evident for ROC area scores).
Better performance of the system for “night-time” precipitation, that is for rainfall predicted between 18Z and 6Z (ranges 30-42h, 54-66h, …).
The amplitude of the cycle is somewhat reduced throughout the years and with increasing forecast range.
The bad performance in Summer 2006 is confirmed.
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Score dependence on the domain size (1)
Verification of COSMO-LEPS against synop reports over the MAP D-PHASE area (~ 470 stations; MAPDOM) and the full domain (~ 1500 stations; fulldom):
different statistics of the verification samples; up to now, performance of the system over the 2 domains assessed only for the
last 6 months (March-August 2007). difficult to draw general conclusions
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Score dependence on the domain size (2)
RPSS
RPSS score… the higher the better… (and positive).
ROC area… the higher the better… (and above 0.6).Smoother transitions from month to month in “fulldom” scores.
Slightly better performance of COSMO-LEPS over the MAPDOM, but the signal varies from month to month.
Higher predictability with orographic forcing?
Need to check individual regions and/or to stratify for type of stations.
OUTL
ROC
Outliers percentage … the lower the better.
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Main results
• COSMO-LEPS system runs on a daily basis since November 2002 (6 “failures” in almost 5 years of activity) and it has become a “member-state time-critical application” at ECMWF ( ECMWF operators involved in the suite monitoring).
• COSMO-LEPS products used in EC Projects (e.g. PREVIEW) , field campaigns (e.g. COPS, MAP D-PHASE) and met-ops rooms across COSMO community.
• Nevertheless, positive trends can be identified:• increase in ROC area scores and reduction in outliers percentages;• positive impact of increasing the population from 5 to 10 members (June
2004);• although some deficiency in the skill of the system were identified after the
system upgrades occurred on February 2006 (from 10 to 16 members; from 32 to 40 model levels + EPS upgrade!!!), scores are encouraging throughout 2007.
Time series scores cannot easily disentangle improvements related to COSMO-LEPS itself from those due to better boundaries by
ECMWF EPS.
2 more features:
• marked semi-diurnal cycle in COSMO-LEPS scores (better skill for “night-time” forecasts);
• better scores over the Alpine area rather than over the full domain (to be confirmed).
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
• Verification vs synop: assess performance over the full domain and
consider other scores (Rank histograms, Reliability diagrams,…).
• Implement “weighted” derived probability products:
– weighted ensemble mean;
– weighted ensemble standard deviation.
• Introduce more model perturbations (see COSMO-SREPS approach).
• Verification will be further developed → new variables verified.
Future plans
• Implement EFI index in the operational suite?
• Implement “TIGGE-LAM compliant” COSMO-LEPS which can run
on any domain, on any boundary conditions (relocatable
COSMO-LEPS).
• Develop “hybrid” clustering (considering both ECMWF EPS and
UKMO MOGREPS) ---> work with GRIB2 format!
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Thank you !
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
COSMO-LEPS
16-MEMBER EPS
51-MEMBER EPS
tp > 1mm/24h
tp > 5mm/24h
Average values (boxes 0.5 x 0.5)MAM06
As regards AVERAGE precipitation above these two threshols, the 3 systems have similar performance.
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
ENSEMBLE SIZE REDUCTIONIMPACT EVALUATED ON CASE STUDIES (1)
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
2003082512 Friuli
(+72-+96)
20 mm 100 mm
5 RMs
10 RMs
All 51
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
ENSEMBLE SIZE REDUCTIONIMPACT EVALUATED ON CASE STUDIES (2)
Observed precipitation between 15-11-2002 12UTC and 16-11-2002 12 UTC
Piedmont case
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
2002111212 Piedmont
20 mm 150 mm
5 RMs
10 RMs
All 51
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
COSMO-LEPS
Real time products
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Operational COSMO-LEPS ~ Operational EPS
“Friuli case”S.E. 1535 RMs
The youngestEPS
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
>20mm/24h >50mm/24h
Operational COSMO-LEPS ~ Operational EPS : Friuli case
probability maps – fc. range +96
COSMOLEPS
EPS 51
members
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
old suite new suite
BSS outliers
EVALUATION OF S.E. SIZE (either 51, or 102, or 153) & ENSEMBLE SIZE REDUCTION (either 5 or 10 RMs)
Regarding the 5-RM ensembles results would seem to suggest that the use of just two EPS in the super-ensemble can be a good compromise, permitting to decrease the percentage of outliers significantly with respect of using just 1 EPS but leading only to a small decrease of the skill.
Regarding the impact of the ensemble size, the difference between each 5-member ensemble and the correspondent 10-member ensemble is remarkable.The impact of doubling the ensemble size is almost the same for every configuration and is larger than impact of changing the number of EPSs on which the Cluster Analysis is performed (2 or 3).
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
TEST OF NEW CLUSTERING VARIABLES AND/OR DIFFERENT CLUSTERING INTERVALS
Consider a fixed configuration in terms of ensemble size (10 RMs selected out of 2 EPS sets, 2eps-10rm) and the properties of the “reduced” (10-member) global ensemble in 4 different cases:
OPE: the 10 members are selected like in the operational set-up (clustering variables: z,u,v,q; clustering levels: 500, 700, 850 hPa; clustering times: fc+96h, fc+120h);
D2: like OPE, but clustering times: fc+24h, fc+48h;D3: like OPE, but clustering times: fc+48h, fc+72h;D4: like OPE, but clustering times: fc+72h, fc+96h.BS
S
Brier Skill Score: OPE has slightly better scores at all verification ranges (less evident for ROC area .. not shown);
Outliers percentage: results heavily depends on the verification range.
outliers
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
COSMO-LEPS derived probability products
The generation of the Ensemble Mean (EM) and Ensemble Standard deviation (ES) was implemented and tested for COSMO-LEPS runs in terms of:
Z700 - T850 – MSLP - T2M - UV10M - TP
from fc+0h to fc+132h every 3 hours
Dissemination will start at the end of the month.EM and ES are calculated without considering the weights relative to each integration.
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Why Limited Area Ensemble Prediction?
• Global Ensemble Prediction Systems
– have become extremely important tools to tackle the problem of predictions beyond day 2
– are usually run at a coarser resolution with respect to deterministic global predictions → skill in forecasting intense and localised events is currently still limited.
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Why Limited Area Ensemble Prediction?(2)
As regards high resolution deterministic forecast in the short range, where limited-area models play the major role, a “satisfactory” QPF is still one of the major challenges. The same can be said for other local parameters.
This is due, among other reasons, to the inherently low degree of predictability typical of severe and localised events.
Probabilistic/Ensemble approach is so required also for the short range at higher resolution
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
From Global EPS to LAM EPS
• In the Limited-area ensemble systems, tailored for the short range, perturbations must be already “active” during the first hours of integration
• The characteristic of the LAM ensemble are strongly dependent by the lateral boundaries forcing.
• Due to the “regional” application of these Limited Area Ensembles, methodologies can be different in different geographycal regions.
A pratical consideration:Global EPS ~ Big Centres
Limited Area EPS ~ (also) Relatively Small Centres
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
In the last period the verification package is being developed keeping into account two measure of precipitation:
the cumulative volume of water deployed over a specific region
the rainfall peaks which occur within this region
OBJECTIVE VERIFICATION OF COSMO-LEPS
COSMO observations
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
CLEPS EPS
Verification grid
OBS MASK
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Archiving of C0SMO-LEPS
From July 2005, products are archived on MARS at ECMWFDeterministic run (fc+0h to fc+132h every 3h).
Ensemble Prediction System:16 perturbed forecasts (fc+0h to fc+132h every 3h):
PLEV (500, 700, 850 hPa): Z, RH, T.
SURF: albedo, LCC, MCC, TCC, SW radiation flux, CAPE, hzerocl, snowlmt, mslp, T_2m, Td_2m, TMAX_2m, TMIN_2m, U_10m, V_10m, UVMAX_10m, large-scale rain, convective rain, large-scale snow, TP.
Forecast probability (various intervals and thresholds):SURF: CAPE, hzerocl, TMAX_2m, TMIN_2m, UVMAX_10m, TP, snowfall, showalter index.
Clustering information (population, clustering variable used, …).Derived products (fc+0h to fc+132h every 3h):
ens-mean and ens-spread: Z700, T850, MSLP, T2M, UV10M, TP.
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
COSMO-LEPS
16-MEMBER EPS
51-MEMBER EPS
tp > 1mm/24h
tp > 5mm/24h
Average values boxes 0.5x0.5 deg3 sis
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
COSMO-LEPS
16-MEMBER EPS
tp > 1mm/24h
NOCC=610
NOCC=1195
tp > 5mm/24h
NOCC=2671
tp > 10mm/24h
ave 0.5
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Maximum values boxes 0.5x0.5 deg
tp > 1mm/24h
tp > 5mm/24h
tp > 10mm/24h
COSMO-LEPS
16-MEMBER EPS
51-MEMBER EPS
3 sis
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
COSMO-LEPS
16-MEMBER EPS
tp > 20mm/24hNOCC=227
Average values boxes 0.5x0.5 deg
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Precipitation: average on 1.5 x 1.5 boxes
tp > 10mm/24h tp > 20mm/24h
COSMO-LEPS
5-MEMBER EPS
51-MEMBER EPSSON 2003
ROC area
As regards AVERAGE precipitation above these two threshols, EPS wins.
Worsening due to the ensemble reductionPositive impact of LM downscaling
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
maxima on 1.5 x 1.5 boxes
tp > 20mm/24hCOSMO-LEPS
5-MEMBER EPS
51-MEMBER EPS
tp > 50mm/24h
SON 2003
ROC area
COSMO-LEPS is more skilful than EPS in forecasting correctly high precipitation values over a rather large area.
Number of occurrences: 600 (20 mm threshold) and 150 (50 mm).
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Time series of ROC area for COSMO-LEPS (2)
Area under the curve in the FAR vs HIT diagram. Valuable forecast systems have ROC area values > 0.6
ROC
the positive impact of increasing the ensemble size in 2004 is evident for all thresholds and for different fcst ranges.
poor performance of the system in Spring and Summer 2006, despite system upgrades.
Jun04: 5m 10m
Feb06: 10m 16m; 32ML 40 ML
fc step: 78-90h
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
COSMO-LEPS
16-MEMBER EPS
51-MEMBER EPS
Average values (boxes 0.5 x 0.5 deg)
MAM06
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Time series of TEST OF NEW CLUSTERING VARIABLES AND/OR DIFFERENT CLUSTERING INTERVALS
Consider a fixed configuration in terms of ensemble size (10 RMs selected out of 2 EPS sets, 2eps-10rm) and the properties of the “reduced” (10-member) global ensemble in 4 different cases:
OPE: the 10 members are selected like in the operational set-up (clustering variables: z,u,v,q; clustering levels: 500, 700, 850 hPa; clustering times: fc+96h, fc+120h);
D2: like OPE, but clustering times: fc+24h, fc+48h;D3: like OPE, but clustering times: fc+48h, fc+72h;D4: like OPE, but clustering times: fc+72h, fc+96h.BS
S
Brier Skill Score: OPE has slightly better scores at all verification ranges (less evident for ROC area .. not shown);
Outliers percentage: results heavily depends on the verification range.
outliers
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Main results
• COSMO-LEPS system runs on a daily basis since November 2002 and it has become a “member-state time-critical application” at ECMWF ( partial involvement of ECMWF operators in the suite management).
• Time series scores identify positive trends in the performance of the system:• increase in ROC area scores and reduction in outliers percentages;• positive impact of increasing the population from 5 to 10 members (June
2004);• some deficiency in the skill of the system identified after the system
upgrades occurred on February 2006 (from 10 to 16 members; from 32 to 40 model levels + EPS upgrade!!!) need of further investigation.
• High-resolution verification shows better scores of COSMO-LEPS with respect to EPS in forecasting precipitation maxima within boxes.• No clear impact of the weighting procedure as regards precipitation, BUT COSMO-LEPS ensemble mean for 2m temperature has better skill (lower standard deviation to observations) with weighting the ensemble members according to cluster populations compared to the mean with unweighted members.
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Downscaling
Global Ensemble
LAMNesting
GCM members
LAM members
Total Downscaling
(Brute-Force Approach)
Global Ensemble
LAMNesting
GCM members
LAM members
EnsembleSize
Reduction
Representative members
COSMO
LEPS
Approach
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
COSMO-LEPS forecast for 72h precipitation
by AW (Meteoswiss)
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Probability precipitation > 250mm/72h
C. Frei, MeteoSwiss
by AW (Meteoswiss)
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Standard deviation: 2m temperature 12UTC
Forecast range (h) Forecast range (h)
DJF 2004/2005 JJA 2005
STD2 = RMS2 – BIAS2 uw: unweighted / w: weighted according to cluster population
0C 0C
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
COSMO-LEPS NW
COSMO-LEPS W
tp > 1mm/24h
COSMO-LEPS weighting procedure
maximum values (boxes 0.5x0.5 deg)
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Outliers: time series + seasonal scoresHow many times the analysis is out of the forecast interval spanned by the ensemble
members. … the lower the better … Performance of the system assessed for 4 different Autumns.
OUTL
Continuous reduction of outliers from one year to the other, also related to the increase of ensemble size (more evident for the 5 to 10 increase, rather than from 10 to 16).
Need to take into account the different statistics for each season (SON 2006 less rainy than the others).
Jun04: 5m 10m Feb06: 10m 16m; 32ML 40 ML
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
COSMO-LEPS NW
COSMO-LEPS W
tp > 1mm/24h
COSMO-LEPS weighting
procedure
maximum values (0.5x0.5 boxes)
average values (0.5x0.5 boxes)
tp > 1mm/24h
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Synop-Verification: COSMO-LEPS & aLMo
aLM
o do
mai
n
Common verification domain,about 1000 synop stations
aLMo: - deterministic 7 km model (45 levels) of MeteoSwiss for 72h forecasts - same code (LM) as COSMO-LEPS
by AW (Meteoswiss)
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Standard deviation: 2m temperature 00UTC
Forecast range (h) Forecast range (h)
DJF 2004/2005 JJA 2005
w
0C 0C
STD2 = RMS2 – BIAS2 uw: unweighted / w: weighted according to cluster population
by AW (Meteoswiss)
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Case study: flood event in Switzerland in August 2005
Photos:Tages-Anzeiger
Courtesy of Andrè Walser (Meteoswiss)
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Synoptic overview: 22 August 2005
Temperature 850 hPa and geopotential 500 hPa:
2º
18º
10º
by AW (Meteoswiss)
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Total precipitation over 3 days (20/8 – 23/8)
About 400 stations, precipitation sum locally over 300 mm!
(06 - 06 UTC)
C. Frei, MeteoSwiss
by AW (Meteoswiss)
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
COSMO-LEPS forecast for 72h precipitation
by AW (Meteoswiss)
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
COSMO-LEPS vs ECMWF EPS probability of 72h cumulated precipitation
by AW (Meteoswiss)
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Probability precipitation > 100mm/72h
C. Frei, MeteoSwiss
by AW (Meteoswiss)
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Probability precipitation > 100mm/72h
C. Frei, MeteoSwiss
by AW (Meteoswiss)
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
– high-resolution network (COSMO-LEPS vs ECMWF EPS)
Objective verification of COSMO-LEPS
Main features:
variable: 24h cumulated precip (06-06
UTC);
period: MAM 2006;
region: Germany, Switzerland, Italy;
method: BOXES (0.5 x 0.5);
obs: COSMO network (2500 stations
x day);
fcst ranges: 18-42h, 42-66h, 66-90h, 90-114h;
thresholds: 1, 5, 10, 20, 50 mm/24h;
systems:
- COSMO-LEPS (16m, 10 km, 40
ML)
- reduced EPS (16m, 50 km, 60 ML)
- full EPS (51m, 50 km, 60 ML)
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
Verification of the distributions
Station observation
Grid point forecast
The verification has been made in terms of:
• Average value
• Maximum value
• 50th percentile (Median)
• 90th percentile
in a box
two measures of precipitation: the cumulative volume of
water deployed over a specific region;
the rainfall peaks occurring within the same region.
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
tp > 1mm/24h
tp > 5mm/24h
Maximum values (boxes 0.5 x 0.5)
COSMO-LEPS
16-MEMBER EPS
51-MEMBER EPS
tp > 10mm/24h
NOCC=3681
NOCC=1592
NOCC=2492
MAM06
A.Montani; The COSMO-LEPS system: 5-year milestoneCOSMO meeting, Athens, 18-21 September 2007
COSMO-LEPS
16-MEMBER EPS
51-MEMBER EPS
Maximum values (boxes 0.5 x 0.5)
MAM06
COSMO-LEPS is more skilful than EPS in forecasting correctly high precipitation values over 0.5 x 0.5 boxes.