regional production, quarterly report on the daily ... · air quality species and analyses of the...

ECMWF COPERNICUS REPORT

Copernicus Atmosphere Monitoring Service

Regional Production, Quarterly report on the daily analyses and forecasts activities, and verification of the EURAD-IM performances

June – July – August 2018

Issued by: METEO-FRANCE / G. Collin

Date: 21/01/2019

Ref: CAMS50_2018SC1_D5.2-3.1.EURAD-IM-2018JJA_201901_Production_Report_v2

This document has been produced in the context of the Copernicus Atmosphere Monitoring Service (CAMS). The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, operator of CAMS on behalf of the European Union (Delegation Agreement signed on 11/11/2014). All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose. The user thereof uses the information at its sole risk and liability. For the avoidance of all doubts, the European Commission and the European Centre for Medium-Range Weather Forecasts has no liability in respect of this document, which is merely representing the authors view.


CAMS50_2018SC1 – EURAD-IM Production report – JJA2018 of 23

Contributors

FZJ-IEK8 E. Friese H. Elbern

METEO-FRANCE M. Pithon M. Plu V. Petiot G. Collin N. Assar INERIS A. Colette



Table of Contents

1. The EURAD-IM model 6

1.1 Product portfolio 6 1.2 Availability statistics 6 1.2.1 Indicators 6 1.2.2 Problems encountered 7 1.3 Use of observations for data assimilation 7 1.3.1 Use of observations – June 2018 8 1.3.2 Use of observations – July 2018 9 1.3.3 Use of observations – August 2018 10

2. Verification report 11

2.1 Verification of NRT forecasts 11 2.1.1 EURAD-IM forecasts: ozone skill scores against data from representative sites 12 2.1.2 EURAD-IM forecasts: NO2 skill scores against data from representative sites 13 2.1.3 EURAD-IM forecasts: PM10 skill scores against data from representative sites 14 2.1.4 EURAD-IM forecasts: PM2.5 skill scores against data from representative sites 15 2.2 Verification of NRT analyses 16 2.2.1 EURAD-IM analyses: ozone skill scores against data from representative sites 17 2.2.2 EURAD-IM analyses: NO2 skill scores against data from representative sites 18 2.2.3 EURAD-IM analyses: PM10 skill scores against data from representative sites 19 2.2.4 EURAD-IM analyses: PM2.5 skill scores against data from representative sites 20 2.3 Analysis of the EURAD-IM performances over the quarter 21



Executive summary The Copernicus Atmosphere Monitoring Service (CAMS, atmosphere.copernicus.eu/) is establishing the core global and regional atmospheric environmental service delivered as a component of Europe's Copernicus programme. The Regional forecasting service provides daily 4-day forecasts of the main air quality species and analyses of the day before, from 7 state-of-the-art atmospheric chemistry models and from the median ENSEMBLE calculated from the 7 model forecasts. The Regional service also provides posteriori re-analyses using the latest validated observation dataset available for assimilation. 2 other individual models will soon join operational production. This report covers the EURAD-IM Near Real Time Production (NRT), for the quarterly period ending August 31st, 2018. Verification is done against in-situ surface observations; these are described in the D50.1.1.2 report covering the same period. Verification of analyses is done against non-assimilated observations. Once again during this quarter, production reliability was excellent for both forecasts and analyses with all availability indicators at 100%. These excellent results confirm the stability of the EURAD-IM operational production. For the JJA2018 period, the RMSE of the EURAD-IM forecast is 6 µg/m3 above the target value for the daily maximum of O3 and well below the target value for the daily maximum of NO2 (12 µg/m3) and the daily mean of PM10 (12 µg/m3). Compared to JJA2017, the EURAD-IM prediction of the daily O3 maximum is worse (about 4 µg/m3), but the prediction of the NO2 daily maximum is about 2 µg/m3

more accurate than for JJA2017. The prediction of PM daily mean concentration did not significantly change compared to JJA2017. The skill scores of the ENSEMBLE median are generally better than those of the EURAD-IM forecast. The RMSE of the EURAD-IM analysis is about 1 µg/m3 above the target value for the daily maximum of O3 and well below the target value for the daily maximum of NO2 (11 µg/m3) and the daily mean of PM10 (14 µg/m3). Compared to JJA2017, the EURAD-IM analysis of the daily O3 maximum is worse (about 3 µg/m3), but the analysis of the NO2 daily maximum is about 1.5 µg/m3 more accurate than for JJA2017. The analysis of PM daily mean concentration did not significantly change compared to JJA2017. With the exception of the MMB of PM2.5 the skill scores of the EURAD-IM analysis are clearly better than the corresponding values for the forecast. In case of the most skill scores and for the most situations, the ENSEMBLE analysis performs better than the EURAD-IM analysis. In September 2017, an update of the list of stations used for the verification of forecasts and of analyses was done, and a one-hour time shift was applied to the hour of validity of observations. Thus, if any improvement of the EURAD-IM performance since last year, it is hardly possible to disentangle it from the effect of the new observation dataset that is used for assimilation and for evaluation. Consistently with this change of dataset, we note in particular an improvement of RMSE for all pollutants, an increase of PM bias (from negative to less negative or to positive), and a degradation of NO2 correlation.



1. The EURAD-IM model

1.1 Product portfolio

Item Forecast Analysis Description Forecast at surface, 50m, 250m,

500m, 1000m, 2000m, 3000m, 5000m above ground

Analysis at the surface, 50m, 250m, 500m, 1000m, 2000m, 3000m, 5000m above ground

Available for users at 4:00 UTC 10:30 UTC for the day before Species O3, NO2, CO, SO2, PM2.5, PM10,

NO, NH3, NMVOC, PANs, Birch, olive and grass pollen at surface during season

O3, NO2, CO, SO2, PM2.5, PM10, NO*, NH3*, NMVOC*, PANS*

Time span 0-96h, hourly 0-24h for the day before, hourly * Non-assimilated species

1.2 Availability statistics The statistics below describe the ratio of days for which the EURAD-IM model outputs were available on time to be included in the ENSEMBLE fields (analyses and forecasts) that are computed at Meteo-France. They are based on the following schedule for the provision at Meteo-France of: • Forecasts data before: 05:30 UTC for D0-D1 (up to 48h), 07:30 UTC for D2-D3 (from 49h to 96h); • Analyses data: before 11:00 UTC. These schedules were set to meet the IT requirements for ENSEMBLE products (no later than 8 UTC for 0-48h, 10 UTC for 49-96h and 12 UTC for analyses).

1.2.1 Indicators

Availability_model_Forecast Quarterly basis

D0: 100% D1: 100% D2: 100% D3: 100%

Availability_model_Analysis Quarterly basis

D: 100%



1.2.2 Problems encountered No incident was encountered by the EURAD-IM production system this quarter.

1.3 Use of observations for data assimilation Please see the next three pages.



1.3.1 Use of observations – June 2018

Day O3 NO2 NO SO2 CO PM10 PM2.5 1 12299 9540 0 4097 3960 6390 2003 2 12394 10195 0 4123 3775 6422 1979 3 12211 9948 0 4059 3668 6376 1973 4 11819 10037 0 4091 3898 6419 1978 5 12294 10317 0 4035 3774 6511 2028 6 12408 10243 0 4232 3686 6412 2054 7 12392 10570 0 4276 3750 6349 1985 8 12648 9825 0 4444 3681 6350 2077 9 12769 10343 0 4539 3685 6298 2042

10 12567 10203 0 4413 3577 6132 2006 11 12346 10190 0 4118 3588 6269 2046 12 12415 10210 0 3912 3718 6336 2025 13 12003 10705 0 4173 3575 6400 1919 14 11804 10633 0 4257 3599 6439 2046 15 12349 9507 0 4312 3882 6416 1956 16 12390 10375 0 4600 3671 6320 1996 17 12364 10147 0 4510 3453 6293 1940 18 12440 10195 0 4157 3698 6269 1838 19 12139 10643 0 4042 3614 6292 1785 20 12071 10694 0 4215 3755 6320 1767 21 12250 10682 0 4169 3583 6298 1800 22 8004 7384 0 2550 2789 3124 960 23 6637 6090 0 2026 2650 2118 801 24 10932 10271 0 3826 3252 5944 1915 25 9762 9802 0 3657 3467 6121 1887 26 9813 9901 0 3982 3309 6134 1975 27 9971 10233 0 3969 3495 6222 1903 28 9495 9740 0 3750 3437 5711 1712 29 10209 9705 0 4241 3433 6309 1888 30 10416 10113 0 4196 3368 6233 1898



1.3.2 Use of observations – July 2018


10 10020 10147 0 3734 3273 6302 1864 11 10265 10429 0 3775 3280 6361 1996 12 10119 10136 0 3699 3438 6232 2004 13 10168 10365 0 4036 3439 6309 2028 14 10063 10135 0 3771 3225 6197 1951 15 9993 9961 0 3939 3185 6180 1900 16 12792 10307 0 4341 3758 6366 1953 17 12508 10406 0 4300 3724 6218 1900 18 12480 10393 0 4287 3789 6234 1887 19 12643 10404 0 4513 3679 6231 1963 20 12531 10124 0 4409 3780 6349 2042 21 12159 10202 0 4574 3425 6193 2052 22 12354 10162 0 4281 3673 6262 1995 23 12596 10106 0 4109 3692 6303 2015 24 12598 10362 0 4418 3759 6231 2143 25 12822 10561 0 4358 3815 6271 2173 26 11771 11095 0 4187 3442 6148 2046 27 12669 10574 0 4839 3885 6156 1976 28 12003 11037 0 4449 3709 6097 1985 29 12405 10721 0 4460 3579 6176 1848 30 12674 10584 0 4680 3739 6313 1867 31 12779 10413 0 4435 3716 6311 1893



1.3.3 Use of observations – August 2018


10 12272 10414 0 3947 3590 6101 1826 11 8790 8457 0 2986 3041 3476 1034 12 6633 6471 0 2423 2619 1969 598 13 12423 10608 0 4215 3603 6179 1767 14 12324 10148 0 3806 3474 6091 1921 15 0 0 0 0 0 0 0 16 12352 10676 0 4081 3489 6331 1990 17 12266 10201 0 4324 3553 6261 2023 18 12354 10973 0 4086 3617 6219 1991 19 12099 10920 0 4309 3459 6190 1971 20 12409 10028 0 4047 3687 6076 1896 21 12737 10237 0 4216 3630 6292 1970 22 12950 10573 0 4689 3895 6499 2106 23 11665 10358 0 4181 3737 5703 1835 24 12854 10415 0 4170 3735 6324 2029 25 12624 10022 0 3989 3659 6066 1921 26 12472 9771 0 3863 3508 6052 1848 27 12693 10266 0 4132 3648 6316 1977 28 12884 10380 0 4134 3651 6481 2047 29 12840 10373 0 4361 3743 6299 2218 30 12710 10309 0 4010 3718 6482 2089 31 12311 10489 0 3955 3824 6167 1929



2. Verification report The EURAD-IM skill scores are successively presented for 4 pollutants: ozone, NO2, PM10 and PM2.5. The skill is shown for the entire forecast horizon from 0 to 96h (hourly values), allowing to evaluate the entire diurnal cycle and the evolution of performance from day 0 to day 3. The forecasts and the analyses cover a large European domain (25°W-45°E, 30°N-70°N). The statistical scores that are reported are the root-mean-square error, the modified mean bias and the correlation. The surface observations that are acquired by Meteo-France and used for verification are described in D50.1.1.2 covering the same period. In September 2017, an update of the list of stations used for the verification of forecasts and of analyses was done. This update intends to improve the representativeness of the observations used by the Regional CAMS. Besides, the hour of validity of observations was checked and corrected by a one-hour time shift. The impact of these changes on scores was documented in a previous report (D50.5.5.1 Update on the best use of EIONET observation database for assimilation and verification of the Regional NRT Production). As a consequence, the yearly evolution of scores is interpreted in regards of this update.

2.1 Verification of NRT forecasts The following figures present, for each pollutant (ozone, NO2, PM10 and PM2.5): • In the upper-left panel, the root-mean square error of daily maximum (for ozone and NO2) or of

daily mean (PM10 and PM2.5) for the first-day forecasts with regard to surface observations, for every quarter since DJF2014/2015, a target reference value is indicated as an orange line;

• In the upper-right panel, the root-mean square error of pollutant concentration forecasts with regard to surface observations as a function of forecast term;

• In the lower-left panel, the modified mean bias of pollutant concentration forecasts with regard to surface observations as a function of forecast term;

• In the lower-right panel, the correlation of pollutant concentration forecasts with regard to surface observations as a function of forecast term.

The graphics show the performance of EURAD-IM (black curves) and of the ENSEMBLE (blue curves).



2.1.1 EURAD-IM forecasts: ozone skill scores against data from representative sites

The RMSE of the daily maximum of the EURAD-IM O3 forecast is 6 µg/m3 above the target value and 4 µg/m3 larger than the RMSE of the ENSEMBLE median. Compared to JJA2017, the RMSE of the EURAD-IM forecast is higher in the afternoon. The RMSE and the MMB of the EURAD-IM ozone forecast have a strong maximum at about 06:00 UTC. Compared to the ENSEMBLE, the daily cycles of booth skill scores are more pronounced for EURAD-IM. The MMB is generally positive with the exception of a negative bias of the ozone maximum in the afternoon. The correlation coefficient of the EURAD-IM forecast is low compared to the ENSEMBLE (about 0.1 to 0.2 smaller) and it decreases slightly with increasing forecast time.



2.1.2 EURAD-IM forecasts: NO2 skill scores against data from representative sites

The RMSE of the daily maximum of the EURAD-IM NO2 forecast is about 12 µg/m3 below the target value, about 1 µg/m3 larger than the RMSE of the ENSEMBLE and 2 µg/m3 smaller than for JJA2017. As usual the daily cycle of the RMSE of NO2 shows a characteristic double peak, which is mainly caused by high negative model biases at the morning and afternoon rush hours. The MMB of the EURAD-IM forecast is negative in general and 0.2 to 0.4 µg/m3 smaller compared to the ENSEMBLE. There is a phase shift of about 1 hour between the daily cycles of the MMB of the ENSEMBLE and of the EURAD-IM forecast. The correlation coefficient decreases with increasing forecast time. Compared to the ENSEMBLE, the correlation coefficient of the EURAD-IM forecast is slightly lower.



2.1.3 EURAD-IM forecasts: PM10 skill scores against data from representative sites

The RMSE of the daily mean of the EURAD-IM PM10 forecast is well below the target value (12 µg/m3) and approximately at the same level as in JJA2017. The RMSE of the EURAD-IM forecast exhibits a maximum in the forenoon and its value and daily cycle are comparable to the ENSEMBLE. The MMB of the EURAD_IM forecast is generally still negative. Compared to the ENSEMBLE, the daily cycle of the MMB has larger amplitude with lower values around midnight. The correlation coefficient of the EURAD-IM PM10 forecast is about 0.1 lower than that of the ENSEMBLE median. It shows a double peak daily cycle with weak amplitude and decreases strongly with increasing forecast time.



2.1.4 EURAD-IM forecasts: PM2.5 skill scores against data from representative sites

The RMSE of the EURAD-IM forecast of the PM2.5 daily mean has approximately the same value as in JJA2017. Compared to the ENSEMBLE median, the amplitude of the daily cycles of RMSE and MMB are large with maximum values at night. In contrast to the ENSEMBLE, the MMB of the EURAD-IM PM10 forecast is generally positive. The correlation coefficient of the EURAD-IM forecast is 0.1 to 0.15 smaller than that of the ENSEMBLE and decreases stronger with increasing forecast time.



2.2 Verification of NRT analyses The following figures present, for each pollutant (ozone, NO2, PM10 and PM2.5): • In the upper-left panel, the root-mean square error of daily maximum (for ozone and NO2) or of

daily mean (PM10) for the analyses (solid line) and for the first-day forecasts (dashed line) with regard to surface observations, for every quarter since DJF2014/2015, a target reference value is indicated as an orange line;

• In the upper-right panel, the root-mean square error of pollutant concentration of the analyses (solid line) and of the first-day forecasts (dashed line), with regard to surface observations as a function of forecast term;

• In the lower-left panel, the modified mean bias of pollutant concentration forecasts of the analyses (solid line) and of the first-day forecasts (dashed line), with regard to surface observations as a function of forecast term;

• In the lower-right panel, the correlation of pollutant concentration of the analyses (solid line) and of the first-day forecasts (dashed line), with regard to surface observations as a function of forecast term.

The graphics show the performances of EURAD-IM (black curves) and of the ENSEMBLE (blue curves). The superposition of the analysis scores (solid lines) and of the forecast scores (dashed lines) computed over the same observation dataset is helpful to assess the added value of data assimilation.



2.2.1 EURAD-IM analyses: ozone skill scores against data from representative sites

The RMSE of the EURAD-IM analysis of the daily O3 maximum is about 2 µg/m3 above the target value. The skill scores of the EURAD-IM O3 analysis are clearly better than the forecast skill scores. There are not any large differences between the diurnal cycles of the skill scores of the EURAD-IM forecast and analysis. The performance of the ENSEMBLE analysis is generally better than that of EURAD-IM: the RMSE is about 3 µg/m3 smaller, the MMB up to 0.1 µg/m3 smaller, and the correlation coefficient about 0.15 larger. O3 in situ observations are assimilated in EURAD-IM.



2.2.2 EURAD-IM analyses: NO2 skill scores against data from representative sites

The RMSE of the EURAD-IM analysis of the daily NO2 maximum is about 11 µg/m3 below the target value and about 1.5 µg/m3 smaller than in JJA2017. Skill scores of the EURAD-IM NO2 analysis are generally better than the forecast skill scores: the RMSE is 0.5 to 2 µg/m3 smaller, the MMB up to 0.3 µg/m3 smaller, and the correlation coefficient is 0.05 to 0.1 larger. Compared to the forecast, the amplitude of the daily cycle of RMSE and MMB of the analysis is smaller, the amplitude of the correlation coefficient slightly larger. The daily cycle of the MMB shows a small phase shift of about 1 hour between EURAD-IM and the ENSEMBLE. The performance of the ENSEMBLE analysis is generally better compared to EURAD-IM. NO2 in situ observations are assimilated in EURAD-IM.



2.2.3 EURAD-IM analyses: PM10 skill scores against data from representative sites

The RMSE of the EURAD-IM analysis of the daily mean of PM10 is 14 µg/m3 below the target value and approximately at the same level as in JJA2017. Skill scores of the EURAD-IM PM10 analysis are clearly better than the forecast skill scores: the RMSE is about 2 µg/m3 smaller, the MMB up to 0.2 µg/m3

smaller, and the correlation coefficient is about 0.15 larger. The relatively small difference between the MMB of the forecast and analysis up to about 06:00 UTC is probably caused by the initialization of the forecast with analysis results for the previous day. Apart from that there are not any large differences between the diurnal cycles of the skill scores of the EURAD-IM forecast and analysis. Also the diurnal cycles of the ENSEMBLE median and of the EURAD-IM analysis are comparable. The performance of the ENSEMBLE analysis is generally better than that of EURAD-IM: the RMSE is about 1 µg/m3 smaller, the MMB up to 0.15 µg/m3 smaller, and the correlation coefficient about 0.1 larger. PM10 in situ observations are assimilated in EURAD-IM.



2.2.4 EURAD-IM analyses: PM2.5 skill scores against data from representative sites

Compared to JJA2017, the RMSE of the EURAD-IM analysis of the PM2.5 daily mean is slightly larger (about 1 µg/m3). RMSE and correlation coefficient of the EURAD-IM PM2.5 analysis are clearly better than the corresponding values for the forecast: the RMSE is 1 to 3 µg/m3 smaller and the correlation coefficient about 0.2 larger. The MMB of the EURAD-IM analysis is only slightly smaller than the forecast MMB. This is probably caused by the correction of the negative PM10 bias of the EURAD-IM forecast. Compared to the forecast the diurnal cycles of the RMSE and MMB of the analysis are smaller. The diurnal cycles of the ENSEMBLE median and of the EURAD-IM analysis are comparable. The RMSE of the ENSEMBLE analysis is up to 1.5 µg/m3 smaller than the RMSE of the EURAD-IM analysis and the correlation coefficient is 0.05 to 0.1 µg/m3 smaller. The MMB of the EURAD-IM analysis and of the ENSEMBLE is comparable. PM2.5 in situ observations are assimilated in EURAD-IM.



2.3 Analysis of the EURAD-IM performances over the quarter For the period JJA2018 the RMSE of the EURAD-IM forecast is 6 µg/m3 above the target value for the daily maximum of O3 and well below the target value for the daily maximum of NO2 (12 µg/m3) and the daily mean of PM10 (12 µg/m3). Compared to JJA2017, the EURAD-IM prediction of the daily O3 maximum is worse (about 4 µg/m3), but the prediction of the NO2 daily maximum is about 2 µg/m3

more accurate than for JJA2017. The prediction of PM daily mean concentration did not significantly change compared to JJA2017. With the exception of an underestimation of the daily maximum in the early afternoon, O3 surface concentrations are generally overestimated by the EURAD-IM forecast. NO2 and PM10 are generally underestimated. The positive MMB of PM2.5 at the first forecast day may be caused by the initialization of EURAD-IM with a PM analysis for the previous day. Like for previous periods some skill scores show a significant degradation with increasing forecast time, especially the correlation coefficient. The degradation is more pronounced for particulate matter than for the gaseous air pollutants. As usual the daily cycle of the RMSE of NO2 show a characteristic double peak, which is mainly caused by high negative model biases at the morning and afternoon rush hours. In the morning rush hours the NO2 under prediction is correlated with an O3 over prediction. The skill scores of the ENSEMBLE median are generally better than those of the EURAD-IM forecast. The RMSE of the EURAD-IM analysis is about 1 µg/m3 above the target value for the daily maximum of O3 and well below the target value for the daily maximum of NO2 (11 µg/m3) and the daily mean of PM10 (14 µg/m3). Compared to JJA2017, the EURAD-IM analysis of the daily O3 maximum is worse (about 3 µg/m3), but the analysis of the NO2 daily maximum is about 1.5 µg/m3 more accurate than for JJA2017. The analysis of PM daily mean concentration did not significantly change compared to JJA2017. With the exception of the MMB of PM2.5 the skill scores of the EURAD-IM analysis are clearly better than the corresponding values for the forecast. The MMB of the EURAD-IM PM2.5 analysis is generally positive and around noon even larger than the forecast MMB. This is probably caused by the correction of the negative PM10 bias of the EURAD-IM forecast. In case of the most skill scores and for the most situations the ENSEMBLE analysis performs better than the EURAD-IM analysis. In contrast to the other pollutants, the MMB of the EURAD-IM PM2.5 analysis is slightly smaller than the MMB of the ENSEMBLE (up to 0.1 µg/m3).


atmosphere.copernicus.eu copernicus.eu ecmwf.int

ECMWF - Shinfield Park, Reading RG2 9AX, UK Contact: [email protected]

regional production, quarterly report on the daily ... · air quality species and analyses of the...

Documents