Towards an Ensemble Forecast

Air Quality System for New York StateMichael Erickson1, Brian A. Colle1, Christian Hogrefe2,3, Prakash Doraiswamy3, Kenneth

Demerjian3, Winston Hao2, Mark Beauharnois3, Jia-Yeong Ku2, and Gopal Sistla2

1 School of Marine and Atmospheric Sciences, Stony Brook University, Stony Brook, NY2 New York State Department of Environmental Conservation, Albany, NY

3 Atmospheric Sciences Research Center, State University of New York at Albany, Albany, NY

Motivations and Goals

- Project Goal: Develop an air quality ensemble forecast system to aid operational

forecasters for New York State.

- Motivation: Could errors in the atmospheric models impact air quality forecast

simulations? Can these errors be corrected via post-processing?

- Goal of this talk: Evaluate the air quality models (AQM) and Stony Brook (SBU)

ensemble with a focus on similar biases and errors within each ensemble.

- Future: Use a post-processing technique

called Bayesian Model Averaging (BMA) to

improve the deterministic and probabilistic

forecasts within the ensembles.

Air Quality Model (AQM) Ensemble: Air Quality Model (AQM) Ensemble:

CAMx, CAMQCAMx, CAMQEmissions Inventory:Emissions Inventory:

NYSDEC, EPANYSDEC, EPA

Ensemble of Air Quality Model Ensemble of Air Quality Model

ForecastsForecasts

Ensemble Air Quality Model Flowchart

Atmospheric Model Ensemble: SBU, Atmospheric Model Ensemble: SBU,

NCEP NAM, ASRC, NYSDECNCEP NAM, ASRC, NYSDEC

AQM Operational Ensemble Members

*Currently two SBU members are run in the operational AQM ensemble. Retrospective simulations

used all SBU members except those with the Ferrier microphysics.

**ASRC model was not run in the retrospective simulations.

Member Name

Met. Emis. Inv. AQM Grid Res

Initial-ize

Start Date

NCEP_12z WRF-NMM

EPA CMAQv4.6

12-km 12z Summer 2004; Winter 2004-2005; everyday since June 2005

NCEP_00z WRF-NMM

EPA CMAQv4.6

12-km 00z May 2008

SBU* MM5/WRF

NYSDEC CMAQv4.6

36-km, 12-km

00z June 2008

NYSDEC_3x WRF-NMM

NYSDEC CMAQv4.6

12-km 00z November 2008

ASRC** WRF-ARW

NYSDEC CAMxv4.5.1

12-km 00z March 2009

Synoptic Setup8-hr Max. Ozone Daily Total PM 2.5

150100

7040302010

0

10080604020

0

1-hr max PM 2.5AQI Categories1-hr Max. Ozone

50

40

30

20

10

0

130

115

100

85

70

55

30

Operational AQM Example - 8/18/2009ASRC Member - http://asrc.albany.edu/research/aqf/aqfms/camx/mfb.php

Data and Methods Retrospective simulations of particulate matter 2.5 and ozone were verified over following time periods:

- June 4, 2008 – July 22, 2008- December 1, 2008 – February 28, 2009

Regions 1, 2, and 7 were selected to represent coastal, urban and inland New York, respectively.

AQM output was compared against daily 8-hr maximum ozone and 24-hr average PM 2.5 model predictions from the AIRNOW database and official NYSDEC forecasts.

To elucidate potential error sources in the AQM ensemble, the SBU 10-m wind speed and 2-m temperature were verified with ASOS observations over the same time period.

NYSERDA Regions

SBU Ensemble Domain

AQM Retrospective Simulations SBU Ensemble Members

• F2 and F9 were used to drive CMAQ forecasts each day since June 1, 2008. They were selected based on temperature and wind verification results for summer 2007 and operational considerations.

• Two additional SBU members use the Ferrier microphysics scheme that is currently not compatible with CMAQ.

Name Model Cloud PBL Radiation Microphysics Initialization

F1 MM5 BM MY CCM2 Simple Ice GFS

F2 MM5 Grell MRF CloudRad Simple Ice WRF-NMM

F3 MM5 Grell MY CloudRad Reisner2 WRF-NMM

F5 MM5 Grell Blackadar CCM2 Simple Ice NOGAPS

F6 MM5 KF2 MY CCM2 Simple Ice CMC

F7 MM5 KF2 MRF CloudRad Reisner2 GFS

F8 WRF KF2 MY RRTM WSM3 CMC

F9 WRF BM MY RRTM WSM3 WRF-NMM

F10 WRF KF2 MY RRTM WSM3 GFS

F13 MM5 Grell Blackadar CCM2 Simple Ice GFS

F14 WRF BM YSU RRTM WSM3 NOGAPS

F15 WRF KFE MY RRTM Thompson GFS

Ozone Retrospective SimulationsTime Series – 6/4/08 to 7/22/08

•Model simulations generally track observations (in red) well.

10080604020

0

10080604020

0

10080604020

0

NCEP NAM 12zNCEP NAM 00zNYSDEC 3x SBU MM5 BMMY-GFSSBU MM5 GRMRF-NAMSBU MM5 GRMY-NAMSBU MM5 GRBK-NGPSSBU MM5 KFMY-CMCSBU MM5 KFMRF-GFS

SBU WRF KFMY-CMCSBU WRF BMMY-NAMSBU WRF KFMY-GFSSBU MM5 GRBK-GFSSBU WRF BMYSU-NGPSSBU WRF KFMY-GFSEnsemble AvgEnsemble MedianDEC Forecast

Ozone Retrospective SimulationsBias and RMSE – 6/4/08 to 7/22/08

4

0

-4

-8

4

0

-4

-8

4

0

-4

-8

12

8

4

0

12

8

4

0

12

8

4

0

•Ozone is underpredicted by SBU MM5 members and overpredicted by most remaining models.

•RMSE varies between members, with the ensemble mean/median outperforming individual members.

NCEP NAM 12zNCEP NAM 00zNYSDEC 3x SBU MM5 BMMY-GFSSBU MM5 GRMRF-NAMSBU MM5 GRMY-NAMSBU MM5 GRBK-NGPSSBU MM5 KFMY-CMCSBU MM5 KFMRF-GFS

SBU WRF KFMY-CMCSBU WRF BMMY-NAMSBU WRF KFMY-GFSSBU MM5 GRBK-GFSSBU WRF BMYSU-NGPSSBU WRF KFMY-GFSEnsemble AvgEnsemble MedianDEC Forecast

MM5

WRF Mean

PM 2.5 Retrospective SimulationsTime Series – 12/1/08 to 2/28/09

•Model simulations generally track observations (in red) well.

100

80

60

40

20

0

100

80

60

40

20

0

100

80

60

40

20

0

NCEP NAM 12zNCEP NAM 00zNYSDEC 3x SBU MM5 BMMY-GFSSBU MM5 GRMRF-NAMSBU MM5 GRMY-NAMSBU MM5 GRBK-NGPSSBU MM5 KFMY-CMCSBU MM5 KFMRF-GFS

SBU WRF KFMY-CMCSBU WRF BMMY-NAMSBU WRF KFMY-GFSSBU MM5 GRBK-GFSSBU WRF BMYSU-NGPSSBU WRF KFMY-GFSEnsemble AvgEnsemble MedianDEC Forecast

PM 2.5 Retrospective SimulationsBias and RMSE – 12/1/08 to 2/28/09

10

5

0

-5

10

5

0

-5

10

5

0

-5

15

10

5

0

15

10

5

0

15

10

5

0

•PM is overpredicted for region 2 but underpredicted for region 7 and all other inland stations (not shown).

•NCEP members exhibit the least amount of bias overall.

•The WRF SBU members exhibit greater negative bias than the MM5 SBU.

NCEP NAM 12zNCEP NAM 00zNYSDEC 3x SBU MM5 BMMY-GFSSBU MM5 GRMRF-NAMSBU MM5 GRMY-NAMSBU MM5 GRBK-NGPSSBU MM5 KFMY-CMCSBU MM5 KFMRF-GFS

SBU WRF KFMY-CMCSBU WRF BMMY-NAMSBU WRF KFMY-GFSSBU MM5 GRBK-GFSSBU WRF BMYSU-NGPSSBU WRF KFMY-GFSEnsemble AvgEnsemble MedianDEC Forecast

MM5

WRF

Mean

•Ozone and PM forecasts are “L” shaped (biased) or “U” shaped (underdispersed).

•Biases and dispersion issues have also been noted in the SBU ensemble and may be negatively affecting the AQM.

•Therefore it is important to verify the SBU ensemble in juxtaposition with the AQM.

Retrospective Simulations - Rank Histograms

Winter Particulate Matter Summer Ozone

SBU/AQM Ensemble Comparison – TemperatureOzone and Bias – 6/4/08 to 7/22/08

SBU Ensemble AQI Ensemble •The cooler, shallower and cloudier simulated PBL in the MM5 MY scheme is likely resulting in lower model ozone.

•This affect may be offset in one MY member by the KF convective scheme, which has been shown to decrease cloudiness and increase simulated ozone. (Tao et al. 2008). •The MYJ WRF members have greater ozone concentrations than MY MM5, which could be the result of a higher PBL growth within the MYJ scheme. (Zielonka et al. 2008).

oC

oC

oC

SBU/AQM Ensemble Comparison – TemperaturePM 2.5 and Bias – 12/1/08 to 2/28/09

SBU Ensemble AQI Ensemble•The MM5 members using the Reisner microphysics have more PM than those using Simple Ice. PM sensitivity to cloud microphysics schemes have also been noted in Meij et al. 2009.

•Lower WRF PM concentrations have been noted compared to MM5 (Meij et al 2009) due to the increase of vertical mixing within WRF caused by warmer surface temperatures.

oC

oC

oC

SBU/AQM Ensemble Comparison – Rank Histogram

Summer Ozone AQMRegion 7

Winter PM 2.5 AQMRegion 7

Winter Wind SBURegion 7

Summer Temp. SBURegion 7

•After bias correction, the SBU ensemble is underdispersed for temperature and wind speed in all regions.

•The AQI ensemble also appears to be underdispersed in the absence of biases, suggesting that a lack of variability in atmospheric forecasts could affect the air quality models.

•Post-processing techniques, such as Bayesian Model Averaging (BMA), could help correct this lack of variability in ensemble forecasting.

Post-Processing - Bayesian Model Averaging Bayesian Model Averaging (BMA, Raftery et al. 2005) has been shown to correct some model deficiencies associated with reliability and dispersion.

BMA creates a probability density function (PDF) for each ensemble member depending on the uncertainty in the model forecast and weights the result based on its performance and uniqueness in the recent past.

The main advantages of BMA appear to be with probabilistic skill, although deterministic skill is also increased.

An example using the 24 hour temperature forecast from the SBU ensemble will be presented.

PDF for Temperature PDF for Wind Speed

BMA weights each member based on past performance and assigns an uncertainty.

BMARegion 1

Region 7

Region 2

Region 1

Region 7

Region 2

Bias Corrected BMA

BMA Example – Temperature Hour 24Rank Histogram- Warm Season 2007-2009

BMA Example – Temperature Hour 24Reliability > 295 K- Warm Season 2007-2009

Region 1 Region 7Region 2

Conclusions•An operational air quality forecast ensemble is currently being run using a variety of

atmospheric models, air quality models (AQM) and pollutant emission inventories.

•Particulate matter and ozone simulations track observations reasonably well in the warm and

cool seasons, although the ensemble exhibits systematic biases and underdispersion.

•Ensemble biases may be sensitive to the PBL parameterization, with the decreased (increased)

vertical mixing within the MY (YSU) scheme resulting in lower (higher) ozone and higher (lower)

PM forecasts.

•Bayesian model averaging (BMA) has been shown to correct dispersion and improve reliability

for 2-m temperature and 10-m wind speed within the SBU ensemble. Therefore BMA could

improve AQM forecasts through direct application or insertion of the post-processed SBU

forecasts into the AQM ensemble.