post-processing of numerical ozone model forecasts: the land-sea problem
DESCRIPTION
Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem. Bill Ryan Department of Meteorology The Pennsylvania State University [email protected]. 2010 International Workshop on Air Quality Forecasting Research Quebec City, Quebec. The Problem: False Alarms of High O 3. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/1.jpg)
Post-Processing of Numerical Ozone Model Forecasts:
The Land-Sea Problem
Bill RyanDepartment of Meteorology
The Pennsylvania State [email protected]
2010 International Workshop on Air Quality Forecasting ResearchQuebec City, Quebec
![Page 2: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/2.jpg)
The Problem: False Alarms of High O3
• Operational air quality forecasters along the I-95 Corridor in the northeastern US are increasingly reliant upon photo-chemical models for O3 forecast guidance.– Significant recent changes in regional emissions have reduced skill of
guidance methods that require training period.• Overall model performance in this region is good but suffers from
frequent “false alarms” of unhealthy O3 levels.
• The false alarm rate is primarily a function of strong forecast O3 gradients along land-sea boundaries.
• Numerical guidance output (1200 UTC initialization) arrives close (within minutes in some cases) to forecast deadlines. Forecasters require a simple, quick method to deal with this issue.
![Page 3: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/3.jpg)
Example of the Land-Sea O3 Gradient
NOAA-EPA Numerical Ozone Forecast Model (www.weather.gov/aq)
![Page 4: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/4.jpg)
Pattern Recurs in Certain Locations
![Page 5: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/5.jpg)
Not an Issue with PM2.5 Forecasts
Daily maximum 8-Hour AverageOzone (ppbv), NOAA-EPA Model
24-hour (midnight-midnight)average PM2.5 (µg/m3)
![Page 6: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/6.jpg)
Complicating the Issue: Sea Breezes Can Drive Steep Coastal O3 Gradients
In this case (August 11, 2010) sea breeze circulations developed north of afrontal boundary and re-circulated the previous day’s polluted air mass.
MODIS/Aqua~ 1730 UTC
![Page 7: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/7.jpg)
Simplest Application: Philadelphia Forecast Area
The Philadelphia Metro Forecast Area, roughly enclosed in rectangle, is peripherallyaffected by modeled sea breeze-O3 effects, primarily in southern NJ
![Page 8: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/8.jpg)
How Do US Operational Forecasters Receive Local Scale Model Forecast Information?
Model forecast output is automaticallygenerated by NOAA-EPA by extracting
peak O3 concentrations over land areas within designated warning areas
(using zipcodes) that are used foremail/Web notification. This is posted in
timely fashion at AirNow Tech(http://www.airnowtech.org/).
Millville, NJ monitor location shown at left.
Interstates and zip code boundaries shown.
Millville
![Page 9: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/9.jpg)
Forecast Peak O3 Frequently Skirts Coastal Areas
Simplest post-processing method toremove land-sea effects is
to extract O3 forecast at locationsnear monitors and/or away from the
boundary. This can be quicklydone using point data extraction from the
NOAA forecast web site.
![Page 10: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/10.jpg)
Results of Simple Method (2010)
• Removing coastal zip codes from the Millville forecast reduces bias at that location from 17 ppbv to 9 ppbv.
• Because this location is often the location of peak modeled O3 in the Philadelphia metro area, Code Orange model forecast frequency in PHL is reduced from 44 to 20 days (Threshold for Code Orange - Air Quality Alert is 76 ppbv 8-hour average) .
• Of the 20 forecast Code Orange cases, 16 observed Code Orange in PHL in 2010.
![Page 11: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/11.jpg)
Effect on Metropolitan Scale Forecasts (2010)
• When this simple post-processing is applied to the PHL metro domain, numerical model skill equals, or exceeds, expert forecast skill and is a large improvement on earlier forecast methods (not shown).
Hit False Alarm Threat0
0.10.20.30.40.50.60.70.80.9
1
ForecastModel
![Page 12: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/12.jpg)
Simple Approach Less Useful When Monitors are Located Immediately Along Sea-Land Boundary
Southern Delaware forecastarea includes monitor alongcoast at Lewes.
![Page 13: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/13.jpg)
Bias Correction Post-Processing
• Use of Sonoma Tech Air Quality MOS (AQMOS)– http://aqmos.sonomatech.com/index.cfm
• Not a true MOS, method is long term bias removal with additional correction in higher O3 cases (hierarchal bias correction method).
• Applied to southern Delaware in 2010, reduces overall forecast bias by 4.6 ppbv.
• Also, reduces number of false alarms of Code Orange O3 (76 ppbv) from 19 to 5.
• At cost of 4 additional missed Code Orange cases although “close” forecasts (71,74,75 ppbv).
![Page 14: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/14.jpg)
More Difficult Application: Baltimore Metropolitan Area
In 2010, the monitor at Edgewood,located northeast of Baltimore,reached the Code Orange rangeon 75% of all Code Orange days
in the Baltimore region.
On half of the Code Orange days,the three bayside monitors
were the only locations reaching the alert threshold.
![Page 15: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/15.jpg)
An Example of the Bay Breeze Effect on O3 inthe Baltimore Metro Area
2200 UTC hourly O3 observations and wind barbs, June 25, 2009.
![Page 16: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/16.jpg)
Note on Determining Air Quality Forecast Skill in the mid-Atlantic
• False alarms can be due to factors other than water-land effects.
• A frequent problem is forecasting the timing and extent of convection.
• At right, a late day thunderstorm on June 3 limited O3 to 74 ppbv.
• A set of 7 convection cases excluded in further analysis of 2010 results.
![Page 17: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/17.jpg)
Model Forecast O3 in Baltimore
• The problem is solving the “false alarm” issue in non-convective cases.
• Using raw model output, 26 false alarm cases in 2010.
• Using the simple method of excluding near-water zipcode locations reduces false alarms to 19.
![Page 18: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/18.jpg)
Further Post-Processing Approaches
• Due to the close proximity of emissions sources to the sea-land boundary, improvement by removing bay side grid points (simple method) is useful but limited.
• Skill decreases quickly if further exclude near-bayside locations.• Analysis of 2010 Cases (a very active O3 season) gives some
hints:– On average, observed Code Orange cases feature a stronger modeled
gradient from bay side to inland, but too much variability to use operationally (CART method).
– Most useful method combines the magnitude of modeled bayside O3 and the mean domain modeled O3.
![Page 19: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/19.jpg)
Forecast Results for Code Orange Threshold (Air Quality Alert, 8-hour O3 ≥ 76 ppbv, 2010)
Bias False Alarm Hit Cnull Accuracy Heidke Threat0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
NAQC POST PERS
Blue: Model Forecast (NAQC)Red: Post-Processed Model (POST)Green: Persistence (PERS)
![Page 20: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/20.jpg)
Summary of Baltimore Results
• Pro: Removal of bias and reduction of false alarm rate.
• Hit rate also reduced but, at 75%, approximately equal to, or better than, historical expert forecast accuracy.
• Small (~5%) improvement in skill scores.• Half of the misses and false alarms occur in
range [74, 78] ppbv (close calls).
![Page 21: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/21.jpg)
Application to Connecticut Ozone
The majority of high O3 events occur at coastalmonitors (62%). Only 5 of 24 observed Code Orangedays did not have a coastal monitor above thethreshold (76 ppbv, 8-hour average).
![Page 22: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/22.jpg)
Background on Model Performance in CT
• The NOAA-EPA model, on a statewide domain basis, did a reasonable job in 2010. Over-prediction bias of 5 ppbv (~10%).
• Bias correction, on the seasonal or synoptic scale, slightly improves forecasts but has little impact at the Code Orange (Air Quality Alert) threshold.
• Hit rate of 84% but high false alarm rate (34%). High bias for warning forecasts.
• Expert forecast, typical of most locations along I-95, has fewer false alarms but many more “misses”. Low bias for warning forecasts.
![Page 23: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/23.jpg)
Behavior of Model Forecasts in Long Island Sound Similar to Chesapeake Bay/Baltimore
• Used NOAA-EPA model forecasts for locations of all CT monitors.
• As in Baltimore, exploratory analysis with CART suggests domain wide average peak O3 ([65-70 ppbv])and/or land-sea gradient ([15-20 ppbv]) are useful thresholds for analysis.
![Page 24: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/24.jpg)
Results of Post Processed Forecasts (Statewide)
NAQC Mean Mean+Grad Forecast0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
HitFalse AlarmThreat
Model forecasts (NAQC) have a high hit and false alarm rate (high bias) whilethe expert forecast (Forecast) is the reverse. The two post-processed forecasts
Mean and Mean plus Gradient straddle the bias threshold but provide better skill scores (Threat Score, or CSI shown above in green.
![Page 25: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/25.jpg)
Summary and Conclusions
• In populated NE US Corridor, numerical model O3 forecast guidance is useful but has high false alarm rate due to modeled steep O3 gradients along land-sea boundaries.
• A variety of simple post-processing methods, available within operational time constraints, can limit false alarm frequency.
• Efficacy of methods vary from location to location and include:– Removal of coastal locations– Bias correction (of various lengths)– Domain mean peak concentrations and/or strength of gradient
![Page 26: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/26.jpg)
Acknowledgements
• This research is supported by ongoing grants from the Delaware Valley Regional Planning Commission (including the State of Pennsylvania Department of Environmental Protection) and the State of Delaware Department of Natural Resources and Environmental Control.
• Assistance for this presentation also provided by Sonoma Technology (Dianne Miller and Jessica Johnson) and Michael Geigert of the Connecticut Department of Environmental Protection.
![Page 27: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/27.jpg)
Additional Slides
![Page 28: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/28.jpg)
Preliminary Comments
• This presentation is from the operational forecaster’s perspective.
• Air quality forecasting in the US is a state/local government responsibility.
• The air quality forecasting community in the US is therefore non-centralized and diverse in terms of expertise, experience and resources.
![Page 29: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/29.jpg)
![Page 30: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/30.jpg)
Sea Breeze Fronts Can Lead to High O3
NOAA-EPA Model O3 forecast forAugust 11, 2010 (1200 UTC run)
HPC Surface Analysis, 1800 UTC,August 11, 2010
![Page 31: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/31.jpg)
Hi-Res NAM Forecast:Sea Breeze Front Forming Behind Frontal Boundary
![Page 32: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/32.jpg)
Sea Breeze is Observed
Sea Breeze Verified
(left) 3-hour temperature changeand winds, 1900 UTC, August 11
Figure Courtesy:SPC Mesoscale Analysis
http://www.spc.noaa.gov/exper/mesoanalysis/
![Page 33: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/33.jpg)
Example of Mean Domain Model O3 and Observations
Scatter plot of observed peakO3 and mean model
Air Quality Index (AQI) forin CT.
![Page 34: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/34.jpg)
CT Domain Wide Code Orange Threshold Forecasts: Model and Expert
Bias Hit False Alarm0
0.2
0.4
0.6
0.8
1
1.2
1.41.28
0.840000000000001
0.34
0.710000000000001
0.58
0.18
NAQC Forecast
![Page 35: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/35.jpg)
Mean Mean+Grad0.600000000000001
0.700000000000001
0.800000000000001
0.900000000000001
1
1.1
1.2
Bias
at C
ode
Ora
nge
Thre
shol
d
![Page 36: Post-Processing of Numerical Ozone Model Forecasts: The Land-Sea Problem](https://reader035.vdocument.in/reader035/viewer/2022062520/568161cc550346895dd1b863/html5/thumbnails/36.jpg)
Connecticut Miss and False Alarm Cases
• Missed cases (4) were marginal Code Orange cases (76-78 ppbv) with only one hour exceedance in each case.
• Three of seven False Alarms were “close calls” (≥72 ppbv) and remaining four were False Alarms under any of the post-processing methods.
• False Alarms: 6/20 (lingering clouds), 7/20 (clouds/frontal boundary), 8/1 (SE winds), 8/31(?).