rtma/urma, nws’ mesoscale analysis system, with analysis
TRANSCRIPT
N A T I O N A L O C E A N I C A N D A T M O S P H E R I C A D M I N I S T R A T I O N
1
Verification and Validation (V&V) of Forecasters’ Needs for RTMA/URMA, NWS’ Mesoscale Analysis System,
with Analysis and Nowcast V&V FrameworkYoung-Joon (YJ) Kim1, Tatiana Gonzalez1, Eric Guillot1,
Bradley Brokamp2,3 and Mark Tew2
Analysis and Nowcast Branch (AFS11)1
Analysis and Mission Support Division2
Analyze, Forecast, and Support OfficePathways Internship Program3
NWS/NOAASilver Spring, MD
July 28, 2020
2020 UFS Users’ WorkshopVerification, Evaluation, and Post Processing (Parallel Session 1)
N A T I O N A L O C E A N I C A N D A T M O S P H E R I C A D M I N I S T R A T I O N
2
❏ In order to address the needs of NWS forecasters for improving Analysis and Nowcast systems, we (Analysis and Nowcast Branch of AFS) are collecting field’s input by various means, ranging from community web fora to direct field surveys.
❏ The collected information indicates that the RTMA/URMA (Real-Time Mesoscale Analysis/UnRestricted Mesoscale Analysis) system reveals systematic biases, especially over complex terrain, and urgently need to be improved.
❏ In order to identify the systematic biases and develop the requirements for RTMA/URMA, we are analyzing outstanding weather events reported by the field utilizing the Analysis and Nowcast V&V Framework.
Introduction
N A T I O N A L O C E A N I C A N D A T M O S P H E R I C A D M I N I S T R A T I O N
3
RTMA/URMA VLab Forum Feedback Statistics(Nov. 2016 ~ Present, Over 100 Postings, Processed by Eric Guillot)
* not a true NWS Mission Service Area
Posts by RegionWestern 49.5%Central 26.2%
Southern 13.6%Eastern 6.8%
HQ 1.9%Alaska 1.0%NCEP 1.0%
Posts by NWS Mission Service AreaPublic Weather 36.4%
Mountain Weather* 24.3%Marine and Coastal Weather 16.8%
Aviation Weather 12.1%Fire Weather 4.6%
Water Resources 3.5%Winter Weather 1.7%
Tropical Weather 0.6%
Posts by Phenomenon/IssueRTMA Quality Control 25.5%
Cold Pools 16.1%Wind 15.3%
RTMA Station Flagging 10.9%
Lakes 11.7%Ocean 7.3%Fires 5.1%
Precipitation 4.4%Clouds 2.2%
Fog 0.7%Hurricane 0.7%
Posts by RTMA/URMA Parameter
Temperature 36.6%Terrain* 29.0%
Wind 15.2%Water Temp 6.2%
Moisture 4.8%QPE 4.1%
Sky Cover 2.1%Station
Locations* 1.4%Visibility 0.7%
More issues are reported in WR (Western Region) and CR (Central Region), over areas of large terrain variations (mountains, valleys, coasts & islands) for temperatures and winds, and are largely associated with observation data quality control & station flagging.
*Most reports are for Version 2.7. New version (v2.8) scheduled to be implemented in July 2020.
+ not an explicit Parameter
FY21 AFS/PMO AOP AFS11 Presentation (3/4/2020)
N A T I O N A L O C E A N I C A N D A T M O S P H E R I C A D M I N I S T R A T I O N
4
AFS11 RTMA/URMA Survey(5/7-6/12/2020, 172 Submissions, Processed by Eric Guillot)
RTMA/URMA Difficulty Capturing PhenomenonCold pools in complex terrain (e.g.,
mountains, coastlines, islands) 57.0%
High wind events in complex terrain 55.2%Precipitation in complex terrain 42.4%Other (mostly "I don't know" or
"marine/ocean phenomena") 32.0%
High wind events in flat terrain 29.7%Moisture anomalies 27.9%
Ocean and/or lake temperatures 25.0%
RTMA/URMA Technical ProblemsQuality Control (e.g., good data getting thrown out, bad data being assimilated) 58.1%
Data Influence (e.g., station data does not modify the background field enough) 49.4%
Error in Model Background (e.g., NWP models are not accurate enough) 48.8%
Lack of Data (e.g., surface obs datasets not available to be ingested) 41.9%
Functionality (e.g., ability to specify which data is used/unused in analysis) 37.8%
RTMA/URMA Parameters Not Adequate for OpsWind Speed 55.2%Wind Gust 53.5%Sky Cover 32.6%
Min Temperature 31.4%Ceiling 29.7%
Visibility 29.1%QPF 29.1%
Hourly Temperature 28.5%Dew Point Temperature /
Relative Humidity 28.5%Max Temperature 26.7%
● Winds and temperatures (collectively) are reported as the most inadequate RTMA/URMA parameters.
● Errors in quality control and model background, and lack of data are reported as major technical problems.
● Cold pools, high wind and precipitation events in complex terrain are reported as the most difficult phenomena to capture.
N A T I O N A L O C E A N I C A N D A T M O S P H E R I C A D M I N I S T R A T I O N
5
plus
Source: METviewer TrainingOct. 2018
(Model Evaluation Tools)
Data covering Outstanding Weather Events in the field
Multi-Scale Common Environmental Data
(For Consistent V&V)
Less Rejected Observations through better Quality Control based on Field Input
Selected set of Observations & Analyses
UFS METplus Verification Framework
V&V (Verification & Validation) with AN Framework coupled with UFS METplus (Model Evaluation Tools plus) Framework
Analysis and Nowcast V&V Framework
Common Env Data
RTMA/URMA
V&VTestbed
with Common ICs & BCs (Lateral, Lower & Upper)
NWP (e.g., HRRR)
N A T I O N A L O C E A N I C A N D A T M O S P H E R I C A D M I N I S T R A T I O N
6
❏ Forecaster at WFO Pueblo, CO, noted that RTMA/URMA low temperatures were too warm in the Rio Grande River Valley in the mornings of 12/1/19.
❏ Temperatures are warmer than station observations in the river valley by as much as 20 deg F!
❏ This case highlights the RTMA/URMA biases in complex terrain, particularly with mountain valley temperatures/cold pools.
Case 1: Cold Pool over Complex TerrainKathleen Torgerson @ WFO Pueblo, CO (CR): Rio Grande Valley, am 12/1/2019
Cold Pool generated by Valley Effect
METAR station (Wolf Creek Pass - KCPW) located near the Rio Grande Valley in Pagosa Springs, CO.
This station’s elevation is nearly 12,000 feet.
N A T I O N A L O C E A N I C A N D A T M O S P H E R I C A D M I N I S T R A T I O N
7
Temperature Tolerances Number Percentage2 to < 3 deg F 74 43.0%1 to < 2 deg F 66 38.4%
Other (mostly "0" or "depends on the type of sensor") 18 10.5%
3 to < 4 deg F 10 5.8%4 to < 5 deg F 3 1.7%5 to < 6 deg F 1 0.6%
AFS11 Survey on RTMA/URMA
Rio Grande ValleyKCPW
KGUC
KALS
K04V
Case 1: Cold Pool over Complex TerrainKathleen Torgerson @ WFO Pueblo, CO (CR): Rio Grande Valley, am 12/1/2019
URMA MeanMETAR Mean
The cold pool represented in some METAR station data (and also WxUnderground sites) is not represented by URMA mean (nor by METAR mean). Possible reasons include inaccurate model background (HRRR/RAP) and/or inadequate QC, which provides under-represented verification for RTMA/URMA.
cold pool reported
N A T I O N A L O C E A N I C A N D A T M O S P H E R I C A D M I N I S T R A T I O N
8
❏ Forecaster at WFO Portland, OR, noted that RTMA/URMA wind speeds were too low in the Columbia River Gorge on 10/29/19.
❏ Wind speeds at the Portland International Airport were sustained at 29 knots (~15 m/s), while RTMA/URMA values were 20-25 knots (~10-13 m/s) too low!
❏ This case highlights the RTMA/URMA biases in complex terrain, particularly with winds associated with mountain gap flow.
Case 2: Wind Gust over Complex Terrain Dan Miller @ WFO Portland, OR (WR): Columbia River Gorge, am 10/29/2019
KPDX 45.6 -122.6
Portland International Airport (KPDX) relative to the
Columbia River in Portland, OR. The river flows east to
west, parallel with the airport. Strong easterly winds blow during gap flow events.
Gap Flow generated by Directional Channeling
N A T I O N A L O C E A N I C A N D A T M O S P H E R I C A D M I N I S T R A T I O N
9
AFS11 Survey on RTMA/URMA
Wind Tolerances Number Percentage3 to < 5 knots (1.5-2.6 m/s) 75 43.6%1 to < 3 knots (0.5-1.5 m/s) 74 43.0%Other (mostly "0" or "N/A") 14 8.1%
5 to < 7 knots 8 4.7%7 to < 9 knots 0 0.0%9 to < 11 knots 1 0.6%
Columbia River Gorge
KDPX
KTTD
KSPB
KHIO
KUAO
Case 2: Wind Gust over Complex Terrain Dan Miller @ WFO Portland, OR (WR): Columbia River Gorge, am 10/29/2019
URMA Mean
METAR Mean
The wind gust represented in some METAR station data is not represented in URMA mean (nor by METAR mean). Possible reasons include inaccurate model background (HRRR/RAP) and/or inadequate QC, which provides under-represented verification for RTMA/URMA.
N A T I O N A L O C E A N I C A N D A T M O S P H E R I C A D M I N I S T R A T I O N
10
Conclusions
❏ We performed the V&V of the field’s needs on RTMA/URMA utilizing the Analysis & Nowcast V&V Framework for sample cases found from the vLab Forum.
❏ We presented example cases of systematic biases over complex terrain involving cold pool and wind gust events reported by the field.
❏ The biases are considered to be due to a combination of inaccurate model background and/or improper observation data quality control.
❏ We are developing “operational requirements” based on the V&V of these and other cases as well as gap analysis of survey results, which will be validated through the NWS Governance process.
❏ Developed requirements will be delivered to the developers for alleviating the systematic biases of RTMA/URMA.
N A T I O N A L O C E A N I C A N D A T M O S P H E R I C A D M I N I S T R A T I O N
11
Are these effects fully represented in current model physics, downscaling, & error estimation & observations quality control algorithms, meeting the field’s needs?
Terrain of large variation (e.g., mountains) can have significant effect on temperature and humidity as well on winds.
Gap Analysis: Complex Physical Mechanisms of Flow around Topography - To Be Represented in the Analysis
Wave Breaking
Low-level Flow
Blocking
Windstorm
GapFlowWind
Gust
Wind Stagnation
Wind Stagnation
Valley Effect (Cold Pool)