Real Time Mesoscale Analysis

John HorelDepartment of Meteorology

University of Utahjohn.horel@utah.edu

RTMA Temperature 1500 UTC 14 March 2008

• Acknowledgements– Dan Tyndall (Univ. of Utah)– Dave Myrick (WRH/SSD)– Manuel Pondeca (NCEP/EMC)

• References– Benjamin, S., J. M. Brown, G. Manikin, and G. Mann, 2007: The RTMA

background – hourly downscaling of RUC data to 5-km detail. Preprints, 22nd Conf. on WAF/18th Conf. on NWP, Park City, UT, Amer. Meteor. Soc., 4A.6.

– De Pondeca, M., and Coauthors, 2007: The status of the Real Time Mesoscale Analysis at NCEP. Preprints, 22nd Conf. on WAF/18th Conf. on NWP, Park City, UT, Amer. Meteor. Soc., 4A.5.

– Horel, J., and B. Colman, 2005: Real-time and retrospective mesoscale objective analyses. Bull. Amer. Meteor. Soc., 86, 1477-1480.

– Tyndall, D., 2008: Sensitivity of Surface Temperature Analyses to Specification of Background and Observation Error Covariances. M.S. Thesis. U/Utah

An Analysis of Record• Forecasters have needs for higher resolution

analyses than currently available– Localized weather forecasting– Gridded forecast verification– Climatological applications

• AOR program established in 2004– Three phases

1. Real Time Mesoscale Analysis2. Delayed analysis: Phase II3. Retrospective reanalysis: Phase III

Real-Time Mesoscale Analysis (RTMA)

• Fast-track, proof-of-concept intended to:– Enhance existing analysis capabilities at the NWS and generate

near real-time hourly analyses of surface observations on domains matching the NDFD grids.

– Background errors can be defined using characteristics of background fields (terrain, potential temperature, wind, etc.)

– Provide estimates of analysis uncertainty

• Developed at NCEP, ESRL, and NESDIS– Implemented in August 2006 for CONUS (and southernmost

Canada) & recently for Alaska, Guam, Puerto Rico– Analyzed parameters: 2-m T, 2-m q, 2-m Td, sfc pressure, 10-m

winds, precipitation, and effective cloud amount– 5 km resolution for CONUS with plans for 2.5 km resolution

More Info… www.meted.ucar.edu

The Real-Time Mesoscale Analysis

• Several layers of quality control for surface observations

• Two dimensional variational surface analysis (2D-Var) using recursive filters

• Utilizes NCEP’s Gridpoint Statistical Interpolation software (GSI)

• Uses various surface observations and satellite winds– METAR, PUBLIC, RAWS, other mesonets– SSM/I and QuikSCAT satellite winds

• Analysis became available in 2006, and is being evaluated by WFOs and several universities

The actual ABCs…• The RTMA analysis equation looks like:

• Covariances are error correlation measures between all pairs of gridpoints

• Background error covariance matrix can be extremely large– 2,900 GB memory requirement for continental scale– Recursive filters significantly reduce this demand

1 1T T T T Tb b o b b o o b

a b b

P P H P HP v P H P y H x

x x P v

The Real-Time Mesoscale Analysis

RTMA Domains

CONUS Alaska Hawaii Puerto Rico Guam

Resolution 5-km ~6-km 2.5-km 2.5-km 2.5-km

Background (downscaled) RUC NAM NAM NAM GFS

Precip NCEP Stage II analysis No No No No

Effective Cloud

Amount

GOES sounder No No No No

RTMA Vancouver Area

CVOC &CVOI

CVOD

Background DownscalingBenjamin et al. (2007)

• CONUS RTMA background = 1-h forecast from the NCEP-operational 13-km RUC downscaled to the 5-km NDFD terrain

1.Horizontal - bilinear interpolation2.Vertical interpolation – varies by variable, for temperature it is

based on near-surface stability and moisture from the RUC native data used to adjust to the RTMA 5-km terrain

– If RTMA terrain lower than RUC, then local RUC lapse rate used (between dry adiabatic and isothermal)

– If RTMA terrain higher than RUC, interpolated between native RUC vertical levels, but shallow, surface-based inversions maintained

3.Coastline sharpening

• Real-time RTMA analysis begins ~30 min past the hour• Getting the data in time is a challenge

– RTMA uses obs taken (+/-12 min from top of hour)• RAWS and Quickscat winds (-30 min to +12 min)

• Many remote obs still don’t make it in time, as well as some obs with longer latencies (Snotel = every 3 hrs)

Surface Data Issues

Observation Network MADIS NCEP

RTMA CONUS Temperature Analysis

Subjective Evaluation of the RTMA Temperature

• Development of web based graphics for many analyses over selected subdomains– Wasatch Valley, UT– Puget Sound, WA– Norman, OK– Shenandoah Valley, VA

• Analysis problems based on subjective evaluation– Non-physical treatment of temperature inversions– Smoothed features– Bull's-eye features around observations– Overfitting issues

• Case study– 0900 UTC 22 October 2007– Shenandoah Valley, VA

Case Study Orientation0900 UTC 22 October 2007RTMA

RTMA Topography vs. Actual Topography

0 100 200 300 400 500 600 700 800 900 1000

Sterling, VA Atmospheric Sounding

1200 UTC 22 October 2007KIAD

RUC 1-hr Forecast

Valid 0900 UTC 22 October 2007RUC 1-hr Forecast

RTMA Temperature Analysis

0900 UTC 22 October 2007RTMA

RTMA Temperature Analysis Increments

0900 UTC 22 October 2007

RTMA Temperature Analysis Increments

Observations

• Surface observations: METAR, RAWS, PUBLIC, OTHER

• Observations must fall in ±12 min time window (−30/+12 min for RAWS)

• Observations undergo quality control by MADIS and internal checks by RTMA

• Approximately 11,000 observations for almost 900,000 gridpoints for CONUS

Observation Density

METAR

16/59/1,744

PUBLIC

215/575/6,486

OTHER

10/75/1,961

RAWS

3/11/1,301

Observation Density

Observation and Background Error Variances

• Ratio of σo2/σb

2 estimated by accumulating statistics over a time period for entire CONUS– 8 May 2008 – 7 June 2008

• RTMA used σo2/σb

2 = 1 for METAR observations and σo

2/σb2 = 1.2 for mesonet observations

• Statistics computed in our research suggest σo

2/σb2 between 2 and 3

– Background should be “trusted” more than observations

Estimation of Observation and Background Error Covariances

• Temperature errors at two gridpoints may be correlated with each other

• Error covariances specify over what distance an observation increment should be “spread”

• RTMA used decorrelation lengths of:– Horizontal (R): 40 km– Vertical (Z): 100 m

Error Correlation Example – KOKV

R = 40 km, Z = 100 mError Correlation- KOKV

Error Correlation Example – KOKV

R = 80 km, Z = 200 mError Correlation- KOKV

Local Surface Analysis• RTMA experiments run on NCEP’s Haze supercomputer

but limited computer time available• Development of a local surface analysis (LSA)

– Same background field– Same observation dataset, but without internal quality

control– Similar 2D-Var method, but doesn’t use recursive filters– Smaller domain– Control run uses same characteristics of RTMA

(decorrelation length scales; observation to background error ratio) at time of study

LSA Temperature Analysis

R = 40 km, Z = 100 m, σo2/σb

2 = 1LSA

RTMA Temperature Analysis

0900 UTC 22 October 2007RTMA

Data Denial Experiments• Evaluation of analyses done by splitting up all observations

randomly into 10 groups• Two error measures:

– Root-mean-square error calculated using the withheld observations only vs. using the observations assimilated into the analysis

– To avoid overfitting, want RMSE at withheld locations to be similar to RMSE at locations used in the analysis

– Root-mean-square sensitivity computed at all gridpoints– Want analyses to be less sensitive to withheld observations

2

1 1

M Nij ij

j i

a oRMSE

MN

2

1 1

M Lij ij

j i

d cS

ML

Withholding Observations

Group 5 R = 40 km, Z = 100 m, σo2/σb

2 = 1

Withheld Data Groups 1-5

0 0.5 1-0.5-1 2.5 4.5 6.5 8.5 10.5 12.5 14.5 16.5

R = 40 km, Z = 100 m, σo2/σb

2 = 1

Withheld Data Groups 6-10

0 0.5 1-0.5-1 2.5 4.5 6.5 8.5 10.5 12.5 14.5 16.5

R = 40 km, Z = 100 m, σo2/σb

2 = 1

RTMA Temperature Analysis Sensitivity

Group 1 RTMA

LSA RMSE and Sensitivity Compared to RTMA

Experiment

RMSE at Withheld

Observations (°C)

RMSE at All Observations

(°C)

Sensitivity (°C)

Background 2.15 2.15 -R = 40 km, Z = 100 m, σo

2/σb2 = 1 1.93 1.62 0.26

R = 80 km, Z = 200 m, σo2/σb

2 = 2 1.89 1.83 0.22

RTMA (group 1 only) 2.12 1.64 0.47

RTMA Background (group 1 only) 2.32 - -

Suggests overfittingOverly sensitive to withheldobservations

How good does the RTMA have to be?

• Is the downscaled RUC background field good enough?– RUC RMSE approximately 2°C to all

observations in CONUS• How much does a 2D-Var analysis

improve RMSE?– 0.1-0.2°C based on LSA results

Verifying Forecasts with Analyses

• Motivating Question: Is there a way we can define what constitutes a “good enough” forecast?

• Problem: Concerns about grid-based verification

• Reality: Forecasters need feedback for entire forecast grids not just selected key observation locations

Forecaster Concerns

• Quality of the verifying analysis• Penalty for adding mesoscale detail to

grids in areas unresolved by analysis• Bad (mesonet) observations influencing

analysis• Analysis in remote areas – driven mostly

by the background model

Using RTMA Analyses and RTMA Uncertainty Estimates for Forecast Verification

• RTMA uncertainty estimate grids are experimental products under development for temperature, moisture, wind

• Goal:– Higher uncertainty in data sparse areas or areas with

larger representativeness errors– Lower uncertainty in data dense areas or areas with

smaller representativeness errors• Basic premise

– Forecasters not penalized as much where the analysis is more uncertain

6

Temperature (oC) Forecast Verification Example

Forecast RTMA RTMA Uncertainty

7

8

9

8

10

12

1

2

1

23

4Valle

y w

man

y ob

s

Mou

ntai

ns w

unre

p ob

s

6

Temperature (oC) Forecast Verification Example

Forecast RTMA RTMA Uncertainty

7

8

9

8

10

12

1

2

1

23

4Valle

y w

man

y ob

s

Mou

ntai

ns w

unre

p ob

s

No color= Differences between the forecast and analysis are less than analysis uncertainty

Red = abs(RTMA – Forecast) > Uncertainty

Summary• Improving current analyses such as RTMA requires improving

observations, background fields, and analysis techniques– Increase number of high-quality observations available to the

analysis – Improve background forecast/analysis from which the analyses

begin– Adjust assumptions regarding how background errors are

related from one location to another• Future approaches

– Treat analyses like forecasts: best solutions are ensemble ones rather than deterministic ones

– Depend on assimilation system to define error characteristics of modeling system including errors of the background fields

– Improve forward operators that translate how background values correspond to observations