Local Predictability of the Performance of an

Ensemble Forecast System

Liz Satterfield and Istvan SzunyoghTexas A&M University, College Station, TX

Third THORPEX International Science SymposiumMonterey California, 14-18 September 2009

Introduction Ensemble prediction systems account for the

influence of spatio-temporal changes in predictability on forecasts

Performance of an ensemble prediction system is flow dependent

The goal of our study is to lay the theoretical foundation of a practical approach to predict spatio-temporal changes in the performance of an ensemble prediction system

Experiment Design We use an implementation of the Local Ensemble

Kalman Filter (LETKF) on T62L28 resolution version of the NCEP GFS

Experiments with Observations: Simulated Observations in Random Location: 2000 randomly

placed vertical soundings that provide 10% coverage of model grid points (Kuhl et al. 2007, JAS).

Simulated Observations at the Location of Conventional Observations: Observational noise added to “true states”, location and type taken from conventional observations

Conventional Observations of the Real Atmosphere: Observations used to obtain the type and location for simulated observations (excludes satellite radiances)

Linear Diagnostics calculated in local regions using energy rescaling Explained Variance Fraction of forecast error contained in the space

spanned by the ensemble Minimum value of zero when the error lies orthogonal to the space

spanned by ensemble perturbations Maximum value of 1 when the ensemble correctly captures the space

of uncertainty

E-Dimension A local measure of complexity based on eigenvalues of the ensemble-based error covariance matrix in the local region (Introduced in Patil et al. 2001)

Minimum value of 1 when the variance is confined to a single spatial pattern of uncertainty

Maximum value of N when the variance is evenly distributed between N independent spatial patterns of uncertainty

Relationship between Explained Variance, E-Dimension, and Forecast Error shown for conventional observations

Lower E-Dimension

Higher Forecast ErrorStrong Instabilities

Linear space provides an increasingly better representation of the space of uncertainty up to 120 hours

Colors show mean E-dimension

Joint ProbabilityDistribution

Local Relative Nonlinearity a measure of linearity in the local regions

= || xa,f-xa,f|| / ___(1/k)||xa,f(k)||Modified from Gilmour et al (2001)

Standard deviations of values computed using localization show a high degree of variability

Time mean of globally averaged values for conventional observations

Local Regions

Global

Distance between ensemble mean and control forecasts normalized by the average perturbation magnitude

Forecast Lead Time

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

12 24 36 48 60 72 84 96 108 120 132

Forecast Lead Time

Co

rrel

atio

n

Northern Hemisphere

Correlation between relative nonlinearity and explained variance shown for conventional observations

High values of explained variance at the 120 hour lead time are not due to strong linearity of the evolution of uncertainties

TV = Square of the magnitude of the error in the ensemble mean forecast

TVs =Portion of TV which lies in the space spanned by the ensemble perturbations

Vs = ensemble variance

Evolution of Forecast Error shown for randomly placed simulated observations

Forecast Lead Time

For a perfect ensemble, TV and TVs would equal Vs at the initial time

At initial time, TVs equals Vs therefore further inflating the variance would not improve analyses

Evolution of Forecast Error results shown for the Northern Hemisphere Extratropics

Simulated obsRealsitic location

Conventionalobs.

TV

Vs

Forecast Lead TimeForecast Lead Time

The total ensemble variance underestimates the forecast error captured by the ensemble

TVs

Spectrum of the Ratio Between Observed and Predicted Probability (d-ratio) at analysis time Modified from Ott et al (2002)

Simulated obsRandom location

Simulated obsRealsitic location

Conventionalobs.

eigen-direction

Optimal performance in this measure would be indicated by 1 for all k

Uncertainty is underestimated

Uncertainty is overestimated

dk=(xtk)2/k

Spectrum of the Ratio Between Observed and Predicted Probability (d-ratio) at 120-hour lead time

Conventional obs

Simulated obsRealsitic location

Simulated obsRandom location

By 120-hr lead time, the ensemble underestimates uncertainty in all directions

The spectrum is steepest for observations of the real atmosphere

di < 1 : ensemble overestimates

di > 1 : ensemble underestimates

eigen-direction

The leading direction of d-ratio calculated for temperature at 850hPa

For realistically placed observations, the regions of largest underestimation are those of highest observation density

Simulated, Realistically Placed Obs. of the real atmosphere

Conclusions The linear space spanned by ensemble perturbations

provides an increasingly better representation of the space of uncertainties with increasing forecast time.

The improving performance of the space of ensemble perturbations with increasing forecast time is not due to local linear error growth, but rather to nonlinearly evolving forecast errors that have a growing projection on the linear space.

At analysis time, we find that the ensemble typically underestimates uncertainty more severely in regions of high observation density than for regions of low observation density. This result indicates that implementing a spatially varying adaptive covariance inflation technique may improve analyses.