slide 1 ecmwf data assimilation training course – may 2010 ensemble data assimilation massimo...

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

Ensemble Data Assimilation

Massimo Bonavita

ECMWF

Acknowledgments: Lars Isaksen, Elias Holm, Mike Fisher,

Laure Raynaud


Outline

•The Ensemble Data Assimilation method

•What do we do with the EDA?

•Use of EDA variances in ECMWF 4DVar

•4DVar assimilation experiments

•Conclusions and plans

Slide 2


Outline



•Use of EDA variances in 4DVar



Slide 3


“When simulating the error evolution of the reference system one should use the reference gain matrix K” (Berre et al. 2007)

The EDA method

KyxKHIx fa

ofa KeeKHIe

Slide 4


This is what an ensemble of 4DVar analyses with random observation and SST perturbations does!

εa/f/o = perturbations w.r.to ensemble mean

The EDA method

ofa KεεKHIε

Analysis

xb+εb

y+εo

SST+εSST (etc.)

xa+εa

Forecastxf+εf

Slide 5


• After a few cycles the EDA system will have forgotten the initial background perturbations

• We are diagnosing the background error statistics of the actual analysis system

However:

• We assume that observation and SST errors are correctly specified

• We need to account for the model error component of the background error

The EDA method

Slide 6


“Stochastic Kinetic Energy Backscatter”

SKEB (Berner et al., 2009)

ΔXperturbed physics= f(KE,ΔXphysics,n)

A fraction of the dissipated energy is backscattered upscale and acts as stream function forcing on resolved-scale flow.

Spectral Markov chain: temporal and spatial correlations prescribed

Only vorticity perturbed, so only wind field directly affected

All levels are perturbed. Either randomly or partially correlated.

“Stochastically Perturbed Parameterisation Tendencies”

SPPT (Leutbecher, 2009)

ΔXperturbed physics= ( 1+μr) ΔXphysics

Random pattern r varies smoothly in space and time, with de-correlation scales 500 km and 6 hours.

Gaussian distribution with no bias and stdev 0.5 (limited to ±3stdev)Same random number r

for X=T, q, u, v

No perturbations in lowest 300 m and above 50 hPa (0≤ μ ≤1).

from: Lars Isaksen

The EDA method

Slide 7


The EDA methodComparing the impact of various stochastic perturbation methods: zonal mean

u wind ensemble background forecast spread

N S

Slide 8

Impact of SKEB on U spread Additional Impact of SPPT on U spread

from: Lars Isaksen


The EDA methodComparing the impact of various stochastic perturbation methods: zonal mean

Temperature ensemble background forecast spread

N S

Slide 9

Impact of SKEB on T spread Additional Impact of SPPT on T spread

from: Lars Isaksen


•In the EDA context model error parametrizations should increase spread where/when errors are larger => should increase spread-error correlation

•Are they doing this?

Slide 10

The EDA method


EDA (baseline) - EDA+SKEBEDA+SPPT - EDA+SPPT+SKEB

Slide 11

The EDA method


EDA (baseline) - EDA+SKEBEDA+SPPT - EDA+SPPT+SKEB

Slide 12

The EDA method


•Model error parametrizations do increase spread-error correlations

•Large effect in the lower troposphere Tropics

•Small effect in the Extratropics

•Similar behaviour-performance of the two schemes and their combination

Slide 13

The EDA method


10 ensemble members using 4D-Var assimilations

T399 outer loop, T95/T159 inner loop (reduced number of iterations)

Observations randomly perturbed

Cloud track wind (AMV) correlations taken into account

SST perturbed with realistically scaled structures

Model error represented by stochastic methods (SPPT, Leutbecher, 2009)

All 107 conventional and satellite observations used

The EDA method

from: Lars Isaksen

Slide 14


Outline






Slide 15


• The EDA simulates the error evolution of the 4DVar analysis cycle. As such it can be applied to:

1. Compute climatology of B matrix for use at the initial time of 4DVar (Analysis-Ensemble method, Fisher 2003; see talk on B matrix modelling)

2. Provide initial conditions for ensemble forecasts (EPS)

3. Provide a flow-dependent sample of background errors at the initial time of 4DVar

What do we do with the EDA?

Slide 16


Improving Ensemble Prediction System by including EDA perturbations for initial uncertainty

The Ensemble Prediction System (EPS) benefits from using EDA based perturbations. Replacing evolved singular vector perturbations by EDA based perturbations improve EPS spread, especially in the tropics.The Ensemble Mean has slightly lower error when EDA is used.


Slide 17

EVO-SVINIEDA-SVINI

EVO-SVINIEDA-SVINI

N.-Hem. Tropics

Ensemble spread and Ensemble mean RMSE for 850hPa T


• The EDA simulates the error evolution of the 4DVar analysis cycle. As such it can be applied to:

1. Compute climatology of B matrix for use at the initial time of 4DVar (Analysis-Ensemble method, see talk on B matrix modelling)

2. Provide initial conditions for ensemble forecasts (EPS)

3. Provide a flow-dependent sample of background errors at the initial time of 4DVar


Slide 18


• 4DVAr does not cycle error information (B), only the state estimate

• 4DVar behaves like a Kalman filter in which the covariance matrix is reset to some static matrix B every few hours (12h in ECMWF implement.)

• Two possible ways around this:

1. Run 4DVar over a long enough window so that the influence of the initial state and errors on the final state analysis is negligible

Slide 19



T+120S.hem Lat -90.0 to -20.0 Lon -180.0 to 180.0

Root mean square error forecast500hPa Geopotential

Time series curves

15

AUGUST 200516 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

30

40

50

60

70

80

90

100

110

120

all obs

all obs

Analysis experiment started with satellite data reintroduced on 15th August 2005

Analysis experiment started with satellite data reintroduced on 15th August 2005

From Mike Fisher

Memory of the initial statedisappears after approx. 3 days

Slide 20


• An analysis window of ≥3 days would allow 4DVar analysis at final time to be independent of initial state and error estimates

• However:

1. An effective model error representation must be applied to reconcile model and measurements over a long analysis time window (i.e., weak constraint 4DVar)

2. We would still lack an estimate of analysis errors

Slide 21



• Two possible ways around this:

1. Run 4DVar over a long enough window so that the influence of the initial state and errors on the final state analysis is negligible

2. Use a sequential method (EDA, EnKF) to cycle error covariance information

Slide 22



Hybrid methods

• Use of ensemble perturbations in a 3-4DVar analysis

• Cycle error information through ensemble of Data Assimilations

• Retain the implicit full rank error representation of 3-4DVar

µεσον τε και αριστονAristotle, Nic. Ethics 2.6

Slide 23


Hybrid systems

• Ensemble perturbations can be used in a 3-4DVar analysis in a number of different ways:

1. Use ensemble variances for observation QC

2. Use ensemble (co)variances as starting B matrix of minimization (often in linear combination with climatological B, extra control variable)

3. Use of ensemble covariances inside 4DVar minimization (En4DVAR)

Slide 24



Outline






Slide 25


• We want to use EDA perturbations to simulate 4DVar flow-dependent error covariance evolution

• We start with the diagonal of the Pf matrix, i.e.:

“Estimate the first guess error variances with the variance of the EDA short range forecasts”

• This has been tried before (Kucukkaraca and Fisher, 2006, Fisher 2007, Isaksen et al., 2007) but results have been inconclusive

Use of EDA variances in 4DVar

Slide 26


Define: Veda = EDA sampled variance

V* = True error variance

Then the estimation error can be decomposed:

Veda-V* = E[Veda-V*] + (Veda-E[Veda])

systematic random

Similarly for mean square of the estimation error:

E[(Veda-V*)2] = (E[Veda-V*])2 + E[(Veda-E[Veda])2]

systematic random

We should try to minimize both!Slide 27



What raw ensemble variances look like?

Spread of Vorticity FG t+9h ml=64

Slide 28



• Noise level is due to sampling errors: 10 member ensemble

• EDA is a stochastic system: variance of variance estimator ~ 1/Nens

• We need a system to effectively filter out noise from first guess ensemble forecast variances: Reduce the random component of the estimation error

Slide 29



“Mallat et al.: 1998, Annals of Statistics, 26,1-47”

Define Ge(i) as the random component of the sampling

error in the estimated ensemble variance at gridpoint i:

Then the covariance of the sampling noise can be shown to be a simple function of the expectation of the ensemble-based covariance

matrix:

(1)

iiiie BEBiG

~~

2~

1

2ij

ee BEN

jGiGE

Slide 30



“Mallat et al.: 1998, Annals of Statistics, 26,1-47”

A consequence of (1) is that:

(2)

i.e., sampling noise is smaller scale than background

error. If the variance field varies on larger scales then the background error

=> we can use a spectral filter to disentangle noise

error from the sampled variance field

2

iLiL b

ee

G

Slide 31


ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010 Slide 32



• There is indeed a scale separation between signal and sampling noise!

• Truncation wavenumber is determined by maximizing signal-to-noise ratio of filtered variances (details in Raynaud et al., 2009, and forthcoming Tech. Memo)

• Optimal truncation wavenumber depends on parameter and model level

Slide 33



Raw Ensemble StDevVO ml64

Filtered Ensemble StDevVO ml64

Slide 34



Raw Ensemble StDevSpec. Hum. ml64

Filtered Ensemble StDevSpec. Hum. ml64

Slide 35



Operational StDevRandom. Method (Fisher & Courtier, 1995

Filtered Ensemble StDevVO ml64

Slide 36



Is Filtering the Ensemble Variances enough to improve the analysis?

Not quite…

N.Hem. Z 500hPA AC S.Hem. Z 500hPA AC

Slide 37



Should we also do something about the systematic component of the estimation

error?

E[(Veda-V*)2] = (E[Veda-V*])2 + E[(Veda-[Veda])2]

systematic random

A statistically reliable ensemble satisfies:

Slide 38


Mean_Errord_Ens_Square_VarianceEns11

1111

ensens NN


Vorticity ml 30 (~50hPa) Ensemble Error Ensemble Spread

Spread - Error

Slide 39



Vorticity ml 78 (~850hPa) Ensemble Error Ensemble Spread

Spread - Error

Slide 40



Is the ensemble fg statistically calibrated?

• Calibration factors needs to be model level, latitude and parameter dependent

• Calibration factors seems also to be flow-dependent, i.e. depend on the size of the expected error

Slide 41



• Calibration factors need to be flow-dependent, too!

• Do they also change in time?

Slide 43



• There is not a large day-to-day variability but seasonal variability is important

• General solution: slowly varying adaptive calibration coefficients

Slide 46



Variance post-process

xa+εia

Analysis ForecastSST+εi

SST

y+εio

xb+εib

xf+εif

i=1,2,…,10

EDA Cycle

εif raw

variances

Variance Recalibration

Variance Filtering

EDA scaled variances

4DVar Cycle

xa

Analysis ForecastEDA scaled Varxb xf

Slide 47


Outline






Slide 49


4DVar assimilation with EDA variances

Deterministic DA experiments with EDA variances

1. CY35r3_esuite, T799L91, 7/01 – 16/02 2009

2. Control f8a4

3. Experiment fb4k with ensemble DA variances:

a) Calibration step: adaptive, flow-dependent, regionally varying, for each parameter and model level

b) Filtering step: “Optimal” spectral filtering

c) EDA variances are used both in observation QC and start of 4DVar minimization

Slide 50



Slide 51



N.HEM Z acJan-Feb

Slide 52



S.HEM Z acJan-Feb

Slide 53



TROP. VW RMSEJan-Feb

Slide 54




1. CY36r2_esuite, T1279L91, 7/08 – 16/09 2008

2. Control far9

3. Experiment fb9x with ensemble DA variances:

a) Calibration step: adaptive, flow-dependent, regionally varying, for each parameter and model level

b) Filtering step: “Optimal” spectral filtering

c) EDA variances are used both in observation QC and start of 4DVar minimization

Slide 55



N.HEM Z acAug-Sep

Slide 56



S.HEM Z acAug-Sep

Slide 57



TROP VW RMSEAug-Sep

Slide 58




1. Results are generally positive in the Extra-Tropics (more so in the NH)

2. Small positive impact in the Tropics

Slide 59



Slide 60

• Observation departures statistics support the idea that (slightly) different observation usage is the result of smarter OBS QC decisions


Outline






Slide 61


• First step towards an error cycling DA

• Use of flow dependent EDA variances has the potential to improve the deterministic scores

• A careful post-processing step of the raw ensemble first guess forecast is necessary to:

a)Filter sampling noise

b)Adaptively calibrate the ensemble

Conclusions and plans

Slide 62


•Better OBS QC decisions seems to be playing a part

•Further improvements in model error parameterizations will directly benefit the system

•Increase in ensemble size will benefit the system


Slide 63


WHERE NEXT

•Operational implementation and testing

•Further tuning of system at full operational resolution (T1279L91)


Slide 64


•Generalize the use of EDA variances to unbalanced components of control vector

•Relax the assumption: analysis=“truth” in the calibration step


Slide 65


Medium term:

•Refine the representation of initial uncertainties (correlated perturbations, surface fields uncertainties) in stochastic EDA

•Evaluate EnKF covariances

•Further develop the hybridization of 4DVar with EDA (investigate the use of EDA covariances)


Slide 66


Thanks for your attention!

I welcome your questions/comments…

Slide 67

slide 1 ecmwf data assimilation training course – may 2010 ensemble data assimilation massimo...

Documents

skeb slide

f slide

combination slide

eda baseline eda

lars isaksen slide

sppt eda

skeb eda

laure raynaud slide