on instability in data assimilation · motivation and introduction data assimilation mathematical...

On Instability in Data Assimilation

Roland Potthast1,2,3, Boris Marx3, Alexander Moodey2, Amos Lawless2, Peter Jan

van Leeuwen2

1German Meteorological Service (DWD),2University of Reading, United Kingdom

3University of Gottingen, Germany

Roland Potthast 1/43

Outline

1 Motivation and Introduction.

2 Variational Data AssimilationTikhonov and 3dVar

Spectral Representation

3 Convergence Analysis for Cycled Assimilation 1: Range ArgumentsSetup: Constant System

Convergence for Data f ∈ R(H) / Divergence for f /∈ R(H)

4 Convergence Analysis 2: Regularization of AssimilationSetup: High-Frequency Damping Systems

Analysis Error Bounds

Outline

Motivation and Introduction Data Assimilation

Motivation I: Numerical Weather Forecast ...

Warn and Protect

Plan Travel

Motivation II: Numerical Weather Forecast ...

Logistics

Rivers and Environment

Air Control

Data Assimilation at the Deutscher Wetterdienst

Modelling of the Atmosphere: Geometry

GME/ICON

Resolution 30km

COSMO-EU

Resolution 7km

COSMO-DE

Resolution 2.8km

Measurements for State Determination ...

Synop,

Radiosondes,

Buoys,

Airplanes

(AMDAR),

Radar,

Wind Profiler,

Scatterometer,

Radiances,

GPS/GNSS,

Ceilometer,

LidarRoland Potthast 7/43

Mathematical Setup for Data Assimilation

State Space:X state space, containing all state

variables in one vector ϕϕ state of the atmosphere

tk time discretization point

ϕk state at time tkMk : X → X model operator at time tk ,

ϕk 7→ ϕk+1 = M(ϕk )

Observation SpaceYk observation space at time tkfk observation vector at time tkHk : X → Yk observation operatorx, y points in physical space

ϕk state at time tk

Mk : X → X model operator at time tk ,

ϕk 7→ ϕk+1 = M(ϕk )

Data Assimilation Task

Definition (Data Assimilation Task)

Given measurements fk at tk for k = 1, 2, 3, ...determine the states ϕk from the equations

Hϕk = fk , k = 1, 2, 3, ... (1)

taking care of the model dynamics given by Mk .

Usually the measurement space is dynamic,

i.e. changing in every time-step.

In general H is a non-linear operator,

non-injective, ill-posed.

The value fk contains significant data error

with stochastic components and a dynamic

Data Assimilation Task

Definition (Data Assimilation Task)

Given measurements fk at tk for k = 1, 2, 3, ...determine the states ϕk from the equations

Hϕk = fk , k = 1, 2, 3, ... (1)

taking care of the model dynamics given by Mk .

Usually the measurement space is dynamic,

i.e. changing in every time-step.

In general H is a non-linear operator,

non-injective, ill-posed.

The value fk contains significant data error

with stochastic components and a dynamic

Variational Data Assimilation Tikhonov and 3dVar

Outline

Tikhonov Data Assimilation

In every assimilation step k ∈ N we solve the variational minimization problem to find

the minimum of

J(ϕ) := α||ϕ− ϕ(b)k+1||

2+ ||fk+1 − Hϕ

(b)k+1||

2, ϕ ∈ X , (2)

ϕ(b)k+1 := Mϕ

(a)k , k = 0, 1, 2, ... (3)

For linear operators the minimum is given by the normal equations, which can be

reformulated into the update formula

ϕ(a)k+1 = ϕ

(b)k+1 + Rα

(fk+1 − Hϕ

(b)k+1

with Rα = (αI + H∗H)−1H∗, or in terms of the analysis fields ϕ(a)k

ϕ(a)k+1 = Mϕ

(a)k + Rα

(fk+1 − HMϕ

for k = 0, 1, 2, ....Roland Potthast 11/43

3dVAR - 1

1 Start with ϕ(a)0 and for k = 1, 2, 3, ... do:

2 Calculate first guess

ϕ(b)k = Mk−1ϕ

(a)k−1 (6)

3 Assimilate data fk at time tk calculating ϕ(a)k .

3dVAR - 1

ϕ(b)k = Mk−1ϕ

(a)k−1 (6)

3dVAR - 1

ϕ(b)k = Mk−1ϕ

(a)k−1 (6)

3dVAR - 1

ϕ(b)k = Mk−1ϕ

(a)k−1 (6)

3dVAR - 2

Functional at time slice

J(ϕ) = ||ϕ− ϕ(b)||2B−1 + ||f − Hϕ||2R−1 (7)

Update Formula

ϕ(a)k = ϕ

(b)k + (B−1 + H∗R−1H)−1H∗R−1(fk − H(ϕ

(b)k ))

= ϕ(b)k + BH∗(R + HBH∗)−1︸︷︷︸

Kalman gain matrix

( fk − H(ϕ(b)k )︸︷︷︸

obs - first guess

). (8)

3dVar = Tikhonov Regularization in a weighted space

We study a weighted scalar product⟨ϕ,ψ

⟩:=⟨ϕ, B−1ψ

⟩L2,⟨

f , g⟩

f ,R−1g⟩

with some self-adjoint invertible matrices B and R. The adjoint with respect to the

weighted scalar product is denoted by H′.

Then⟨f ,Hϕ

⟩=⟨

f ,R−1Hϕ⟩

L2=⟨

R−1f ,Hϕ⟩

L2=⟨

H∗R−1f , ϕ⟩

H∗R−1f , BB−1ϕ⟩

BH∗R−1f , B−1ϕ⟩

L2=⟨

BH∗R−1f , ϕ⟩

H′f , ϕ⟩.

This leads to

H′ = BH∗R−1 (10)

and thus

BH∗(R + HBH∗)−1 = BH∗R−1(I + HBH∗R−1)−1

= H′(I + HH′)−1 = (I + H′H)−1H′.

⟩:=⟨ϕ, B−1ψ

⟩L2,⟨

f , g⟩

f ,R−1g⟩

weighted scalar product is denoted by H′. Then⟨f ,Hϕ

⟩=⟨

f ,R−1Hϕ⟩

L2=⟨

R−1f ,Hϕ⟩

L2=⟨

H∗R−1f , ϕ⟩

L2=⟨

BH∗R−1f , ϕ⟩

H′f , ϕ⟩.

This leads to

H′ = BH∗R−1 (10)

and thus

BH∗(R + HBH∗)−1 = BH∗R−1(I + HBH∗R−1)−1

= H′(I + HH′)−1 = (I + H′H)−1H′.

⟩:=⟨ϕ, B−1ψ

⟩L2,⟨

f , g⟩

f ,R−1g⟩

⟩=⟨

f ,R−1Hϕ⟩

L2=⟨

R−1f ,Hϕ⟩

L2=⟨

H∗R−1f , ϕ⟩

L2=⟨

BH∗R−1f , ϕ⟩

H′f , ϕ⟩.

This leads to

H′ = BH∗R−1 (10)

and thus

BH∗(R + HBH∗)−1 = BH∗R−1(I + HBH∗R−1)−1

= H′(I + HH′)−1 = (I + H′H)−1H′.

⟩:=⟨ϕ, B−1ψ

⟩L2,⟨

f , g⟩

f ,R−1g⟩

⟩=⟨

f ,R−1Hϕ⟩

L2=⟨

R−1f ,Hϕ⟩

L2=⟨

H∗R−1f , ϕ⟩

L2=⟨

BH∗R−1f , ϕ⟩

H′f , ϕ⟩.

This leads to

H′ = BH∗R−1 (10)

and thus

BH∗(R + HBH∗)−1 = BH∗R−1(I + HBH∗R−1)−1

= H′(I + HH′)−1 = (I + H′H)−1H′.

Variational Data Assimilation 4dVar

Outline

4dVar - 1 (functional)

Functional on time interval, minimize by a Gradient Method

J(ϕ) = ||ϕ− ϕ(b)||2B−1 +K∑

||fk − Hϕk ||2R−1 , ϕk = M[t0, tk ]ϕ

where M[s, t] denotes the model operator from time s to time t .

4dVar - 2 (cycling)

Functional

J(ϕ) = ||ϕ− ϕ(b)` ||

B−1 +K∑

||fk − Hϕ`+k ||2R−1 , ϕ`+k = M[t`, t`+k ]ϕ

for ` = 0, K , 2K , 3K , ...

Minimizer gives the analysis ϕ(a)` , next background is

ϕ(b)`+K = M[t`, t`+K ]ϕ

(a)` (11)

4dVar - 3 (with linear M rewrite as Tikhonov)

Assume that M is linear. Then by

H`ϕ :=

HM[t`, t`+1]ϕHM[t`, t`+2]ϕ

HM[t`, t`+K ]ϕ

f`+2...

we can rewrite 4dVar as a three-dimensional assimilation algorithm solving

Hϕ = f`. (13)

The regularized solution provides ϕ(a)` , from which ϕ

(b)`+K is calculated by an application

of M[t`, t`+K ].

Variational Data Assimilation Spectral Representation

Outline

Spectral Representation 1

Assume that H : X → Y is linear and compact. Then, there is a singular system{(ψn, gn, µn), n ∈ N} such that Hψn = µngn, H′gn = µnψn and

H′Hψn = µ2nψn, (14)

This yields

(αI + H′H)ψn = (α + µ2n)ψn, n ∈ N. (15)

With measurements

f =∞∑

fngn ∈ Y (16)

and denote the spectral coefficients of the Tikhonov solution ϕ by γn. 3dVar or Tikhonov

regularization, respectively, (αI + H′H)ϕ = H′f is equivalent to the spectral dampingscheme

γn =µn

α + µ2n

fn, n ∈ N. (17)

This yields

(αI + H′H)ψn = (α + µ2n)ψn, n ∈ N. (15)

With measurements

f =∞∑

fngn ∈ Y (16)

γn =µn

α + µ2n

fn, n ∈ N. (17)

This yields

(αI + H′H)ψn = (α + µ2n)ψn, n ∈ N. (15)

With measurements

f =∞∑

fngn ∈ Y (16)

γn =µn

α + µ2n

fn, n ∈ N. (17)

The true Inverse is

γ truen =

µnf truen . (18)

This inversion is unstable, if µn → 0, n→∞!

Tikhonov regularization is stable for α > 0

γn =µn

α + µ2n

fn, n ∈ N. (19)

Tikhonov shifts the eigenvalues of H′H by α.

The true Inverse is

γ truen =

µnf truen . (18)

γn =µn

α + µ2n

fn, n ∈ N. (19)

The true Inverse is

γ truen =

µnf truen . (18)

γn =µn

α + µ2n

fn, n ∈ N. (19)

Convergence Analysis for Cycled Assimilation 1: Range Arguments Setup: Constant System

Outline

A system with constant dynamics

As a simple model system for study we use constant dynamics M = Identity , i.e

ϕ(b)k+1 = ϕ

(a)k , k = 1, 2, 3, ... (20)

for 3dVar. Also, we employ identical measurements fk ≡ f , k ∈ N.

Then, 3dVar is given by the iteration

ϕk = ϕk−1 + (αI + H′H)−1H′(f − Hϕk−1), k = 1, 2, 3, ... (21)

(This coincides with work of Engl on ’iterated Tikhonov regularization’!)

For the spectral coefficients γn,k of ϕk this leads to the iteration

γn,k = γn,k−1 +µn

α + µ2n

(fn,k − µnγn,k−1) (22)

ϕ(b)k+1 = ϕ

(a)k , k = 1, 2, 3, ... (20)

ϕk = ϕk−1 + (αI + H′H)−1H′(f − Hϕk−1), k = 1, 2, 3, ... (21)

α + µ2n

(fn,k − µnγn,k−1) (22)

ϕ(b)k+1 = ϕ

(a)k , k = 1, 2, 3, ... (20)

ϕk = ϕk−1 + (αI + H′H)−1H′(f − Hϕk−1), k = 1, 2, 3, ... (21)

α + µ2n

(fn,k − µnγn,k−1) (22)

Spectral Formula I

We employ

f = Hϕ(true) + δ, fn,k = µnγ(true)n + δn. (23)

and obtain

γn,k = γn,k−1 +µ2

α + µ2n

(γ(true)n − γn,k−1) +

α + µ2n

Theorem (Spectral Formula I)

The 3dVar cycling for a constant dynamics with identical measurements

f = Hϕ(true) + δ lead to the spectral update formula

γn,k = (1− qn)γ(true)n + qnγn,k−1 +

(1− qn)

µnδn (24)

qn =α

α + µ2n

= 1− µ2n

α + µ2n

. (25)

Spectral Formula II

Theorem (Spectral Formula II)

The 3dVar cycling for a constant dynamics with identical measurements

f = Hϕ(true) + δ can be carried out explicitly. The development of its spectral

coefficients is given by

γn,k = (1− qkn )γ(true)

n + qknγn,0 +

(1− qkn )

µnδn (26)

qn =α

α + µ2n

= 1− µ2n

α + µ2n

. (27)

Proof. Induction over k. �

Convergence Analysis for Cycled Assimilation 1: Range Arguments Convergence for Data f ∈ R(H) / Divergence for f /∈ R(H)

Outline

Convergence for f ∈ R(H)

Theorem (Convergence for f ∈ R(H))

Cycled 3dVar for a constant dynamics and identical measurements f (true) + δ ∈ R(H)tends to the true solution ϕ(true) + σ with Hσ = δ for k →∞.

Proof. We study

n + qknγn,0 +

(1− qkn )

µnδn (28)

for k →∞.

Since 0 < qn < 1, we have

qkn → 0, k →∞, (1− qk

n )→ 1, k →∞. (29)

Since δ = Hσ the element σ with spectral coefficients δn/µn is in X and cycled 3dVar

converges towards ϕ(true) + σ. �

Proof. We study

n + qknγn,0 +

(1− qkn )

µnδn (28)

for k →∞.

qkn → 0, k →∞, (1− qk

n )→ 1, k →∞. (29)

Proof. We study

n + qknγn,0 +

(1− qkn )

µnδn (28)

for k →∞.

qkn → 0, k →∞, (1− qk

n )→ 1, k →∞. (29)

Divergence for f 6∈ R(H)

Theorem (Divergence for f 6∈ R(H))

For a constant dynamics and identical measurements f (true) + δ 6∈ R(H) cycled 3dVardiverges for k →∞.

Proof. We study

n + qknγn,0 +

(1− qkn )

µnδn (30)

for k →∞. Let σk ∈ X denote the element with spectral coefficients

σn,k =(1− qk

µnδn, k, n ∈ N.

which is well defined since for every fixed k ∈ N∣∣∣(1− qkn )

∣∣∣ =∣∣∣(α + µ2

n)k − αk

(α + µ2n)kµn

∣∣∣ (31)

is bounded uniformly for n ∈ N.

Theorem (Divergence for f 6∈ R(H))

For a constant dynamics and identical measurements f (true) + δ 6∈ R(H) cycled 3dVardiverges for k →∞.

Proof. We study

n + qknγn,0 +

(1− qkn )

µnδn (30)

for k →∞. Let σk ∈ X denote the element with spectral coefficients

σn,k =(1− qk

µnδn, k, n ∈ N.

which is well defined since for every fixed k ∈ N∣∣∣(1− qkn )

∣∣∣ =∣∣∣(α + µ2

n)k − αk

(α + µ2n)kµn

∣∣∣ (31)

is bounded uniformly for n ∈ N.Roland Potthast 28/43

Since δ 6∈ R(H) we know that

SL :=L∑

∣∣∣ δn

∣∣∣2 →∞, L→∞. (32)

Given C > 0 we can choose L such that SL > 2C. Then

||σk ||2 ≥L∑

∣∣∣(1− qkn )δn

∣∣∣2 > C (33)

for k ∈ N sufficiently large, which proves

||σk || → ∞, k →∞ (34)

and the proof is complete. �

Numerical Example: Dynamic Magnetic Tomography

2.36x 10

time index k

Convergence Analysis 2: Regularization of Assimilation Setup: High-Frequency Damping Systems

Outline

ϕ(a)k+1 − ϕ

(true)k+1 = M(ϕ

(a)k − ϕ

(true)k ) + RαHM

(ϕ(true)k − ϕ(a)

)+ Rαf

(δ)k+1

= (I − RαH)M(ϕ(a)k − ϕ

(true)k

)+ Rαf

(δ)k+1. (35)

We abbreviate the analysis error by

ek := ϕ(a)k − ϕ

(true)k , k = 1, 2, ... (36)

and define

Λ := (I − RαH)M (37)

to obtain the iteration formula

ek+1 = Λek + Rαf(δ)k+1, k = 0, 1, 2, ... (38)

System Evolution

Theorem

Assume that the error f(δ)k does not depend on k, i.e. that we feed some constant error

into the data assimilation scheme. Then, the error terms ek with initial error e0 described

by the update formula (38) evolve according to

ek = Λk e0 +( k−1∑`=0

e(δ) (39)

with e(δ) := Rαf (δ). If (I − Λ)−1 exists, it can be written as

ek = Λk e0 + (I − Λ)−1(I − Λk )e(δ) (40)

The update formula has been already known in the 1970s.

It was derived for a finite dimensional system with some well-posed observation

operator in a stochastic framework for the Kalman filter.

Frobenius Norm

Let {ψ` : ` ∈ N} be a complete orthonormal system in X . Then, any vector ϕ ∈ X

can be decomposed into its Fourier sum

ϕ =∞∑`=1

α`ψ` (41)

with α` := 〈ϕ,ψ`〉 for ` ∈ N. We apply this to Mϕ to derive

Mϕ =∞∑`=1

〈Mϕ,ψ`〉ψ` =∞∑`=1

∞∑j=1

αj〈Mϕj , ψ`〉ψ` =∞∑`=1

∞∑j=1

ψ`M`,jαj .

with the infinite matrix M`,j = 〈Mψj , ψ`〉. The Frobenius norm of M with respect to the

orthonormal system {ψ` : ` ∈ N} is defined as

||M||Fro :=∞∑`=1

∞∑j=1

|M`,j |2. (42)

Frobenius Systems: Damping of High Frequencies

If M is self-adjoint with respect to the scalar product 〈·, ·〉, then its Frobenius norm (42)

is independent of the orthonormal system {ψ` : ` ∈ N}.

The identity operator I does not have a bounded Frobenius norm.

Systems with bounded Frobenius norm are damping on higher modes or

frequencies.

Convergence Analysis 2: Regularization of Assimilation Analysis Error Bounds

Outline

Space Decomposition

Let the orthonormal system {ψ` : ` ∈ N} in X be given by the singular system of the

observation operator H : X → Y . In this case we define an orthogonaldecomposition of the space X by

X(n)1 := span{ψ1, ..., ψn}︸︷︷︸

lowermodes

, X(n)2 :=

highermodes︷︸︸︷span{ψn+1, ψn+2, ...} . (43)

Using the orthogonal projection operators P1 of X onto X1 and P2 of X onto X2, we

decompose M into

M1 := P1M, M2 := P2M. (44)

Using N = (I − RαH) and Λ = NM we obtain

Λ = N|X1 M1︸︷︷︸lower modes

higher modes︷︸︸︷N|X2 M2 . (45)

Norm estimates for N

The operator N maps Xj , j = 1, 2, into itself. This leads to the norm estimate

||Λϕ||2 = ||(N|X1 M1 + N|X2 M2)ϕ||2

= ||N|X1 M1ϕ||2 + ||N|X2 M2ϕ||2, (46)

leading to

||Λ||2 ≤ ||N|X1 ||2||M1||2 + ||N|X2 ||

2||M2||2. (47)

We derive estimates for all the above terms.

Norm Estimates 1

||Λ||2 ≤ ||N|X1 ||2||M1||2 + ||N|X2 ||

2||M2||2. (48)

The norm of the operator N|X2 is given by

||N|X2 || = 1 (49)

for all n ∈ N and α > 0.

Assume that M is self-adjoint with bounded Frobenius norm. Then, given ρ > 0 there is

n ∈ N such that for M2 = M(n)2 we have

||M2|| < ρ. (50)

Norm Estimates 2

||Λ||2 ≤ ||N|X1 ||2||M1||2 + ||N|X2 ||

2||M2||2. (51)

We need the freedom

||M1|| = c. (52)

On X1 for N = I − RαH we have the norm estimate

||N|X1 || = sup`=1,..,n

α + µ2n

| (53)

where µn are the singular values of the operator H ordered according to their size and

multiplicity. In particular, given ε > 0 and n ∈ N we can choose α > 0 sufficiently

small such that

||N|X1 || < ε. (54)

Stabilization of data assimilation

Recall

ek = Λk e0 + (I − Λ)−1(I − Λk )e(δ) (55)

Theorem

Assume that the system M is self-adjoint and its Frobenius norm is bounded and let αdenote the regularization parameter for a cycled data assimilation scheme. Then, for

α > 0 sufficiently small, we have ||Λ|| < 1. Assume that the observation error f (δ) is

bounded in norm by δ > 0. In this case, the analysis error is bounded over time with

lim supk→∞

||ek || ≤||Rα||δ

1− ||Λ||. (56)

Numerical Example

Summary Summary

Summary

Summary on DA Instability

3dVar (and 4dVar for linear systems) can be analysed as a cycled Tikhonovregularization.

We have shown that even for very stable dynamics where M = const cycled

Tikhonov with ill-posed observation operators diverges if we have data f 6∈ R(H).

The divergence carries over to 3dVar and 4dVar for simple systems.

For systems damping high frequencies we have shown that stability depends on

the choice of the regularization parameter α

For self-adjoint systems M with bounded Frobenius norm we can always achieve a

stable assimilation by choosing α appropriately, even if ||M||2 > 1.

Summary Summary

Summary

Summary on DA Instability

3dVar (and 4dVar for linear systems) can be analysed as a cycled Tikhonovregularization.

We have shown that even for very stable dynamics where M = const cycled

Tikhonov with ill-posed observation operators diverges if we have data f 6∈ R(H).

The divergence carries over to 3dVar and 4dVar for simple systems.

For systems damping high frequencies we have shown that stability depends on

the choice of the regularization parameter α

For self-adjoint systems M with bounded Frobenius norm we can always achieve a

stable assimilation by choosing α appropriately, even if ||M||2 > 1.

on instability in data assimilation · motivation and introduction data assimilation mathematical...

Documents

theme 'gwr': data assimilation - university of...

data assimilation - european space agency · 2013. 10....

introduction to data assimilation€¦ · 2019-12-16 ·...

hydrologic data assimilation

ensemble data assimilation: perturbing the …...9th-adjoint...

about data assimilation

altimeter data assimilation

an overview on data assimilation - epic · • data...

data assimilation theory ctcd data assimilation workshop...

radiation belt data assimilation: overview and...

tropical cyclone modeling and data assimilation...hwrd data...

coping with model error in variational data assimilation...

operational data assimilation at ecmwf - national...

data assimilation for state and parameter estimation ... ·...

4.2 land surface data assimilation · 2.4 land data...

data assimilation

overview: data assimilation and model reduction › lms ›...

lecture ii-1: data assimilation overview lecture outline:...

dod cg/ar at colorado state university ams data assimilation...

an introduction to data assimilation - eccorev · data...