chaotic systems can synchronize despite sensitivity

The Synchronization Approach to Data Assimilation, Parameter Estimation

and Fusion of Climate ModelsGreg Duane

University of Coloradoand

University of Miami

Supported byNSF Grant #0838235

and #0327929

In collaboration with:Joe Tribbia (NCAR)Ben Kirtman (U. Miami)Jeff Weiss (U. Colorado)Ljupco Kocarev (UCSD)Eugenia Kalnay (U. Md.)Josh Hacker (NCAR)

CHAOTIC SYSTEMS CAN SYNCHRONIZEDESPITE SENSITIVITY

• two coupled chaotic systems can fall into synchronized motion along their strange attractors when linked through only one variable

(Pecora and Carroll ’90)

x’= (y-x)y’= x-y-xzz’= -z+xy

y1’= x-y1-x(z1)z1’= -z1)+x(y1)

z (t)

(also works for y-coupling, but not for z-coupling)

SUPPOSE THE WORLD IS A LORENZ SYSTEM AND ONLY X IS OBSERVED

• two coupled chaotic systems can fall into synchronized motion along their strange attractors when linked through only one variable

(Pecora and Carroll ’90)

x’= (y-x)y’= x-y-xzz’= -z+xy

y1’= x-y1-x(z1)z1’= -z1)+x(y1)

z (t)

(also works for y-coupling, but not for z-coupling)

“Truth” “Model”

TWO CHANNEL MODELS SYNCHRONIZE WHEN DISCRETELY COUPLED - makes weather prediction possible

(Duane and Tribbia, PRL ’01, JAS ‘04)

Part I: Treatment of Nonlinearities in the Synchronization Approach

Part II: Synchronization for Parameter Estimation, Model Learning and Fusion of Climate Models

SDE’s: dxA/dt = f (xA) dxB/dt = f (xB) + C (xA- xB + ) is white noise < (t) T(t’) > = R (t- t’)

linearize: de/dt = Fe – Ce + C e xA- xB F Df(xA) Df(xB)

Fokker-Planck eqn for PDF p(e): p/ t + e [p (F-C) e] = ½ (CTRCp)

Gaussian ansatz: p = N exp(-eTKe) pdne = 1 p/ t = 0

Choose C to minimize the spread B (2K)-1 of the distribution.

Fluctuation-Dissipation Relation: B (C-F)T + (C-F) B = CRCT

for C C + dC (dC arbitrary), let dB be such that B B + dB dB=0 if C= Copt = (1/ B R-1

Analysis: Synchronization with Noisy Coupling

Standard Data Assimilation As a Continuous Process (as in Einstein’s treatment of Brownian motion)

Standard methods: xA=xbkd + [B(B+R)-1](xT - xbkd + noise) (perfect model) dxT/dt = f(xT) dxbkd/dt = f(xbkd) + (1/[B(B+R)-1](xT - xbkd + ) + O[ (B(B+R)-1)2] = f(xbkd) + (1/BR-1 (xT - xbkd + )is the time between analyses in incremental data assimilation

The coupling C= (1/ B R-1 = Copt

So, the standard methods of data assimilation (3DVar, Kalman Filtering) are also optimal for synchronization under local linearity assumption!

(Exact treatment of discrete analysis cycle as a map gives Copt= (1/ B (B+R)-1. )

OPTIMAL COUPLING IN FULLY NONLINEAR CASE

de/dt = (F-C)e + Cξ + Ge2 + He3+ ξM ansatz: p=N exp(Ke2+Le3+Me4)

Model error covariance Q=< ξM ξMT>

p/ t + e [(F-C) e + Ge2 + He3 ]p = ½ (CTRCp)

In one dimension, Fokker-Planck eqn (F-C)e + Ge2 + He3 = ½ C2R (-2Ke – 3Le2 – 4Me3) F-C = ½ C2R (-2K) G = ½ C2R (-3L) H = ½ C2R (-4M) background error B= B(K,L,M) = ∫e2p(e)de = B(K(C),L(C),M(C))optimize B as a function of C general correction to KF

If we restrict form of C, e.g. C=F BR-1 cov. inflation factor F

Choose G and H so that the dynamics are those of motion in a two-well potential:

dx/dt = f (x)

e.g. for d1,d2 matching the distances between the fixed points in the Lorenz ’84 system with F=1, one finds G = .15 H = -.75

Minimize background error B as a function of coupling CFind C = 1.51, B=0.145

If C = F B/(R), then we have a covariance inflation factor F= 1.04 (where R=1, = 0.1)

f

B

C

No model error (Q=0):

Model error equal to50% of the resolvedtendency:

The need for inflation is shaped by the nonlinearities, regardless of the amount of model error.

d1

d1

d2

d2

WHAT ABOUT SAMPLING ERROR?

Suppose undersampling uncertainty in estimate of B

multiplicative noise in assimilation

dxT/dt = f(xT) dxbkd/dt = f(xbkd) + (1/[B(B+R)-1+ ξS](xT - xbkd + )

Fokker Planck equation: S

2= =< ξS ξST>

p/ t + e [p (F-C + ½S2) e]

= ½ (CTRC+ S2 e2 - 2 Se/ (CTRC)p

Use change of variables p’= p(CTRC+ S2 e2 - 2 Se (CTRC)

Arguably, effect is small if S R-1

g

Multidimensional Case e.g. D=2

Consider two wells separated in one dimension.

Assume R= (can arrange by rescaling)

Choose a basis such that the dynamical equations are given bya direct product of motion in a two-well potential and simplelinear dynamics.

R is still diagonal.

The FP equation p/ t + e [p (F-C) e] = ½ (CTRCp) separates.

00r

r

Summary: Covariance Inflation in the Synchronization Approach

• In the synchronization approach, the rough magnitudes of covariance inflation factors used in practice might be explained from first principles

• Model error due to unresolved physics makes little difference; the requirement for inflation is shaped by nonlinearities in the dynamics

• Refinements may yield treatments of nonlinearities that improve on covariance inflation

Part I: Treatment of Nonlinearities in the Synchronization Approach

Part II: Synchronization for Parameter Estimation, Model Learning and Fusion of Climate Models

WHAT IF THE MODEL IS IMPERFECT?• can synchronize parameters as well as states

Lorenz system example:

add parameter adaptation laws: 1’= (y-y1) x1

’ = (y1-y) y1

’= y-y1• these augmented equations minimize a Lyapunov function V = ex

2 + ey2 + ez

2 + r2+r

2+r2

where ex = x-x1, ey=……….. r = -1, r=……. since it can be shown that dV/dt < 0, and V is bounded below

So as t→∞, (x1,y1,z1) →(x,y,z) and also 1→ , →1, → 0 i.e. the model “learns”

x’= (y-x)y’= x-y-xzz’= -z+xy

x1’= (y-x1)y1’= x1- y1-x1(z1)+z1’= -z1)+x1(y1)

General Rule for Parameter Estimation, If Systems Synchronize with Identical Parameters

dx/dt = f(x,p)dp/dt=0

dy/dt=f(y,q) + u(y,s) s=s(x) (30)dq/dt=N(y,x-y) (31)

ey-x rq-p h f(y,q)- f(y,p)

Truth:

Model:

(Duane , Yu, and Kocarev, Phys. Lett. A ‘06)

Example: A Column Model With an Unknown Surface Moisture Availability Parameter

Column model summary:

Parameter Adaptation Rule

-interpretation: decrease or increase M in proportion to the covariance between the synchronization (forecast) error and the factor multiplied by M in the dynamical equations

Ý M ~ Kz

f (um0,Tm0,......)(qobs(z) qm (z))

Adapt M according to:

Prognostic equation for humidity:

qm

t

z

{K(qm

z Mf (um0,Tm0,....))} c(qobs qm )

nudging term

soil moisturemoisture availability parameter

RESULTS

-alternating periods of slow convergence to synchronization and rapid ``bursts” away

-apparently can always identify the true value of M

time

M-MT

observations at 7 points in columnnudging at 1 pointnudging coeffiicient = .01

….other configurations show same pattern

…as previous, but with nudging coefficient = .015

observations and nudgingat 7 points,coefficient = .0025

observations and nudgingat 4 points,coefficient = .015

Actual details of model asimplemented in softwarewere unknown!

……..because the state variables also do not convergecompletely in the time interval shown

qm

qT

Single-Realization vs Ensembles

• in principle, should be able to replace ensemble averages with time averages to estimate relatively constant quantities (cf. ergodicity)

• “learn on the fly” → AI view of data assimilation

• compare to “Lagged Average Forecasting” (Hoffman and Kalnay ‘83 ): use a single realization with different initialization times to create an artificial ensemble

Which parameters should we adapt?

TAKE A COLLECTION OF THE BEST MODELS, COUPLE THEM TO ONE ANOTHER, AND ADAPT THE COUPLING COEFFICIENTS

Ki constant: data assimilation

-couple corresponding “model elements” l

adapt Clij:

learning

CONSENSUS

Test Case: Fusing 3 Lorenz Systems With Different Parameters

adaptingnot

adaptingCl

ij=0time time time

z Mav

g -zT

Fused ModelsAverage Output

of Models (Unfused)z from Model

With Best z Eqn

z Mav

g -zT

z Mbe

st-z

T

- Model fusion is superior to any weighted averaging of outputs

dCxij/dt = a(xj-xi)(x – ⅓∑xk)

dCyij/dt=…….

dCzij/dt=…….

Parameter Adaptation in the QG Channel Model

n=0“truth ” A “model” B

foB’=∫(q*-qB)(qA-qB)d2x

timestep n

foB→fo

A

Add terms to FB toassimilate medium scales of A.

Then adapt foB:

What if foB ≠ fo

B ?

Proposed Adaptive Fusion of Different Channel Models

* + *’2

(k-dependence suppressed)

Fo =fo(q-q*)Fo’=fo(q’-q*’)

• If the parallel channelssynchronize, their commonsolution also solves thesingle-channel model withthe average forcing

forcing in Atlantic

forcingin Pacific

To find c adaptively: dc/dt = ∫d2x J(,q’-q)(q-qobs) + ∫d2x J(’,q-q’)(q’-qobs)

Annual Mean SST Temperature

COLA-MOM3

Observations

CAM-MOM3

Longitude

oC

FUSION OF REAL CLIMATE MODELS

typicalscenario:

CAM

MOM

Heat Flux

Momentum Flux

SST

CAM_MOM3

COLA

MOM

Heat Flux

Momentum Flux

SST

COLA_MOM3

CAM COLA

MOM

Heat Flux

Momentum Flux

SST

CAM_COLA_MOM3

“Interactive Ensemble”

CAM COLA

MOM

Heat Flux

Momentum Flux

SST

COLA_CAM_MOM3

“Interactive Ensemble”

Heat Flux: COLA; Momentum Flux: CAM

Heat Flux: CAM; Momentum Flux: COLA

Observations

COLA_MOM3 CAM_MOM3

Heat Flux: COLA; Momentum Flux: CAM Heat Flux: CAM; Momentum Flux: COLA

COLA_MOM3 CAM_MOM3

Heat Flux: COLA; Momentum Flux: CAM Heat Flux: CAM; Momentum Flux: COLA

All Model Error

COLA_MOM3 CAM_MOM3

Heat Flux: COLA; Momentum Flux: CAM Heat Flux: CAM; Momentum Flux: COLA

COLA Heat Flux Errors

COLA_MOM3 CAM_MOM3

Heat Flux: COLA; Momentum Flux: CAM Heat Flux: CAM; Momentum Flux: COLA

Error Amplified by CAM Momentum Flux

COLA_MOM3 CAM_MOM3

Heat Flux: COLA; Momentum Flux: CAM Heat Flux: CAM; Momentum Flux: COLA

CAM Heat Flux Error

INFERENCES ABOUT SOURCES OF ERROR WERE USED TO FORM

A FUSED CAM-COLA MODEL

Guiding principle: For each model element, make the choice of model that reduces truth-model synchronization error

-simplified form of the automated adjustment of coupling coefficients (which need not be binary) proposed here

Adaptive Consensus Formation Approach is Empirical

-reminiscent of learning in neural networks: Hebb’s rule: “Cells that fire together, wire together” here: Model elements “wire together” directionally, until they collectively ``fire” in sync with reality

-Can the role of synchronization in the consensus formation scheme be compared to its proposed role in consciousness, via the highly intermittent synchronization of the 40 Hz oscillation in widely separated regions of the brain?

Conclusion: Adaptive consensus formation among models can likely reduce error in long-range climate forecasts

But what if the dynamical parameters change drastically in the 21st

century as compared to the training period?

Lorenz test case:

Other possible issues: -local vs. global optima in coupling coefficients -climate vs. weather prediction

adaptation

=28 =50 =100

Average of outputs (unfused)Fusion

Attractors

Suggestive of Measure Synchronization……-in jointly Hamiltonian systems, trajectories can become the same, while states differ at any instant of time (Hampton & Zanette PRL ‘99)

-Afraimovich et al. ‘97: “nonisochronic synchronization” of dissipatively coupled systems: