1

Introduction to Adjoint Methods in Meteorology

Rolf Langland

Data Assimilation Section

Naval Research Laboratory

Monterey, CA langland@nrlmry.navy.mil

Including Material Provided by

Dr. Ronald M. Errico (NASA-UMBC)

Santa Fe, N.M., 31 July 2012JCSDA Summer Colloquium on Data Assimilation

2

Adjoint Methods Introduction

1. What is an adjoint model

2. Examples of adjoint equations

3. Interpretation of adjoint sensitivity

4. Development / testing of adjoint code

5. Misunderstandings about adjoint methods

Please ask questions during presentation if something is not clear!

3

What is an Adjoint Model? [NWP context]

• The transpose of a tangent linear version of a forecast model

• It can be used to estimate the sensitivity of a forecast aspect (J) with respect to model initial conditions and parameters

• Sensitivity information is useful for short-range forecasts (72 hr or less), subject to tangent-linear approximations

4

Why use adjoint models ?

• Require information about sensitivity to initial conditions, but cannot construct the inverse of a numerical forecast model …

• An adjoint model can provide good estimates of initial condition sensitivity, consistent with actual dynamics of nonlinear forecast model (group velocity, etc.)

• Difficult or impossible to obtain this information by other methods

5

Adjoint Applications

• Sensitivity of Forecast to Initial Conditions and Boundary Conditions – Key analysis errors

• Observation Impact Information • Targeted Observing Guidance• Variational Data Assimilation – 4D-Var• Generation of Perturbations for Ensemble

Forecasting• Optimal Perturbations and Singular Vectors

6

• Global weather prediction models • Regional models• Ocean models• Data assimilation procedures • Observation operators

Adjoint models exist for:

Adjoint codes are used in one form or another at every operational NWP forecast center

7

Forecast and Analysis Procedure

Observation(y) Data

AssimilationSystem

Forecast Model

Forecast(xf)

Gradient ofCost FunctionJ: (J/ xf)

Background(xb)

Analysis(xa)

Adjoint of theForecast Model Tangent Propagator

ObservationSensitivity(J/ y)

BackgroundSensitivity(J/ xb)

AnalysisSensitivity(J/ xa)

Observation Impact<y-H(xb)> (J/ y)

Adjoint of the Data AssimilationSystem

What is the impact of the observations on measures of forecast error (J) ?

Adjoint of Forecast and Analysis Procedure

8

Data Assimilation Equation with K in observation space

OBSERVATIONS Temperature

Winds

Pressure

BACKGROUND (6h) FORECAST ANALYSIS

T T 1a b b b b[ ] ( ) x x P H HP H R y Hx

K

Post-multiplier Solver (self-adjoint)

The analysis, Xa can be changed by perturbations of the observations (δy) or the background (δ Xb)An adjoint can be used to quantify this sensitivity

9

Sensitivity to Observations:

Sensitivity to Background:

Adjoint of Data Assimilation Equation

Adjoint of forecast model produces sensitivity to

ax

T 1b b

a

[ ]J J

HP H R HP

y x

T

b a

J J J

H

x x y

TK

see Baker and Daley 2000, QJRMS

10

Adjoint Operator - Linear algebra 101

*, ,y x y xL L The inner (dot) product of two vectors = a scalar,

If L is an m x n matrix, then L* = LT

Note: transpose is not the same as inverse !

11

TLM and Adjoint Equations

0fX L XThe linear operator L propagates a perturbation vector forward in time

T

0 f

J JLX X

The adjoint operator LT propagates a sensitivity gradient vector backward in time – sensitivity of J to all elements of X at initial time

sensitivity gradientsinitial time final time

perturbations of state variablesfinal time initial time

TLM

Adjoint

L is a tangent linear version of a nonlinear model, M

LT is the transpose of L

12

Forecast Response Function (J)

Examples:• Surface pressure• Temperature, wind component, specific humidity• Vorticity, divergence, enthalpy• Kinetic energy• Forecast error (f –a) or (f-a)2

• Energy-weighted forecast error norm

J can be any differentiable function of the model state variables that comprise Xf

Not valid: J=Anomaly Correlation Coefficient undefined

1313

Estimating δJ using TLM and Adjoint

Equivalent to a 1st-order Taylor Series

TLM

Adjoint

𝛿 𝐽=⟨ 𝜕 𝐽𝜕𝐗 𝑓 ,𝐋𝛿𝐗 0 ⟩=⟨ 𝜕 𝐽

𝜕𝐗 𝑓 ,𝛿𝐗 𝑓 ⟩𝛿 𝐽=⟨𝛿𝐗 0 ,𝐋 T 𝜕 𝐽

𝜕𝐗 𝑓 ⟩=⟨𝛿𝐗 0 , 𝜕 𝐽𝜕𝐗 0 ⟩

δx0 = can be any real or hypothetical perturbation

14

Obtain sensitivity to initial conditions using an adjoint model

1. The trajectory (analysis and forecast values of state variables) of the nonlinear forecast model are saved (at every time step if possible) from t=0 to t=f

2. The adjoint cost function (J) is defined

J J/T, J/u, J/v, J/ps at t=f

3. The adjoint model is integrated backwards in time to obtain:

J/T, J/u, J/v, J/ps at t=0

15

Adjoint sensitivity exampleNavy COAMPS model

J = kinetic energy in lower troposphere

Animation is at 700hPa, from 36hr to 0 hr

16

Forward (conventional) vs. adjoint sensitivity procedure

In a forward-in-time sensitivity procedure, we change the initial conditions (or observations) and evaluate the effect on the forecast (OSE)

In an adjoint sensitivity procedure we select the forecast aspect (J) and evaluate the sensitivity to the initial conditions (or observations)

17

A single adjoint-derived sensitivity yields linearized estimates of the particular measure (J) investigated with respect to all possible perturbations.

Adjoint Sensitivity Analysis Impacts vs. Sensitivities

A single impact study yields exact response measures (J) for all forecast aspects with respect to the particular perturbation investigated.

18

The forecast error norm

A useful way to combine errors of wind, temperature, humidity and surface pressure into a costfunction

𝐽=Ʃ 𝑖 , 𝑗 ,𝑘0.5 [𝐂u (u f −ua )2+𝐂 v ( v f −v a )2+𝐂T (T f −T a )2+𝐂q (q f −q a ) 2+𝐂 p ( p f − p a ) 2 ] are weighting functions that transform the errors of winds, temperature, humidity and pressure into units of energy [J kg-1], accounting for grid volume size/mass …

The adjoint model starting condition is then …for example = , at all grid points (i,j,k), units = Jkg-1m-1s

Energy norm ref: Rabier et al. 1996, QJRMS

19

Sensitivity summary field

Then, to display of initial condition sensitivity, we can transform the gradients back into units of energy and combine the winds, temperature and pressure sensitivities

S0=Ʃ 𝑖 , 𝑗 ,𝑘[𝐂u❑ 𝜕 𝐽

𝜕𝑢0+𝐂v

❑ 𝜕 𝐽𝜕𝑣 0

+𝐂T❑ 𝜕 𝐽

𝜕 𝑇 0+𝐂q

❑ 𝜕 𝐽𝜕𝑞0

+𝐂p❑ 𝜕 𝐽

𝜕 p0 ] are inverses of the weighting functions, that transform the sensitivity gradients of winds, temperature, humidity and pressure into units of energy [J kg-1], accounting for grid volume size/mass

-1

-1 -1 -1 -1 -1

-1 -1 -1 -1

20

Sensitivity of NOGAPS 72-h forecast error to the initial T,u,v,ps fields00 UTC 10 February 2002

J kg-1S0

21

Defining an “Optimal” IC Perturbation Using the Adjoint Sensitivity Vector

1. The sensitivity vector ( 0J ) can be scaled by a linear factor ( )

( )0 0opt J C-1x

the components of 0x are the model prognostic variables, Temperature, Vorticity, Divergence, and Pressure

2. The scale factor is defined by the ratio of the energy-weighted

forecast error norm and the norm of the sensitivity gradient at initial time.

e; e0.5

;t t-1

0 0J J

CC

This is the scale factor that provides, in a tangent linear, perfect model, framework, the largest reduction in the forecast error defined by the adjoint

costfunction (J)

λ units are:

𝐾𝐽𝑘𝑔

𝑚𝑠𝐽𝑘𝑔 etc. -1

-1 -1

22

Optimal correction of initial 500mb temperature based on adjoint sensitivity gradient to improve 72-hr forecast of east coast storm

23

Optimal correction of initial 300mb u-wind based on adjoint sensitivity gradient to improve 72-hr forecast of east coast storm

24

300mb u-wind – Comparison of initial conditions -

Original IC – this produces large forecast error

“Corrected IC” – this produces small forecast error

25

L

L

NOGAPS Sea-Level Pressure Forecasts and Analyses72-hr forecast from 12Z 22 Jan 2000

Forecast with Adjoint-based IC Correction (+72hr)Operational Forecast (+72hr)

L

L

26

1 x 1.25 degree lat-lon grid 0.5 x 0.0625 degree lat-lon grid

Gradient of error energy with respect to Tv 24-hours earlier Units: Jkg-1K-1

Provided by R. Todling

Plots of sensitivity gradient magnitude and grid resolution

Sensitivity magnitude on this grid is larger because a perturbation of initial conditions at any grid point represents a larger area / volume

The same sensitivity gradient has smaller magnitude on this grid because of finer resolution –

The sum of sensitivity for equal areas is independent of grid resolution

27

Singular Vectors in 72-hr East Coast Storm Forecast

Initial Time Total Energy 12Z 22 Jan 2000

Final Time Sfc Pressure 12Z 25 Jan 2000

SV 1

SV 2

SV 3

28

du = - 0.5 [u v]dt

2dv = 0.5 udt

d (u + u ) = - 0.5 [(u + u ) (v + v )] = - 0.5 [ u v + u v u v + + u v ]dt

d (v + v ) = 0.5 [(u + u ) (u + u )] = 0.5 [ u u 2 u + + u u ]dt

u

Example of TLM and Adjoint Derivation

Nonlinear Equations

1st order Linearization

29

'du = - 0.5 [ u v + u v ]dt

'dv = u udt

Tangent Linear Equations

Adjoint Equations (LT)

Example of TLM and Adjoint Derivation

ˆ ˆ ˆdu = - 0.5 v u + u vdt

ˆ ˆdv = - 0.5 u udt

ˆ ˆJ Ju = v = u v

1 1

0.5 0.5' '' 'n n

vu uuv vu

1 1

0.5

0.5n n

vu uuv vu

T0

ˆ ˆL fX X

' 'f 0LX X

30

TLMNLM

Nonlinear model and tangent linear model trajectories

The trajectory of the TLM is an approximation of the nonlinear trajectory

The nonlinear forecast trajectory is saved at specified time intervals and used in the TLM

Time

The TLM is tangent to the nonlinear trajectory at every time step where an update is provided – it is not a purely linear calculation

31

Development of TLM and Adjoint Model Code

OPTIONS:

1. Develop TLM code directly from nonlinear model code, then develop adjoint code

2. Develop TLM versions of nonlinear model original equations, then develop TLM and adjoint code

Option 1 is the best method ….

32

1. Eventually a TLM and adjoint code will be necessary anyway

2. The code itself is the most accurate description of the model algorithm

3. If the model algorithm creates different dynamics than the original equations being modeled, for most applications it is the former that are desirable and only the former that can be validated

Why develop the TLM from the nonlinear model code?

33

Automatic Differentiation Software

TAMC Ralf Giering (superceded by TAF)TAF FastOpt.comADIFOR Rice UniversityTAPENADE INRIA, NiceOPENAD ArgonneOthers www.autodiff.org

Input: Nonlinear code Output: TLM and adjoint code

34

Considerations in development of TLM and Adjoint code

1. TLM and Adjoint models are straight-forward (although tedious) to derive from NLM code, and actually simpler to develop

2. Intelligent approximations can be made to improve efficiency 3. TLM and adjoint codes are simple to test rigorously4. Some outstanding errors and problems in the NLM are typically revealed when the TLM and Adjoint are developed 5. Some approximations to the NLM physics are generally necessary

35

TLM validation Comparison to nonlinear model

Does the TLM or Adjoint model tell us anything aboutthe behavior of perturbation growth in the nonlinearmodel that may be of interest?

36

Non-ConvectivePrecip. ci=0.5mm

Linear vs. Nonlinear results with moist physics, 24-hr forecasts

TLM

TLM

NLM

NLM

Non-ConvectivePrecip. ci=0.5mm

ConvectivePrecip. ci=0.2mm

ConvectivePrecip. ci=0.2mm

37

Adjoint model verification Gradient check

Adjoint model starting condition and IC sensitivity gradient

Note that the dot product in the above equation is computed for all i,j,k in the model domain. This is a fundamental test to determine if the TLM and adjoint are coded correctly

⟨𝛿𝐗 𝑓 , 𝜕 𝐽𝜕𝐗 𝑓 ⟩=⟨𝛿𝐗 0 , 𝜕 𝐽

𝜕𝐗 0 ⟩Evolved TLM perturbation and TLM starting condition

38

Adjoint method accuracy

• Quantitative accuracy best for small perturbations and short forecasts

• Often, qualitatively useful information can be obtained for large perturbations in forecasts as long as 3-5 days…for example, when applied to mid-latitude winter storms – synoptic-scales

• Accuracy is less for highly non-linear flows and smaller-scales

• Adjoint accuracy is equivalent to that of the TLM

39

Tangent Linear vs. Nonlinear Results

In general, agreement between TLM and NLM resultswill depend on:

1. Amplitude of perturbations2. Stability properties of the reference state3. Structure of perturbations4. Physics involved5. Time period over which perturbation evolves6. Metric used for comparison

40

Issues with physics in TLM and adjoint

1. Parts of the NLM code may be non-differentiable, requiring approximations in the TLM and adjoint

2. Numerical instabilities may occur in the TLM as a result of physics linearization

3. Some physical parameterizations are much more suitable than others for linearization

4. Development of the TLM and adjoint may uncover problems with the nonlinear model physics

41

Example of a transient instability in a TLM solution

Errico andRaeder 1999QJRMS

TLM

NLM

42

Adjoint models as paradigm changers

Some results discovered using adjoint models

1. Atmospheric flows are very sensitive to low-level T perturbations2. Evolution of a barotropic flow can be very sensitive to perturbations having small vertical scale3. Error structures can propagate and amplify rapidly 4. Forecast barotropic vorticity can be sensitive to initial water vapor 5. Relatively few perturbation structures are initially growing ones6. Sensitivities to observations differ from sensitivities to analyses

43

Misunderstanding # 1

False: Adjoint models are difficult to understand True: Understanding how to use and interpret adjoints of numerical models primarily uses concepts taught in early college mathematics

44

False: Adjoint models are difficult to develop True: Adjoint models of dynamical cores are simpler to develop than their parent models, and almost trivial to check, but adjoints of model physics can pose difficult problems

Misunderstanding # 2

45

False: Automatic adjoint generators easily generate perfect and useful adjoint models

True: Problems can be encountered with automatically generated adjoint codes that are inherent in the parent model. Do these problems also have a bad effect in the parent model?

Misunderstanding # 3

46

False: An adjoint model is demonstrated useful and correct if it reproduces nonlinear results for ranges of very small perturbations True: To be truly useful, adjoint results must yield good approximations to sensitivities with respect to meaningfully large perturbations. This must be part of the validation process

Misunderstanding # 4

47

False: Adjoints are not needed because the EnKF is better than 4DVAR and adjoint results disagree with our notions of atmospheric behavior True: Adjoint models have uses beyond 4DVAR. Their results can be surprising, but have been confirmed. It is rare that we have a tool that can answer such important questions so directly! It has not been demonstrated that EnKF is superior to TLM/adjoint for either data assimilation or sensitivity calculations.

Misunderstanding # 5

48

Challenges

1. Develop new adjoint models2. Include more physics in adjoint models3. Develop parameterization schemes suitable for linearized applications4. Always validate adjoint results (linearity)5. Many applications (sensitivity analysis, model

tuning, predictability research, etc.) for adjoint models are not being fully examined at the present time

49

Adjoint Workshops

Contact Dr. Ron Erricorerrico@gmao.gsfc.nasa.govto be put on mailing list

1992 – Pacific Grove, CA1995 – Visegrad, Hungary1998 – Lennoxville, Quebec2000 – Moliets-et-Maa, France2002 – Mount Bethel, PA2004 – Aquafredda di Maratea,Italy2006 – Tirol, Austria 2009 – Tannersville, PA2011 - Cefalu,Sicily 2013?

50

The Energy Norm