2005 ROMS/TOMS Workshop2005 ROMS/TOMS WorkshopScripps Institution of OceanographyScripps Institution of Oceanography

La Jolla, CA, October 25, 2005La Jolla, CA, October 25, 2005

ean M od

earch C o m

r a i n - F o l l o w

M o d e l i n g

How Does One Build anHow Does One Build anAdjoint for ROMS/TOMS?Adjoint for ROMS/TOMS?

Hernan G. ArangoIMCS, Rutgers

Andrew M. MoorePAOS, U. Colorado

Emanuele Di LorenzoGeorgia Tech

Bruce D. CornuelleSIO, UCSD

Arthur J. MillerSIO, UCSD

• To design, develop, and test an expert ocean To design, develop, and test an expert ocean modeling system for high-resolution scientific modeling system for high-resolution scientific and operational applications over a wide range and operational applications over a wide range of scales from estuaries to regional to global.of scales from estuaries to regional to global.

• To provide the ocean modeling community with To provide the ocean modeling community with analysis and prediction tools that are available in analysis and prediction tools that are available in meteorology and Numerical Weather Prediction meteorology and Numerical Weather Prediction (NWP)(NWP) . .

Long-Term Goals

Are we closer to operational weather predictionAre we closer to operational weather predictionsystems?systems?

ObjectivesObjectives

• To explore the factors that limit the To explore the factors that limit the predictabilitypredictability of of

the circulation in regional models in a variety of the circulation in regional models in a variety of

dynamical regimesdynamical regimes

• To build 4D variational assimilation platforms: strong To build 4D variational assimilation platforms: strong

and weak constraint and weak constraint 4DVAR4DVAR

• To build a Generalized Stability Theory (GST) analysis To build a Generalized Stability Theory (GST) analysis

platforms to study the platforms to study the dynamicsdynamics, , sensitivity and sensitivity and

stabilitystability of the ocean circulationof the ocean circulation to naturally occurring to naturally occurring

perturbationsperturbations

• To build an To build an ensemble predictionensemble prediction platform by platform by

perturbing forcing, initial, and boundary conditions perturbing forcing, initial, and boundary conditions

with GST singular vectorswith GST singular vectors

ROMS/TOMS Framework

ADSEN_OCEAN

SANITY CHECK S

PERT_OCEAN

PICARD_OCEAN

GRAD_OCEAN

TLCHECK _OCEAN

RP_OCEAN

ESMF

AIR_OCEAN

M

AS

TE

RWAVE S _OCE AN

OCE AN IN IT IA L IZE

F IN A L IZE

RU N

S4DVAR_OCEAN

IS4DVAR_OCEAN

W4DVAR_OCEAN

ENSEMBLE_OCEAN

NL_OCEAN

TL_OCEAN

AD_OCEAN

PROPAGATOR

K ER NELNLM, T LM, RP M, ADM

----

phys ic sbiogeochemic al

sedimentsea ic e

-

-

--

-

-

-

Optimal pertubationsADM eigenmodes

TLM eigenmodes

Forc ing singular vectorsStochastic optimals

Balance Truncation vectors

EOF’s- P seudospec tra

• ROMS/TOMS version 2.2 released to full user ROMS/TOMS version 2.2 released to full user community and version 3.0 released beta testers community and version 3.0 released beta testers on on May 26, 2005May 26, 2005. Currently, ROMS and TOMS are . Currently, ROMS and TOMS are identical.identical.

• Rewrote tangent linear (Rewrote tangent linear (TLMTLM), representer (), representer (RPMRPM) ) and adjoint (and adjoint (ADMADM) models in Fortran 90 to ) models in Fortran 90 to improve the efficiency and multiple levels of improve the efficiency and multiple levels of nesting.nesting.

• Parallelized Parallelized TLMTLM, , RPMRPM and and ADMADM..

• Designed a single makefile structure to facilitate Designed a single makefile structure to facilitate compiling in any computer architecture:compiling in any computer architecture:

AccomplishmentsAccomplishments

477 files, 340514 lines of code, 1093905 477 files, 340514 lines of code, 1093905 words, 13440936 characterswords, 13440936 characters

• Continued to develop web-based documentationContinued to develop web-based documentationhttp://www.ocean-modeling.org/http://marine.rutgers.edu/po/models/roms/index.php

How to Build and AdjointHow to Build and Adjoint

• Build tangent linear model by linearizing the nonlinear Build tangent linear model by linearizing the nonlinear model around a small perturbationmodel around a small perturbation

• It is called tangent because the linearization is around a It is called tangent because the linearization is around a time-evolving solution which geometrically represents the time-evolving solution which geometrically represents the tangent slopes to the tangent slopes to the NLM NLM trajectory in phase spacetrajectory in phase space..

• The The TLMTLM is hand-coded using Giering and Kaminski (1998) is hand-coded using Giering and Kaminski (1998) recipesrecipes..

NLM

TLM

Adjoint OperatorAdjoint Operator

• The discrete adjoint model operator relative to the L2-The discrete adjoint model operator relative to the L2-norm can be derived by multiplying each line of the norm can be derived by multiplying each line of the tangent linear model code by the corresponding adjoint tangent linear model code by the corresponding adjoint variable, , and then differentiate with respect to the variable, , and then differentiate with respect to the tangent linear variable, :tangent linear variable, :

ADM

A Simple ExampleA Simple Example

• Let’s consider a simple 1D horizontal diffusion codeLet’s consider a simple 1D horizontal diffusion code

• The hand-coding of the tangent linear, adjoint, and The hand-coding of the tangent linear, adjoint, and representer codes is illustrated in the following FLASH representer codes is illustrated in the following FLASH animationsanimations

(Need to install Swiff Point Player to insert and play flash movies in

PowerPoint, check http://www.globfx.com) Alternatively,

http://marine.rutgers.edu/po/adjoint/tlm.htmlhttp://marine.rutgers.edu/po/adjoint/adm.htmlhttp://marine.rutgers.edu/po/adjoint/rpm.html

f = c

f = x

f = c * x

f = xn

f = x * y

f = 1 / x

f = 1 / sqrt(x)

f = c / x

f = x / y

f = 1 / (x + y)

f = 1 / ((x + y) * (u + v))

f = (c + u) / (x + y)

f = max(x, y)

f = min(x, y)

f = max(x, c)

f = max(c, x)

f = min(x, c)

f = min(c, x)

f = abs(x)

f = sqrt(x)

f = sqrt(x2 + y

2)

f = log(x)

f = 1 / log(x)

f = exp(x)

f = sin(x)

f = cos(x)

f = tan(x)

f = sinh(x)

f = cosh(x)

f = asin(x)

f = acos(x)

f = atan(x)

f = atan2(x, y)

tl_f = 0

tl_f = tl_x

tl_f = c * tl_x

tl_f = n * xn-1

* tl_x

tl_f = tl_x * y + x * tl_y

tl_f = - f2 * tl_x

tl_f = - f3 * 0.5 * tl_x

tl_f = - c * tl_x / x2 = - f * tl_x * x

tl_f = (y * tl_x – x * tl_y) / y2

tl_f = - f2 * (tl_x + tl_y)

tl_f = - f2 * ((tl_x + tl_y) * (u + v) - (x + y) * (tl_u + tl_v))

tl_f = + tl_u / (x + y) - f2 * (tl_x + tl_y)

tl_f = (0.5 + sign(0.5, x - y)) * tl_x + (0.5 – sign(0.5, x - y)) * tl_y

tl_f = (0.5 + sign(0.5, y - x)) * tl_x + (0.5 – sign(0.5, y - x)) * tl_y

tl_f = (0.5 + sign(0.5, x - c)) * tl_x

tl_f = (0.5 - sign(0.5, c – x)) * tl_x

tl_f = (0.5 + sign(0.5, c - x)) * tl_x

tl_f = (0.5 - sign(0.5, x – c)) * tl_x

tl_f = sign(1.0, x) * tl_x

tl_f = 0.5 * tl_x / f

tl_f = (x * tl_x + y * tl_y) / f

tl_f = tl_x / x

tl_f = - f2 * (tl_x / x)

tl_f = exp(x) * tl_x

tl_f = cos(x) * tl_x

tl_f = - sin(x) * tl_x

tl_f = tl_x / (cos(x) * cos(x))

tl_f = cosh(x) * tl_x

tl_f = - sinh(x) * tl_x

tl_f = tl_x / sqrt(1.0 – x2)

tl_f = - tl_x / sqrt(1.0 – x2)

tl_f = tl_x / sqrt(1.0 + x2)

tl_f = (y * tl_x – x * tl_y) / (x2 + y

2)

Tangent Linear Model RecipesN

on

linear

Tan

gen

t

Consider the product of two time-dependent state variables x and y and let

x = xo + x’ and x’ = x - xo (1) y = yo + y’ and y’ = y - yo (2)

where xo and yo are the basic state variables and x’ and y’ are their perturbations. Then,

x * y = (xo + x’) * (yo + y’) = xo * yo + x’ * yo + xo * y’ + x’ * y’

Eliminating prime terms using (1) and (2) and neglecting high order terms involving prime variables yield

x * y = xo * yo + (x - xo) * yo + xo * (y - yo)

= xo * yo + x * yo - xo * yo + xo * y - xo * yo

= x * yo + xo * y - xo * yo

using the above previous nomenclature (tl_x = x, tl_y = y, x = xo, y = yo) we get

f = x * y tl_f = tl_x * y + x * tl_y – x * y

= tl_x * y + x * tl_y – f

Notice that to obtain the RPM from the TLM for the above expression, we just need to subtract x * y which is the same as f.

Representer Model Recipes

f = c

f = x

f = c * x

f = xn

f = x * y

f = x * y * u

f = 1 / x

f = 1 / sqrt(x)

f = c / x

f = x / y

f = (x * y) / u

f = 1 / (x * y)

f = 1 / (x + y)

f = 1 / ((x + y) * (u + v))

f = max(c, x)

f = sqrt(x)

f = sqrt(x2 + y

2)

f = exp(x)

f = exp(c * x)

f = sin(x)

tl_f = c

tl_f = tl_x SAME!

tl_f = c * tl_x SAME!

tl_f = n * xn-1

* tl_x – (n – 1) * f

tl_f = tl_x * y + x * tl_y – f

tl_f = tl_x * y * u + x * (tl_y * u + y * tl_u) – 2 * f

tl_f = - f2 * tl_x + 2 * f

tl_f = - f3 * 0.5 * tl_x + 1.5 * f = f * (1.5 - f

2 * 0.5 * tl_x)

tl_f = - f * tl_x / x + 2 * f = f * (2 - tl_x / x)

tl_f = (y * tl_x – x * tl_y) / y2 + f

tl_f = (u * (tl_x * y + x * tl_y) – x * y * tl_u) / u2 SAME!

tl_f = - f2 * (tl_x * y + x * tl_y) + 3 * f

tl_f = - f2 * (tl_x + tl_y) + 2 * f

tl_f = - f2 * (tl_x + tl_y) * (u + v) - f

2 * (x + y) * (tl_u + tl_v) + 2 * f

tl_f = (0.5 - sign(0.5, c – x)) * (tl_x – x) + (0.5 + sign(0.5, c - x)) * c

tl_f = 0.5 * tl_x / f + 0.5 * f = 0.5 * (f + tl_x / f)

tl_f = (x * tl_x + y * tl_y) / f SAME!

tl_f = tl_x * f + (1 – x) * f

tl_f = c * tl_x * f + (1 – c * x) * f

tl_f = tl_x * cos(x) + f – x cos(x)

Representer Model Recipes

Beware Beware

• Recursive expressionsRecursive expressions

• Vertical integrations (pressure gradient)Vertical integrations (pressure gradient)

• Tri-diagonal algorithms (implicit vertical mixing, vertical Tri-diagonal algorithms (implicit vertical mixing, vertical sinking, parabolic spline reconstruction)sinking, parabolic spline reconstruction)

• Rescaling of state variables (Rescaling of state variables (TT**HzHz))

• Use of adjoint private arraysUse of adjoint private arrays

• Zeroing out of global and private adjoint variables after Zeroing out of global and private adjoint variables after being used (being used (TLMTLM to to ADMADM))

• Redundant operations are Redundant operations are wrongwrong in adjoint codes in adjoint codes

• Model initialization (Model initialization (ini_fields:ini_fields: compute compute u u andand v v))

DODO i=IstrU, Iendi=IstrU, Iend DC1(i,0)DC1(i,0) = = DC(i,0)DC(i,0) ! intermediate! intermediate cff1 = 1.0_r8 / ( cff1 = 1.0_r8 / ( CF(i,0)CF(i,0) * on_u(i,j) ) * on_u(i,j) ) DC(i,0)DC(i,0) = ( = ( DC(i,0)DC(i,0) * on_u(i,j) - * on_u(i,j) - DU_avg1(i,j)DU_avg1(i,j) ) * cff1 ) * cff1 ! recursive! recursiveEND DOEND DO

ParallelizationParallelization

• The The NLMNLM, , TLMTLM, and , and RPM RPM can be run in either shared-can be run in either shared-

memory (OpenMP) or distributed-memory (MPI)memory (OpenMP) or distributed-memory (MPI)

• The The ADMADM can only be run in distributed-memory ( can only be run in distributed-memory (ADMADM

violates shared-memory collision rules)violates shared-memory collision rules)

• Aggregation of variables for MPI communicationsAggregation of variables for MPI communications

CALLCALL ad_mp_exchange2d (ng, ad_mp_exchange2d (ng, iADMiADM, 3, Istr, Iend, Jstr, Jend, , 3, Istr, Iend, Jstr, Jend, &&

&& LBi, UBi, LBj, UBj, LBi, UBi, LBj, UBj, &&

&& NghostPoints, EWperiodic, NSperiodic, NghostPoints, EWperiodic, NSperiodic, &&

&& ad_Zt_avg1ad_Zt_avg1, , ad_DU_avg1ad_DU_avg1, , ad_DV_avg1ad_DV_avg1))

East-West MPI Communications

With RespectTo Tile R

Nonlinear

Adjoint With RespectTo Tile R

ad_V

ad_V

ad_V

ad_V

ad_V

L

L

L

L

R

R

R

R

i

i

i

i

i

i

i

i1 1 11

=

=+ + + +

ad_V

ad_V

+

+

;

;

0

0

=

=

ad_V

ad_V

ad_V

ad_V

ad_V

ad_V

R

R

R

R

L

L

L

L

i

i

i

i

i

i

i

i1 1 11

2 2 2 2=

=

-

-

-

-

-

-

-

-

ad_V

ad_V

+

+

;

;

0

0

=

=

i-2 i-1 i i+1

Istr Iend

Jstr

Jend

i-2 i-1 i i+1

Istr Iend

Jstr

Jend

ad_receive

ArecvE AsendW

ad_send

AsendE ArecvWTILE L TILE R

-

--

-V

VR

R L

L

i i

i i1 1

=

V

V2 2

=

V

V V

VRLii

+1

=

=L Ri +1i

i-2 i-1 i i+1

Istr Iend

Jstr

Jend

i-2 i-1 i i+1

Istr Iend

Jstr

Jend

send

ArecvE AsendW

receive

AsendE ArecvWTILE L TILE R

ad_V

ad_V

ad_V

ad_V

ad_V

T

T

T

T

B

B

B

B

j

j

j

j

j

j

j

j1 1 11

=

=- - - -

ad_V

ad_V

+

+

;

;

0

0

=

=

ad_V

ad_V

ad_V

ad_V

ad_V

ad_V

B

B

B

B

T

T

T

T

j

j

j

j

j

j

j

j1 1 11

2 2 2 2=

=

+

+

+

+

+

+

+

+

ad_V

ad_V

+

+

;

;

0

0

=

=

j+2

j+1

j

j-1Jend

Jstr

Istr Iend

j+2

j+1

j

j-1

Jend

Jstr

Istr Iend

ad_receive

ArecvS

AsendN

ad_send

AsendS

ArecvN

TILE B

TILE T

V

V V

V BTjj

-1

=

=T Bj -1j

+

++

+V

VB

B T

T

j j

j j1 1

=

V

V2 2

=

j+2

j+1

j

j-1Jend

Jstr

Istr Iend

j+2

j+1

j

j-1

Jend

Jstr

Istr Iend

send

ArecvS

AsendN

receive

AsendS

ArecvN

TILE B

TILE T

North-South MPI CommunicationsN

onlin

ear

Adjo

int

With Respect to Tile B

Profiling

Elapsed CPU time (seconds): Resolution: 0256x0128x030, Parallel Nodes: 4, Tiling: 002x002Elapsed CPU time (seconds): Resolution: 0256x0128x030, Parallel Nodes: 4, Tiling: 002x002

Node 0 Node 1 Node 2 Node 3 TotalNode 0 Node 1 Node 2 Node 3 Total

CPU: 27.910 27.913 27.917 27.916 CPU: 27.910 27.913 27.917 27.916 111.656111.656 Model Elapsed Time Profile:Model Elapsed Time Profile:

Initialization ................................... 2.092 ( 1.8737 %) 2.086 ( 1.8680 %)Initialization ................................... 2.092 ( 1.8737 %) 2.086 ( 1.8680 %) Reading of input data ............................ 1.942 ( 1.7389 %) 1.943 ( 1.7398 %)Reading of input data ............................ 1.942 ( 1.7389 %) 1.943 ( 1.7398 %) Processing of input data ......................... 0.264 ( 0.2362 %) 0.264 ( 0.2360 %)Processing of input data ......................... 0.264 ( 0.2362 %) 0.264 ( 0.2360 %) Computation of vertical boundary conditions ...... 0.007 ( 0.0058 %) 0.008 ( 0.0072 %)Computation of vertical boundary conditions ...... 0.007 ( 0.0058 %) 0.008 ( 0.0072 %) Computation of global information integrals ...... 0.024 ( 0.0212 %) 0.032 ( 0.0288 %)Computation of global information integrals ...... 0.024 ( 0.0212 %) 0.032 ( 0.0288 %) Writing of output data ........................... 1.093 ( 0.9786 %) 1.106 ( 0.9908 %)Writing of output data ........................... 1.093 ( 0.9786 %) 1.106 ( 0.9908 %) Model 2D kernelModel 2D kernel .................................................................... 4.393 ( 3.9345 %) 5.927 ( 5.3084 %)4.393 ( 3.9345 %) 5.927 ( 5.3084 %) 2D/3D coupling, vertical metrics ................. 0.067 ( 0.0602 %) 0.131 ( 0.1171 %)2D/3D coupling, vertical metrics ................. 0.067 ( 0.0602 %) 0.131 ( 0.1171 %) Omega vertical velocity .......................... 0.132 ( 0.1181 %) 0.160 ( 0.1436 %)Omega vertical velocity .......................... 0.132 ( 0.1181 %) 0.160 ( 0.1436 %) Equation of state for seawater ................... 0.199 ( 0.1781 %) 0.320 ( 0.2863 %)Equation of state for seawater ................... 0.199 ( 0.1781 %) 0.320 ( 0.2863 %) 3D equations right-side terms .................... 0.382 ( 0.3422 %) 0.548 ( 0.4905 %)3D equations right-side terms .................... 0.382 ( 0.3422 %) 0.548 ( 0.4905 %) 3D equations predictor step ...................... 0.759 ( 0.6797 %) 1.104 ( 0.9888 %)3D equations predictor step ...................... 0.759 ( 0.6797 %) 1.104 ( 0.9888 %) Pressure gradient ................................ 0.414 ( 0.3704 %) 0.627 ( 0.5619 %)Pressure gradient ................................ 0.414 ( 0.3704 %) 0.627 ( 0.5619 %) Harmonic stress tensor, S-surfaces ............... 0.159 ( 0.1422 %) 0.244 ( 0.2181 %)Harmonic stress tensor, S-surfaces ............... 0.159 ( 0.1422 %) 0.244 ( 0.2181 %) Corrector time-step for 3D momentum .............. 0.487 ( 0.4362 %) 0.569 ( 0.5097 %)Corrector time-step for 3D momentum .............. 0.487 ( 0.4362 %) 0.569 ( 0.5097 %) Corrector time-step for tracers .................. 0.495 ( 0.4431 %)Corrector time-step for tracers .................. 0.495 ( 0.4431 %) 0.652 ( 0.5842 %)0.652 ( 0.5842 %) Total: Total: 12.906 11.5589 15.791 14.142812.906 11.5589 15.791 14.1428 Message Passage profile:Message Passage profile:

Message Passage: 2D halo exchanges ............... 0.790 ( 0.7072 %) 1.115 ( 0.9990 %)Message Passage: 2D halo exchanges ............... 0.790 ( 0.7072 %) 1.115 ( 0.9990 %) Message Passage: 3D halo exchanges ............... 0.272 ( 0.2437 %) 0.286 ( 0.2559 %)Message Passage: 3D halo exchanges ............... 0.272 ( 0.2437 %) 0.286 ( 0.2559 %) Message Passage: 4D halo exchanges ............... 0.028 ( 0.0254 %) 0.052 ( 0.0462 %)Message Passage: 4D halo exchanges ............... 0.028 ( 0.0254 %) 0.052 ( 0.0462 %) Message Passage: data broadcast .................. 0.024 ( 0.0217 %) 0.024 ( 0.0213 %)Message Passage: data broadcast .................. 0.024 ( 0.0217 %) 0.024 ( 0.0213 %) Message Passage: data reduction .................. 0.002 ( 0.0020 %) 0.010 ( 0.0092 %)Message Passage: data reduction .................. 0.002 ( 0.0020 %) 0.010 ( 0.0092 %) Message Passage: data gathering .................. 0.371 ( 0.3321 %) 0.367 ( 0.3283 %)Message Passage: data gathering .................. 0.371 ( 0.3321 %) 0.367 ( 0.3283 %) Message Passage: data scattering.................. 2.612 ( 2.3392 %) 2.603 ( 2.3316 %)Message Passage: data scattering.................. 2.612 ( 2.3392 %) 2.603 ( 2.3316 %) Total: Total: 4.099 3.6713 4.457 3.99154.099 3.6713 4.457 3.9915

TLM ADM

DirectoriesDirectories and and FilesFiles CommentsComments

AdjointAdjoint Adjoint model (Adjoint model (ADMADM))

BinBin Executable scripts: Executable scripts: cpp_cleancpp_clean, , smakedependsmakedepend

CompilersCompilers makefilemakefile rules and dependency files rules and dependency files

DriversDrivers Master program and applications driversMaster program and applications drivers

ExternalExternal Standard input scripts files and IO variables definitionsStandard input scripts files and IO variables definitions

IncludeInclude CPP definitions and other configuration include filesCPP definitions and other configuration include files

LibLib Needed external libraries: Needed external libraries: MCTMCT, , PARPACKPARPACK

ModulesModules Declaration modules and nested grid pointer structuresDeclaration modules and nested grid pointer structures

NonlinearNonlinear Nonlinear model (Nonlinear model (NLMNLM))

ObsoleteObsolete Obsolete filesObsolete files

ProgramsPrograms Support programsSupport programs

RepresenterRepresenter Tangent linear representer model (Tangent linear representer model (RPMRPM))

SeaIceSeaIce Sea Ice modelSea Ice model

TangentTangent Tangent linear perturbation model (Tangent linear perturbation model (TLMTLM))

UtilityUtility Generic support routinesGeneric support routines

VersionVersion Version information fileVersion information file

makefilemakefile A single compilation A single compilation makefile makefile forfor GNU make GNU make

MakefileMakefile

CPP Options Adjoint itCPP OptionCPP Option TLMTLM RPMRPM ADMADM

UV_ADVUV_ADV XX XX XX

UV_CORUV_COR XX XX XX

UV_LDRAGUV_LDRAG XX XX XX

UV_QDRAGUV_QDRAG XX XX XX

UV_VIS2 – MID_S_UVUV_VIS2 – MID_S_UV XX XX XX

UV_VIS2 – MID_GEO_UVUV_VIS2 – MID_GEO_UV XX XX XX

UV_VIS4 – MID_S_UVUV_VIS4 – MID_S_UV XX XX XX

UV_VIS4 – MID_GEO_UVUV_VIS4 – MID_GEO_UV XX XX XX

UV_U3ADVECTIONUV_U3ADVECTION XX XX XX

UV_C2ADVECTIONUV_C2ADVECTION XX XX XX

UV_C4ADVECTIONUV_C4ADVECTION XX XX XX

UV_SADVECTIONUV_SADVECTION XX XX XX

BULK_FLUXBULK_FLUX XX XX XX

NPZDNPZD XX XX XX

CPP OptionCPP Option TLMTLM RPMRPM ADMADM

TS_U3HADVECTIONTS_U3HADVECTION XX XX XX

TS_A4HADVECTIONTS_A4HADVECTION XX XX XX

TS_C2HADVECTIONTS_C2HADVECTION XX XX XX

TS_C4HADVECTIONTS_C4HADVECTION XX XX XX

TS_A4VADVECTIONTS_A4VADVECTION XX XX XX

TS_C2VADVECTIONTS_C2VADVECTION XX XX XX

TS_C4VADVECTIONTS_C4VADVECTION XX XX XX

TS_SVADVECTIONTS_SVADVECTION XX XX XX

TS_DIF2 – MID_S_TSTS_DIF2 – MID_S_TS XX XX XX

TS_DIF2 – MID_GEO_TSTS_DIF2 – MID_GEO_TS XX XX XX

TS_DIF2 – MID_ISO_TSTS_DIF2 – MID_ISO_TS XX XX XX

TS_DIF4 – MID_S_TSTS_DIF4 – MID_S_TS XX XX XX

TS_DIF4 – MID_GEO_TSTS_DIF4 – MID_GEO_TS XX XX XX

TS_DIF4 – MID_ISO_TSTS_DIF4 – MID_ISO_TS XX XX XX

CPP OptionCPP Option TLMTLM RPMRPM ADMADM

NONLIN_EOSNONLIN_EOS XX XX XX

CURVILINEARCURVILINEAR XX XX XX

POWER_LAWPOWER_LAW XX XX XX

SALINITYSALINITY XX XX XX

SOLVE3DSOLVE3D XX XX XX

SPLINESSPLINES XX XX XX

MASKINGMASKING XX XX XX

RHO_SURFRHO_SURF XX XX XX

VAR_RHO_2DVAR_RHO_2D XX XX XX

WJ_GRADPWJ_GRADP XX XX XX

DJ_GRADPSDJ_GRADPS XX XX XX

TCLM_NUDGINGTCLM_NUDGING XX XX XX

SOLAR_SOURCESOLAR_SOURCE XX XX XX

CPP OptionCPP Option TLMTLM RPMRPM ADMADM

EASTER_WALLEASTER_WALL XX XX XX

WESTERN_WALLWESTERN_WALL XX XX XX

NORTHERN_WALLNORTHERN_WALL XX XX XX

SOUTHERN_WALLSOUTHERN_WALL XX XX XX

****_FSCHAPMAN****_FSCHAPMAN XX XX XX

****_FSGRADIENT****_FSGRADIENT XX XX XX

****_FSCLAMPED****_FSCLAMPED XX XX XX

****_M2FLATHER****_M2FLATHER XX XX XX

****_M2GRADIENT****_M2GRADIENT XX XX XX

****_M3GRADIENT****_M3GRADIENT XX XX XX

****_M3CLAMPED****_M3CLAMPED XX XX XX

****_TGRADIENT****_TGRADIENT XX XX XX

****_TCLAMPED****_TCLAMPED XX XX XX

Validation Tests

• ROMS/TOMS is continuously evolving so we need a ROMS/TOMS is continuously evolving so we need a

process to check the validity of all its algorithmsprocess to check the validity of all its algorithms

• Currently, we have four drivers in Currently, we have four drivers in ocean_control.F ocean_control.F to to

test the correctness of the test the correctness of the TLMTLM, , RPMRPM, and , and ADMADM::

tlcheck_ocean.htlcheck_ocean.h TLM_CHECKTLM_CHECK

picard_ocean.hpicard_ocean.h PICARD_TESTPICARD_TEST

grad_ocean.hgrad_ocean.h GRADIENT_CHECKGRADIENT_CHECK

pert_ocean.hpert_ocean.h INNER_PRODUCTINNER_PRODUCT or or SANITY_CHECKSANITY_CHECK

Sanity Check

• This is the most stringent test: This is the most stringent test: it never fails!it never fails!

• It is highly recommended to run this test on all It is highly recommended to run this test on all

applications and CPP configurationsapplications and CPP configurations

• It checks the symmetry between It checks the symmetry between TLMTLM and and ADMADM state state

vectors:vectors:

• If this difference is large, it tell you that either the If this difference is large, it tell you that either the TLMTLM

or or ADMADM is incorrect is incorrect

A – (A )A – (A )TT

= 0 = 0 within round offwithin round off

How To Run Sanity Check

• Simply activate SANITY_CHECK CPP option in your Simply activate SANITY_CHECK CPP option in your

applicationapplication

• Initialize from rest and force with desired nonlinear Initialize from rest and force with desired nonlinear

background statebackground state

• Specify interior point to perturb with a delta function in Specify interior point to perturb with a delta function in

ocean.in generic user parameters:ocean.in generic user parameters:INT(user(1))INT(user(1)) TLMTLM state variable to perturb ( state variable to perturb (zetazeta, , ubarubar, , vbarvbar, , uu, , vv, , tt, …), …)

INT(user(2))INT(user(2)) ADMADM state variable to perturb ( state variable to perturb ( 1 1 , , 22 , , 3 3 , , 44, , 55, , 66, …), …)

INT(user(3))INT(user(3)) I-index ofI-index of TLMTLM variable to perturbvariable to perturb

INT(user(4))INT(user(4)) I-index ofI-index of ADMADM variable to perturbvariable to perturb

INT(user(5))INT(user(5)) J-index ofJ-index of TLMTLM variable to perturbvariable to perturb

INT(user(6))INT(user(6)) J-index ofJ-index of ADMADM variable to perturbvariable to perturb

INT(user(7))INT(user(7)) K-index ofK-index of TLMTLM variable to perturbvariable to perturb

INT(user(8))INT(user(8)) K-index ofK-index of ADMADM variable to perturbvariable to perturb

Sanity Check Examples

Sanity Check - Tangent: Sanity Check - Tangent: tl_zetatl_zeta perturbed at (i,j) = 50 50 perturbed at (i,j) = 50 50Sanity Check - Adjoint: Sanity Check - Adjoint: ad_zetaad_zeta perturbed at (i,j) = 50 50 perturbed at (i,j) = 50 50

Sanity Check - Perturbing variable: zetaSanity Check - Perturbing variable: zetaSanity Check - Tangent: 1.624754814645E-05 at (i,j) 50 50Sanity Check - Tangent: 1.624754814645E-05 at (i,j) 50 50Sanity Check - Adjoint: 1.624754814645E-05 at (i,j) 50 50Sanity Check - Adjoint: 1.624754814645E-05 at (i,j) 50 50Sanity Check - Difference: Sanity Check - Difference: 9.893344823930E-199.893344823930E-19 at (i,j) 50 50 at (i,j) 50 50

Sanity Check - Tangent: Sanity Check - Tangent: tl_u tl_u perturbed at (i,j,k) = 110 40 30perturbed at (i,j,k) = 110 40 30Sanity Check - Adjoint: Sanity Check - Adjoint: ad_uad_u perturbed at (i,j,k) = 110 40 30 perturbed at (i,j,k) = 110 40 30

Sanity Check - Perturbing variable: uSanity Check - Perturbing variable: uSanity Check - Tangent: 1.105644477697E-02 at (i,j,k) 110 40 30Sanity Check - Tangent: 1.105644477697E-02 at (i,j,k) 110 40 30Sanity Check - Adjoint: 1.105644477697E-02 at (i,j,k) 110 40 30Sanity Check - Adjoint: 1.105644477697E-02 at (i,j,k) 110 40 30Sanity Check - Difference: Sanity Check - Difference: -3.469446951954E-18-3.469446951954E-18 at (i,j,k) 110 40 30 at (i,j,k) 110 40 30

Final RemarksFinal Remarks

• Maintenance of Maintenance of TLMTLM, , RPMRPM, and , and ADMADM algorithmsalgorithms

• Automatic differentiation Automatic differentiation

• Linearization of physicsLinearization of physics

• Non-differentiable algorithms (vertical Non-differentiable algorithms (vertical mixing parameterizations)mixing parameterizations)

• Training and documentationTraining and documentation