Max -Planck -Institu t fUr Meteorologie

REPORT No. 88

E ·c c.

0.5

;§ ~o.or.'

-0.5 '

-1 .0 0 10 20 30 40 ·50 60 70 80 90 100 1

OPTIMAL FINGERPRINTS FOR THE DETECTION OF TIME DEPENDENT CLIMATE CHANGE

by

KLAUS HASSEL~ANN

HAt.4BURG, AUGUST 1992

Max-Planck-Institut fUr Meteorologie

REPORT No. 88

-0,5 ' i

-1,00 10 20 30 40 '50 60 70 80 90 100 I

OPTIMAL FINGERPRINTS FOR THE DETECTION OF TIME DEPENDENT CLIMATE CHANGE

by

KLAUS HASSElMANN

HAMBURG. AUGUST 1992

AUTHOR:

Kious Hasselmann

MAX-PLANCK-INSTITUT FOR METEOROLOGIE

BUNDESSTRASSE 55 D-2000 HAMBURG 13 F.R. GERMANY

Max -Planck-I nstitut fUr Meteorologie

Tel.: Telex:

. Telemall: Telefox:

+49 (40) 4 11 73-0 211092 mpime d

MPI.METEOROLOGY +49 (40) 4 11 73-298 REPb88

OPTIMAL FINGERPRINTS FOR THE DETEcrrON OF

TIME DEPENDENT CLIMATE CHANGE

Abstract

K.Hasseimann

lO.Aug. 1992

An optimal linear filter (fingerprint) is derived for the deteclion of a

given lime-dependent. multi-variate climatc-change signal in

of natural climmc variability noise. Applic.niol1 of the

the observed (or model-simulated) climate data yields a

the presence

fingcrprint to

climate-change

detection variable (detector) with maximal signal-to-noise ratio. The

optimal fingerprint is given by the product of the assumed signal pattem

and the inverse of the climate variability covariance matrix. The data can

consist of any, not necessarily dynamically eompletc climate data set for

which estimates of thc natural variability covariance matrix exist. The

single-pattern

climate-change

anal ysis readil y

signal lying

lOW-dimensional)

applied separately

pattern space.

signal-pattern

to each

Multi-pattern

generalizes to tile multi-pattcl11

in a prescribed (in practice

space: thc single-pattern result

case of a

rc!atively

is simply

individual base pattern spalming the signal

detection methods can bc applied either to

test the statistical significance of individual compollcnts of a predicted

muiti-componem

detection tests,

climate

or to

change response,

detennine the

using

statistical

separate single-pattern

significance of the

complcte signal, using a multi-variate test. Both detection modes make use

of the same set of detectors. The difference in direction of the assumed

signal pattem and computed optimal fingcrprint vector allows alternative

interpretations of the estimated

detectors. The present analysis

signal

yields

associated with the sel of optimal

an estimated signal lying in the

assumed signal space, whereas an

detection problem by Hasselmann

earlier analysis of the

yielded an estimated

intelpretations can

time-independent

signal in tbe

be explained computed fmgelprint

by different choices

space. The different

of the metric used to relate the signal space to the

fingerprint

metric,

space (inverse covariance matrix versus standard Euclidean

respectively). Two simple nmural

considered: a space-time separability model, and

POPs (pJincipal Oscillation Pattcl11s). For each

valiabilily models arc

an expansion in temlS of

model the application of

the optimal fingerprint method is illustrated by an example.

ISSN 0937-1060

I, Introduction

The general public coneem and ongoing scientific debatc on the

anticipated global warming due to increasing greenhouse-gas concentrations

and on the impact of other activities of man on lIle earth's climate has

gencrated a strong demand for the devclopment of improved tcchniques for

the early

idcntification

detection of thc predicted elimme change signal. A clear

of the anthropogenic signal in climate obselvations would

reduce the present scientilic ullcertainties regarding the magnitude and

foml of the anticipated climate change and would provide a more reliable

quantitative basis for the dcvelopmCI1l of rational political abatement and

adaptatioll strategies,

At the core of the detection problem is the development of a suitable

strategy for distinguishing between the anticipated externally

time dependent climate change signal and the natural intemal

of the climate system, The problem can be divided into three

identification of lhe climate change signal lhill one wishes

generated

variability

parts: (I)

to detect,

(ii) detelmination of the relevant statistical properties of the natural

climate variability background, and (iii) developmclll of an optimal

detection method, This paper addresses the third problem, However, the

question of tile representation (but not the estimation) of the second

moment statistics of the natural climate variability noise needed for

optimaal detection, which relates to tile second problem, will also be

considered briefly,

It will be assumed throughout that the first problem has been rcsolved and

the general structure of the space and time dependent climate cbange

signal that onc is sceking to detect has been detelmincd, for cxample from

model simulations, It is well known that ill attempting to detcct signals

in noisy Illulti-variate d<lW, the number of degrees of freedom of the

signal must be scvcrcly curtailed in order to cxtract statistically

Significant results, Spccific<llly, it will be assumed thai the signal is

defined to within thc unknown coefficients or a relatively Sill all set of

prescribed time-varying response pallCIl1S eeL Cubaseh et aL, 1992),

2

This should not

anticipated climate

gloss over the di ff1cultics, however,

chnnge signal. This is generally a

of defining the

110n-tIivial problem

reqUlnng an inlercompmisoll and detailed assessment of different climate

change simulations with differcnt models. The problem is compounded by the

fact that realistic time dependent climate change simulations can be

carried out only with coupled ocean-atmosphere general circulution models

(CGCMs), which generate Ulcir own natural climate variability (Washington

and Meehl, 1989, Stouffer et aI., ] 989, Mnnabc Cl aI., 1991. Cubasch et

aL, 1982, Santer et aI., I 992a; see also tile stochastically forced ocean

cxperiment of Mikolajewicz and Maier-Reimer, 1990). The signal detection

problem arises therefore already in the attempt to define the climate

response signal in model simulations, enDing into question the basic

premise of the separability of the signal definition and signal detection

problems. This will nevertheless he ussulllcd in the following as a conceptlJaI starting point.

In the same Spilit, thc estimation of

natural climate variability required for

the core of the second problem, will also

again a non-tliviul ussumpllon, but is

well-defined detection problem. In practice,

whether an anthropogenic climate signal

the statistical properties of the

optimal detection, which is at

he regarded as resolved. This is

revolves around uncertainties over the

natural variability of the real climate.

necessary in

much of the

cun al read y

stIUcturc and

Nevertheless, in

a consistent conceptual framework it will be assumed,

point, that estimates of tile rcquircd natural

statistics are available.

order to pose a

ongoing debate on

bc detected today

magnitude of the

order to develop

again as a starting

climate valiability

This is not such a severe restriction, however, as may at first sight

appear. The detection strategy will be developed in

'observed climate stnte vcctor', which refers \0

the following for an

any set of climate

variables or

The observed

for example,

consist of

indices for which adequate statistical data are available.

climate state need not be dyn<lm ically complete. as required,

for a climate model. Thus the 'observed climate state' can

time series of vnrying Icngth and for diffcrcnt climate

valiables, measured at inilomogeneollsl y distributed stations. Timc

3

dependent signal detection is concemed only with the kinematics. not the

dynamics of climate change. and a dynamically incomplete represenlntion of

Ule climate stale has no consequences for the method of detection (apart.

of course. from the unavoidable loss of detection power associated with a

loss of infonnalion).

After an optimal detection stmtegy lws becn developed under these two

working assumptions. the general

thrce elements of tile ovcrall

question

detection

of the

problem

possible

interdependence of

can be revisited

the

in

approaches to modeling fimhcr iterations. The discussion of some

the relevant second moments' properties

presentcd at the end of this paper

direction.

of the natural climate vaJiahility

The detcction

viewed as the

problem. in the

task of identifying

represents a first step in this

separable fon11 discussed here, is often

tile most sensitive climate index. from a

large set of potentially available indices. for which the anticipated

anthropogenic climate signal can he most readily distinguished [rom the

nal1n'al climate noise. Global or regional mean surface temperature.

veltical temperature differences, sea icc extent. sea level change and

integrated deep ocean temperatures are examples 01 indices which have been

discussed in this context (e.g. Wigley and Jones. 1981. Barnett. 1986,

Barnett and Schlesinger. 1987. Karoly. 1987.1989. Munk and Forbes. 1989.

Mikolajcwicz et aI., 1991). A more systematic approach. however, is the

tlngerprint method. Here all climate variables are regarded as containing

potentially useful information on climate change, and the task is to

extract from the filII set of availahle observed climate variables an

optimal

1980,

net climate change detection index (cf. Madden and Ramanathan,

MacCracken and Moses. 1982), This approach will he pursued in this

paper. While it is theoretically conceivable that a single climate

variable will turn out to be the most effective detector. in the general

case all variables will calTY some signal information. although wi th

varying levels of noise contamination. and the optimal detector. (lefined

as the variable which has a maximal signaHo-noise mIlo. wjlJ consist of

a weighted linear combination (the . fingcrpIinl ') of all variables.

4

Most investigations in the past have considered only partial aspects of

the full multi-vtlliate, space-time dependent problem, The detection of a

time independent equilibrium change of a multi-variate climate system in

response to a constant extelllal forcing has been investigated by

Hasselmann (1979, referred to in the following as H), Hannosch6ck and

Frankignoul (1985), Hense (1986), Hense ct al (1990), Bell (1982, 1986)

and others, The com plem elltary problem of detecting a time-dependent

climate

nature

change signal, while disregarding the spatial and multi-variable

(1992),

of (he climate signal,

These authors did

optimization of the

detection problem

(1988), Wigley and

detection

has been studied by Bloomfield and Nychka

not address, however, the question of the

variable, Various other facets of the signal

have been investigated or reviewed by PreisendOl'fcr

Bamell (1990), Bamett (1991), Barnell el aL (1991),

Solow (1991) and other authors, Recently, however, North et al (prelimary

pre print) have independently investigated the full space-timc depcndcnt

climate change detcction problem, also using pattern analysis tcchniques

rather similar to the approach pursued here,

In com paring

studies appl)'

the theoretical signal predictions with

some form of filtering or pallem

which in effect projects the observed data onto

pattern, This corresponds essentially to the usual

a signal pattern to dllta such that the variance

minimized, While this removes mueh of the irrelevant

observed data, many

correlation technique

the predictcd signal

least-square filling of

of the residual is

noise

does

should

not, however, represent the optimal signal detection

in the data, it

solution, This

maximize the signal-to-noise ratio rather than the explained

variance,

In this paper an optimal space-time dcpendent filter (fingerprint) is

derived which maximizcs the signal-to-noise ratio for the associated

detector for any prescribed multi-variable, spacc-time dependent climate

signal. TIle singlc-pattern solution is then generali7,ed to tile

mulli-pattel11 case to determine it set of p optimal fingerprillls and

associated detectors for any climate-change signal lying in a prescribed

space spanned by p given climatc-changc pattellls, To detenninc the

statistical signiflc,mce of the estimated climate signal it must be

5

assumed that the natural climale vm'iilblility is Gaussian (or is otherwise

known). However, the dct1nilioll or the optimal )'ngerprillls as such is not

dependent on the Gaussian hypothesis.

The detection problem is dolined in Sc~tion 2. Assuming the space-time

depcndelll stlUcturc of the externally generated climate signal is

prcsclibed to within an unknown amplitude factor, tile optimal space-time

integrated detector is dctived in Section 3 as a linear combination (the

optimal 'fingerprint') of the complete set of time dependent climate

vaIiables. The result is generalized in Seclion 4 10 the case of a signal

defined only as an unknown linear combination of a finltc set of

presClibed time dependent pallems. The analysis follows the basic ideas

of H, extended in a straightforward manner to include the time dimension,

but is simplified by the introduction of lile 'fingerprint' tenninology.

Tbe problem of modeling the complex space-time (or space-frequency)

dependent second moments of the naltmli climate valiability required for

optimal detection is discussed in Section 5. Two simplifications arc considered and illustmtcd by examples: the assumption of space-time

separability, and 1I1C il1lroductioll of upproximate POP (Principal

Oscillation Pallem) representations. The results are summarized and some

open questions mentioned in the concluding section (J.

Consider the evolution of ,m 'observed climate state' = in

response to some lime dependent external forcing over a time interval 0 $

t ~ T. The 'observed climate state' can be represented as u discretized

composite vector 2 = (<ili) whose indices i = (v,z) lUll through the climate

variables v (temperature, pressure, moisture, ... ) and the disrete spatial

coordinates Z' which can refer, for example, to a set of observing

stations or a model grid. The climate trajectory 2(1) can represent either

a set of vadables of the real climate system or

some numerical model experiment. As has been

tile simulated response in

pointed out, it is not

necessary that 2(t) provides a dynamically complete description of the

climate system. The 'observed state vector' can (and, indeed, must) be

limited to variables for which sufficient Observational information is

available to adequately define the statistics of the ensemble of

6

trajectories ~(t) characterizing the natural variability of the system,

The detection problem is then to decide whether the climate time history

9!(t) generated by the external forcing can he distinguished, at some given

level of significance, from the statistical ensemble of

natural-variability trajectories ~(t),

For the lime-independent problem, an optimal signal detection strategy has

been presented in H, In the following, the analysis of H is extended to

the gcneral multi-variate, time dependent t:<lse and recast in a simpler

'fingclprint' telminology,

Fonnally, the approach of H can Ile

dependent case by simply discretizing

Ule time index in a combined index

immediately generalized

the time variable and a ;;::: (i,t) nmning

then

from

to the timc

incorporating

1 to n, The

extended climate vector

summarizing tile complete

discretizcd time interval

is

a

set to unity), The

= (v,lS,t) not only

represents a constant vector

climate trajectory:

1,2, ... 1'

IJI a set of all <l>i(t) in the

(the time discretization interval

imroduction of a compact

simplifies the notation, but

variabk-space-time-index

also focusses on the

essentially very simple linear-algebra geometry of the detection problem.

An important requirement 101' a successful detection strategy is the

reduction of the number of degrees of freedom of the signal. This is

achieved in H by considering a signal which is defined II priori only in a

relatively low-dimensional sub-space of the full climate system. Attempts

to test whether the full climate response vector )I! can be distinguished

from an element )I! of the natural variability ensemble

in dimensionality of tilC signal will generally fail

reasons (cf.H, Bamctt and Hassclmann, 1979):

without a reduction

1'01' the following

The climate response te) given cxtcmal forcing can be represented

generally (ignoring nonlinear illlcraetions between the natural climate

variablility and the eXlcl'Ilally forced response) as a superposition

of the forced delenninistic climate signal )l!s and a particular realization

7

of the

denoting the

natural

direction

signal is statistically

variablility ensemble. Let v be

of the signal, ~s~ I ~ll y. significant in the sense that

with the noise component in the v I ~s I is large compared

a unit· length vector

Assume now that the

the signal ampliLUde

direction,

where the cornered parentheses denote

is used, a superscript T denoting the

noise is taken to be zero, <\jI,? priori, it is then possible to test

T 2 the !let square response (y~) in the

the inequality (2) would be positive.

(2)

cnscmble meaDS :md matrix notation

transpose. The mean of the eliuHlte

D. If the direction v is known a

for the statistical signilieancc of

v direction. and the outcome under

If, on the other hand, the signal paltem is unknown, one can lest only

the statistical significance of the complete n·dimensional response vector

~. This requires considering the magnitude and orientation of the response

vector in relation to the joint probability density of the ensemble of

vectors ~ in the full n dimensional climate trajectOlY space. If the

probability density decreases monotonically with distance from the origin,

as is nonnally the case. each (n-I )-dimensional hypersurface of constallt

probability density will divide the n-dimensional space into an intcmal

closed region of some probability measure P around the origin and an

extemal open region of probahility measure (l-P), in Which the

probability density is everywhere smaller than in the intemal region. The

response vector ~ is thcn normally termcd statistically significant at the

significance levcl P if the end.point of vector, drawn from the origin,

lies in the external region. Without entering here into the details of the

analysis, it is qualitatively apparent that the larger the number of

irrelevant noise dimensions, the smaller the relative contribution of the

signal to the total magnitude of the responsc vcctor, and the more

difficult it will be to detect the signal in the full n-dimensional space

- evcn when the signal component is significantly larger than the noise

component in tl1e one (unknown) direction v.

Fortunately, this second 'needle in a haystack' situation

not apply in practice: the direction of the hypothesized

8

will normally

elimnte Signal

can be assumed to be known from model simulations, or at least to lie

within a known sub-space of relatively small dimension_

The case that the climate change signal is known exactly applies for the

simplest yes-no question of climate change detection: one wislles to

detemline whether a specific time-dependent global climate signal which

has been predicted by a model can be detected in the data (or at what

future time it should become detectable in the data). TllC more general

case that the response signal is assumed only to lie within some

prescribed low-dimensional sub-space of the full climatc trajectory space

arises if the predicted climate change signnl is only imperfectly known

(for example. because diITercnt models have predicted different climate

change patterns) or if one wishcs to distinguish hetwccn different climate

Ci1atlgC signals produced by different anthropogenic or natural cxtcmal

forcing mechanisms.

A mulli-pattcm analysis is necessary also if one wishes to test the

statistical significance not only of tllC complete global climate change

signal, but also of particular sub-components of tile global signal. II can

be anticipated. for example. that the most effective single nct global

climate change detector will be based primaJily 011 the largeescale

features of the climate ficlds. But for policy-makers. the regional

climate changes (which at present arc not predicted very reliably by

climate models) will presumably be of grealer eoneem than globally

integrated quantities. They will therefore wish to know not only whelher

the global climate change predicted by models has been dClected, bUI also

whether thc model predictions of climate change on lhe regional scale can

be confirmed by observations.

The following analysis CHn be extcnded to

than detenninistically prescribed climate

lIppropriate, for example. if thc predicted

ensemble of different modcl simulalions

the case

signals.

signal

with

of statistically rather

This could be

is inferred from an

differcnt levels of

credibility. or from a mixture of model simulations and general

theoretical considerations (for cxamplc, regarding the expectation of a

land-sea-contrast signal). However. this Bayesian gcnerali/alian will not

be pursued fUJ1her in the prcsent paper_

9

If the pallern g of the space-lime signal

an unknown constant amplitude factDr c,

. s InueelDry ]jI is knDwn ID within

(3)

the signal deteclion problem reduces (if one limits oneself to linear

techniques) tD the tusk of tlcriving an optimal detection variable Dr

'detector'

d l' (! ]jI), (4)

computed from ]jI by applicatiDn of a linear ' filter function' or

'fingerprint' f co (I' ), for - il

which the sqUilrc signal-lo-noise ratio

(5)

is maximized. Here

s ([1' ]jIs) d . (6)

ilnd

0 (LT \ill (7)

represent, respeeti vel y, the signal and climate-variability noise

cDmponents of the net detector d ~ dS+ o.

The fingerprint vector r (5)) only to

defined only

will be

within a

to within

cstimated

is determined through the condition R2 = max (Eq.

factor. Similarly, the signal pattern g need be

an arbitrary factor, since the amplitude of the

by the detection procedure. Although .f and g signal

could therefore be nonnalizcd [0 unit vectors. it is notationally more

convenicnt to leave the vectors unnormalizcd at this point.

The maximization of R2 with respect to the arbitrarily normalized vector f is equivalent to the minimization of <02> under the side condition (ds)2 ~ COllSt. This yields as determining equation for the I1ngerprim !

10

where C z

variability

= C I~ = au and ), is

normalization chosen lor [.

·u' ( TC,-l )-1 WI I r, = - g "" g .

Tile optimal llllgcl'print

assumed signal direction,

(and is oftcn assumcd

o (8)

t:ovariance matrix of the natural climatc

L<lgrange multiplier

Thc solution is

r

direction is in

as may perhaps

in detcction studies).

whose

general

have

This

value depends on the

(9)

not parallel to the

been expected intuitively

is best understood by

transforming to statisticall y orthogonal coordinates (denoted in the

following by primes),

\jIa = I \jib Cba {I 0)

h

which arc defined with respect to an orthontllmaJ basc cha = ~a consisting

of the eigenvectors (empirical orthogonal functions, EOFs) £a of C;;;,

with

(12)

In EOF coordinates, the covariance matrix of the natural climate

variability lakes thc diagonal fnrl11

2 " = on vab (13)

2 where (Ja is the variance associillcd with the EOF £a' Equation (9) tilus

becomes

11

f' a (14)

The multiplication of lhe signal with the inverse of the covariance matrix

is seen to weight Ihe fingerprinl components f' in tile EOF frame relative a 2

to the signal componenls g:' by the inverse «( of the EOF variances, u a

thereby slewing the fingerprint vector away from the EOF directions with

high noise levels towards the low-noise directions. (In practice, the EOF

spectrum, if estimated rrom daw. should be truncated after a finite

num ber of terms, since tbe higher-index eigenvalues lend 10 he

underestimated, leading to a spurious mnpli Ikation of the higher-index

fingerprint eomponenls,

1988.)

For the special case

cr. v.Storch

of ,1 single

and H annoseh<lek, 1985, Prciscndorfer,

time-dependent variable, Ihe resuli (13)

is well known fl'Ol11 classical signal processing theory (ef. Wninstein and

Zubakov, 1(62). The EOFs for a statistically stationary Ii me sClics are

so simply the harmonic j'unctions of Ihe Fourier series representation,

thai Eq.(14) reproduces in this case the basic Iheorem that the optimal

signal detection IIlter for a station,lry time series is given by the

Fourier transform of the signal divided by thc noise variance spectrum.

Implicit in the definition

which maximizes square

statistical significance of

signal-ta-noise ratio. For

of the optimal detector as the linear variable

signal-to-noise ratio is the assllmption that the

the detector increases monolonically with the

most climate variability distributions, this

will be the case. It !la;; also been assumed thai the only source of

statistical noise in the detector is the natural climate variability. In

practice, data errors will also contribute to the detector noise. However,

these are generally slIIall compared with the climmc variabilily, and, to

avoid complicating the analysis, will be ignored.

The statistical si gni ficanee of the optimally detected signal d can be

computed from the probability distribution of the noise variable a for the

Null hypothesis that there is no signal. For this purpose it is normally

assumed that the natural climate variability is Gaussian, so that all

distributions can be derived from the covariance matrix C.

12

In this case, if !;; is known exactly (as opposed to being estimated from a

I1nitc data set), a is also Gaussian with variance

(15)

If the covDriance matrix is estimated from a finite data set, a has

statistics generally similar to a Student-t variable (Morrison, 1976).

Howeverl a differs from a Student distribution in that the direction for

which the variance of a is estimated is not prescribed a priori. but is

modified relative to the prescribed signal direction by mulliplicatioll

with the inverse of U,e estimated covariance matrix. The resultant

variable does not correspond to a standard tabulated statistical variable.

and its distribution must therefore be estimated by approximate analytical

techniques or Monte Carlo simulations.

In principle. Ule significance level can. of course, be computet! also for

an arbitrary non-Gaussian. but known statistical distribution. However. it

will normally be difficult in tbe gcneral timc-depcndent case to obtain

reliable dircct estimates or statistical distribution using, for example,

pcnnutation methods. This requires creating an ensemble of realizations.

for which time series are needed which arc significantly longer than the

analysis time interval T (the same problem "rises also in the estimation

of the covariance nHlIrix Q. The only recourse in this case may be to

augment the observational or modcl simulatcd data with still longer model

simulations of the llatural Climate vnriability.

The analysis so far has addressed only the problem of delecting a signal

with known direction g. How can the optimal detcctor d and associated

fingerprint ! be tnmslatcd now into an estimate of the signal? This

requires defining the direction amI mab~litude of tile estimated signal as a function of g. ! :md d. The answer is nOt unique and depends on how the

fact that f and g are not panlllcI is interpreted.

In H, the optimal dctection problem was l'OllllUlntcd as the task of finding

an optimal unit-length Nignul detection vcctor g, given the signal

direction 11, which maximizes the signal-to-noise ralio for the

cocft1cient c1 of the estimated signal

13

(16)

The coefficient c I itself was determined in the standard manner by

minimizing the mean square 01- the residual 1'r of the net response

(17)

which yielded

(18)

The optimal detection direction was round to lie in the direction of the

optimal fingerpJint, as given by Eq.(9),

b = .vIII,

so that the best signal cstimate was given by

Alternmively. one

signal is assumed

lie in this direction,

T 2 1'e = «I 1'> / I f I ) I

can adopt the view thm since lhe

to be g, the estimated signal should

(19)

(20)

direclion of the

also be taken to

(21)

The coefficicnt c2 should then be detellllined by the condition that in tile

absence of noise one should recover the true signal (or, equivalently,

that in the presence of noise tile mean-square deviation from the tlUe

signal should be minimized). Tilis yields

T -I -I c2 = d (g ~ g)

Thus tile estimated signal is given in this case by

",e -T Te-l-l :: = <L ~ (g ~ g) g

14

(22)

(23)

This intcrpretation will be adoptcd in the I-ollowing_

One can argue

of Ule signal

interpreted with

either vicwpoinL The suppression i 11 the fingerpri III pallem

components associated with high noise levels call be

change pattcm

existence of

lloise-COlllamillated

H to imply that one has actually

only in the low-noise fingerprint

eli milte-change signal

directions cannot be

components

supported by

tested tile climate

direction,

in the

the data_

and the

suppressed

On the

other hand, onc can adopt the prcscI1l vicw that the climate change signal,

if it exists, is specified a priori as a complete vector, and one is at

liberty to test any linear projection of the signal on to some chosen

direction as evidence of the existence of tire complete signal.

TI]~ _ n.ll!!!!:p~!!~~l] _ [lrgl?!~!!'

The single-pattern analysis can be readily

which the direction of the signal vector is

general ized

no longer

to the case

prescribed b\ll

in

is

postulated only to lie in a space spanned by p given gue.>s vcctors Il.y' y =

l, __ p ,

p s

~- I Cv £y (24)

y=l

The base signal space can be chosen used to represent the prescribed

arbitrarily. orthonormalized, however, in The guess vectors will not be

order to preserve their original physical meaning. The guess vectors may

represent, for example, differellt possible time-dependent climate change

pattcrns induced by a CO2 increase or enhanced acrosol concentrations, or

the climate change associated with variations in Ihe solar constant, or

some regional climillc change pallern. In general, these space-time

patterns will not be orthogonal.

In most applicmions, the coefficicnts cy arc cstimated from the net

response ~ , Eq.(1), by minimizing tile mean ~qllare residual ~, yielding

the standard least-squares Solulion

15

L H~:l T (25) c (gfl ;Jf) Y

II

where

fl vll = (g~ g~) (26)

However. in the present application the goal is not to maximize the

explained variance but lO maximize the signal-to-noise ralio for the

estimated signal. This yields a different solution. The [ollowing

deJivation is based on fl, but is simplified lhrough the application of the

single-pattern fingerprint cOllcept.

As before, the problem is solved in two steps. First, a set of p detectors

dy is derived for which the p-dimensional statistical significance

(assuming a Gaussian distribution) is maximized. It will be shown tl1m

thesc ilrc just the single-pattern detectors of the individual patterns &y'

(27)

where

(28)

In a second step, the coefficicllls dy arc then assigned to p base vectors

2y to construct an estimate of the signal.

From Eqs (27), (28), (4) and (9) it follows lirst that for a.ny signal t of the ronn (24) with given coefficients cY ' and therefore given signal

pallem g = t, the optimal fingerprint and associated optimal detector

arc given by, respectively,

[ (29)

and

(30)

Tbe SCI of

represent a

fingerprints r -v straighlforw;lJ'(1

Eg. (28) and detectors dv' Eq (27), therefore

generalization of the single-patlern solution

16

to the multi-pattern case ill the sense lila! they yield the optimal

fingerprint, Eq.(29), and max i m al-si gna l-lo-no i se detector, Eq.(30), for

any givcn signal, Eq.(24), in the space spmmed hy the sct of pallcrns gy

This result is relevant if one wishcs to test the sLatistical significance

of individual a priori defined components of a mulli-pallcm signal.

Although this is importanL for many llpplications, the more general

situaLion in multi-patLem signal detection is that the direction of the

signal is unknown, except thaL iL is nssullled

signal pattem space, The task is Lhen La

change signal found within Lhe space sp<llliled by

to lie

decide

in the

whether

p-dlmenslonal

the climate

guess pattems

direction of the gil

signal

is sl:itistleally significmH,

a priori,

the sci

withont

of p prescribed

specifying the

On needs in this case to find a set of p detectors whose associated

p-dimensional statistical significance mcasure is maximized for all)' signal

lying in the p-dimcnsional

solution to this prohlem

detectors dVI Eq, (27), defined

guess-pallerll space,

is again given by

by the fingerprints Lv'

H will be

the set

Eq. (28),

shown that the

of single-pattern

The standard measure of the stmistical signifieam;c of the p-dimcllsional

vector d ~ (d) in tilC prescncc of a Gaussian background noise neld is 2 - y

the p statistic

(31)

where

(32)

:jnd

(33)

represent the covariance matrix and individual components, respectively,

of the detector vector il associated with the illituml variability noise in

the absence of a signal (underlined symbols denotc vectors or matrices in

the p-dimensional space component indices ,t,v"" in contrast to the

vectors considered hitherto in [he n·dimensional observed

climate-trajectory space wilh component inciices a,ll",,).

17

If the

indeed

elimate variability and thus the probability distribution

Gaussian, the (p-l)-dimensional hypcrsurfaces p2(d) =

of a is v tonstant

represent

measure

surfaces of conslalll prob<ibility

of the statistical significance

density, Thus P2CQl of the coeff'icielll

provides a

veClor, as

discussed above. Jf the joint probability distribution is nOll~Gaussian. a

rigorous

fraught

computation

with still

one-dimensional case,

of the

greater

SO thal

statistical

sampling p2 will

significance will generally

uncertainties than in

remain a useful statistic also

be

the

in

this case, providing at least a lowest order (second moment) estimate of

the statistical significance,

For a Gaussian distribution

Q), p2@ is a 2 X variable

from data, p2@ is a

However, in analogy with

HatcHing, as

and known (:ovariancc matrix

with p degrees of freedom,

Hotelling-type variable (eL

singlc""pallem case, the

);' (and therefore

If C is estimated ~

Morrison, 1976),

p2 distribution is the

the set of estimated eoefl1cients is not strictly

modified by the

covariance Inalrix.

multiplicmion witl1

The p2 distribution

the

must

inverse );,-1 0 r the

lhcrefnre be estimated

estimated

alst) in

tllis c~se by approximate annlytieal or Monic Carlo techniques,

respect to any linear transrorm alion III a new signal pallclll

property will bc used now

forms an

begitmillg

the signal

similarly

orthonormal base,

of this section

bllSC gv' The

nOI1-orthononnal,

to require that the optimal set of

Similarly, the set of signal pallerns

that orthonormality would not be presumcd

result ani fingerprint solutions (28) wcre

Orthonorm,llity is invoked here only as

interim convenience and will be dropped again in the tinal result)

18

for

then

an

Finally, the climate state space will be tmnsformed,

(34)

to a new coordinate system in which the covariance matrix £ becomes the

unit matrix,

" T C =ACA =1 <:;:; z = = = (35)

The transformalion :;} can be obtained, for example, by first transfonning ,

to EOF coordinates '!! and then normalizing tlle EOF coordinates to unit " ,

variance) '1a \jIa lOa'

In order that tllC signal and detectors ey ' dy remain invaliant under this

transformation, the signal patterns must transform in thc samc way as the

climate state vector,

(36)

while the fingerprints tranSfOll1l as adjoint vectors,

(37)

After these transfol111utions, the statistic p2 reduces [0 the Euclidean

fonn

(38)

It follows immediatcly that the optimal set of fingerpri nt5 which

maximizes p2 for any signal ,!!S lying in " signal patterns gIl is given by

(or any equivalent rotated u

the base g) For this v .

orthonormal base

solution, p 2 =

the space spanned

which spans the

1 '!!" 12

, while

by the set of

(39)

same space as

for any other

fingerprint space, part of the sign;li will be lost in the projection onto

19

2 1 "'s 12. the fingerprint space, so lImt p S :t

Transfomling back to the original coordinates, Eq.(39) becomes, applying

Eqs. (36) and (37),

f -v

or, invoking Eq.(35),

T A A " "'v

(40)

(41) (=(28)

The 0l1hononnality conditions imposed 011 the guess pallcl'lIs J!,v and " fingerprints Iv in the transformed climate stale space transfonn into

similar condilions in the original

products defined

climatc state

now with respect

coordinate

to the

system, but

non-Euclidean with the scalar

metrics ~-l and ~, respectively. However, thcse orthonomlality conditions

can now be dropped, since only the signal and fingerprint spaces as such

arc of interest, the choice of base for either space being arbitrary. The

signal base vectors can therefore be identified with thc original guess

pMtems,

fingerprints

without normalization and orthogonality restrictions, and the

can be Similarly dcllncd, witllout ortllonormality

considerations, by the relation (41). Thus the optimal p-dimensional

detector vector d is identical to tile detcctor vector found prcviously for

the prescribed-pattern casco

It can be shown for the optimal fingerprint solution (using Eqs. (4)-(7)

and (27)-(33)) tilal the multivariate significance measure p2 for any signal ~s = g lying in the signal space gv is identical to the

single-pattern square signal-to-noise ratio R2:

p2(~sl = ~T !;?-l d T -1 (ds)2j < a 2> R2 (42) =g!;;g '" -

The result (42) holds only for the optimal fingerprints and tor the siglml

~s itself, not for the net response ~ consisting of lhe signal plus noise.

conlains (For the case that 1 <p2@> = p, while R = L)

no signal, l()r example, ~ = jij, onc finds

There remains now the

the detect ion vector d.

second step of attribliling an estimated signal ~c lO

Adopting again the view that the estimated signal

20

should reproduce the true signal when the noise eontHm ination of the

observed response ~ is negligible (or, equivalenlly, should exhibit the

smallest n1ean~squarc deviation rrom the true signal in the prescn~c of

noise) one finds, in analogy wil11 the singlc-pallern case,

ul=\d b ;r L V-y (43)

where the base VCCLOrs £y of the estimated signal arc given by

(44)

with

(45)

Thus (he estimaled signal associmcd with the set of maximally significant

detectors dv is gi ven by

)[Ie = L (1f~~~I)[I) (~r\y If" (46)

~L, Y

[n H, an altemative derivation of the maximally significant signal

estimate was given which yielded the same set of optimal dctcctors dy ' but A

a different set of base vectors Qv' and thus a different optimally

estimated signal pattern. Starting from the gcncml form (43) (with b A ~

replaced by ~v)' the detectors d" were first detcrmined by a least square A

fit to each realization )[I for a fixed base !!v' Subscqucntly, thc base was detcrmined by applying a maximal·significance condition to the set of

detectors. This yielded (as already menlionec\ fo!' the single~pattcrn case)

U,e solution ~y = Lv = C 1 gv' In contrast to the base ~v found in the

present analysis (Eq. (44», which defines the same space as the signal A

space Ji.v' the space different from the signal

spanned by the fingc'print base ~v is gene!'ally

The present result

solution using an

metric'

space.

(46) can be derived also as altcrnative definition or the square crror

C~ I rather than the usual Euclidean ~

leasl ~square~error

based on the

metric This 'significance

con'esponds to finding an estimated signal in the prescribed signal space

21

[or which the probability llllll the residual error reprcse11ls a realization

or tile natural variability ensemble is maximize';.

Representing the obscrved climalc response as the superposition

of an estim atcd sigmll

)IIc = A

\' c !' L v;;;:v V

(47)

(48)

in the signal space &1' and

error norm

I' a residual error )J!. minimization of the square

(49)

one obtains in this case UIC solution

1\ [(0-1) (,TC-I c = )II) v = V~l gft '"

(50)

~l

Equations (48) lind (50) yield the estimated

previously. but expressed now in lerms of Ihe

signal

original

(46) derived

set of base

functions g v instead of the transformed base it

I t should be emphasized

detectors dv = (L: )II) or

derivations, inde[lcndcl1t or

signal.

again A

C is v ' Ihe

thai the

maximized

definition

SLatisticil! significance

and is idclllicni for

o[ the associated

of the

all three

estimated

In practice. multi-pattern signal detection will nOlmally be carried out

in a hieruchical mode: first, a single pallem is tested; if this is

detected as signi ficalll, a second pallem is added. and so forth until the

p2 statistic is no longer accepted as statistically significant The

success of the method depends critically, as always in signal detection

problems, on the realistic choice or the prescribed signal [lallcrns &1"

22

The EOFs cab characterizing the 'pace-time covariance matrix s.;ub

represcnl familiar functions. For a fixed SUb-index (suppressed in lhe

following relations), the vector 'VCi,l) = 'Vl can first be decomposed into

EOFs eft with respect to the time index. As pointed oul in Section 3, for

a statistically stationary process, Ihe limc-(lomain EOFs arc simply the

hanllonie fUlletions of the Fourier representation (Illis foliows from thc

statistical orthogonality of the Fourier components, although these arc

nOI normally ordered with respect [0 variance, as in other applications).

Diagonalizatioll of Ihe covariance spectra with respect to the remaining

index characterizing the climate variables ami spatial coordinates

(rcfcl'cd to for brevity in tile following as U,C 'spalial' index) yields

then the standard complex EOF represelllation of the covariance spectrum of

a lIlulti-variate process (Wallace and Dickinson, 1972, Bamcll, 1983).

The complete

consists of a

space-time SCI of BOFs

different set or complex

Cab is generally

BOFs with rcspect

vcry large: it

to the spatial

index i for eacl1 frequency bund r of Ihe SpectlUnl. In practicc, it will be

difficult both to cstimale such a large set of functions from a finite

data set and to work with the complete representation in numcrical

computations. One will therefore need 10 resort to some form of

approximation or simplified model based on a reduced set of functions. Two

such models me considered in the l'olloll'ing. The first assumes statistical

separability of the lime and spHlial dimensions, while the second uses a reduced Principal Oscillation

1988, v.Slorch et aI, 1988).

Separability

defined ilere

of

as lhe space and

tile propel'll'

Pattern (POP)

time coordinmcs

thm lhe

represenla!ion (Hasselmann,

and t, respectively, is

call be factorized

inlo a spatial ei"cllvceto!' cS. and il temporal

o JI

eigenvector eab

eigenvector eh,

(51)

23

\vhcre a (i ,t), b (i,I) and the upper indices sand t will be used

gencrally

respectively.

to distinguish between spatial and temporal quantilies,

This propCl'ly holds if thc time dependent coefficients of the sWndllrd

spatial EOFs are complelel), uncorrelated, i.e. if the coefficients are

uncolTclatcd not onl)' for

definition of the spatial EOFs,

the same tillle arguments, as

but also for non-zero time lags.

impliCit in the

Cnder these condiliol1s the spatial eigenvectors satisfy the eigenvalue

equations

IC s s 2 s (52) (i,l)(i,u) ekj (011l ,k) e

ki j

in which the time indices t,u appear as parameters. Only the eigenvalues

(0~U,k)2 depend on these paramcters, not the eigenvectors themselves.

The temporal eigenvectors satisfy tile eigenvalue equmiolls

\' (s )2 et I.. 0 m,k fu (53)

u

which yields then for tile flill system tile cigcllv,llue equation

\' C e S c t I.. (i,t)(j,u) kj fu j, U

Tilc eigenvalue (0(k.O)2 rcprcscllIs the variance

of the Hutovanance spectrum of the k'th spatial

s),stem, the full space-time climate CDvariance

characterized by the spatial EOFs cki and 2

0(k,r)

(54)

of the !"th spectral band

EOF. Thus for a separable

matrix Cab is completely

their aUlOvariance spectra

In tenns v ,

of EOF coordinates (g(k,f)l

now as a superscript to relieve index

(writing the signal-pattern

congeslion) the expressions

index v for the

24

guess-signal patterns take the form

which yields for the fingerprints

The hasie advantage of

first be determined in

a separable system

tile standard manner

is thai

without

(55)

(56)

Ihe spatial EOFs can

filtering in the timc

domain. The analysis in the time-frequency dOlllllin can thell be carried out

for each EOF separately as a second step.

As illustrmion, consider singic-pattern signal consisting of a superposition of a number of spalial EOPs C~i'

(57)

where each of Ihe time dependent spatial-EOP coefficients Ykt can be

represented as a linear trend,

(58)

with mk = cons!.

Taking the Pourier trnnsform of Ykt one obtains for the coefl1cient of the

signal. in EOF coordinates.

,IT exp( - n:if/T) / 2sin(n:f/T) for f '" 0 (59)

for f = 0

The IIngerprint C,Hl then be obtained from Eq (56).

25

a)

-1

-4

-5

IT

0.5

E ·c e-j 0.0 r--

-0.5

-1.0 0 10 20 30 40 50 60 70 80 90 100 t

Fig. 1: Noise spectra 0 2f (panel a) and optimal fingerprints (panel b) for the

detection of a linear time-dependent signal in presence of power law noise. Definitions of the cases a - i and values of the associated signal-to-noise enhancement factors are given in Table I. The length of the time series is 100 (0 $ t $ 99; -50 $ IT $ 49; only the positive-frequency branehes of the specn'a are shown.)

26

__ M

Case (cf. Fig. 1) I a b c d e f g h i :

spectral power q

!

0 0.2 0.4 0.6 0.8 1.0 1.5 2 !

3

. --: :

: enhancement factor E : I 1.01 1.05 1.14 1.27 1.5 3.1 10.4 > 100 I i

i I I Table 1: Signal-to-noise enhancement factors E = [R2 (optimal fingerprint) IR2 (signal pattern)]

for linear signal and various power law spectra.

Fig. J shows the spcclrn nnd fingerprints f(i,l) computcd far red power-law

spectra, (J~k,f) = ~ (I' + T- i (1 1'01' various values of q> 0 (thc frequency

is offset by one disc!'Cle frequency unit T- l to avoid the singularily at f

0), Thc enhancement of lile square signal-la-noise !'alio R2 computed with

me optimal jingerpriIll relativc 10 the refcrence non-optimized case in

which the fingerprint is simply set equal to the signal pallclll is shown

in Table 1. The computations were carried OUi for T = 100 discrete time

sleps,

The curves detliOnSIl'ale Ihnt the optimal detection fingerprint generally

differs

by a

significantly

straight-line.

frolll

The

the nOil·optimized

non-optill1ii,ed solution

fingerprilll,

reduces

which is given

to the estim atioll of a lincarly increasing signal by

in the present case

standard method the

of constructing a regression line through the data, However, this

represents the optimal analysis method only for a whilc-lloise natural

vaJiability spectrum, For rcd the optimal weighting is

27

I

!

distributed more

spectra steeper

towards

than -I I' ,

the

the

end-points or the time series. For power-law

optimal detection strategy is to use the end

points of the lime series only. The enhancement of the statistical

significance through optimization of the fingerprilll can be quite large

for steep spectra.

An allel11ative method of reducing the complexity of the 1'1111 covaJiance

matrix Cab wi thout introducing the rather srlingenl assumption of

space-time separability is to represent the natural climate variability as

a superposition of a !inite number of Principal Oscillation Patterns

(POPs, d. Hasselmann, 1988, v.Storch et al., 1988). The basic idea of the

POP method is to combine an EOF-type pattern expansion in the spatial

domain with an ARlvlA-type dynamical modeling approach in the time domain.

In the original papers of Hassclmann and v.Storch et al. (and in a number

of subsequent applications, cf. Xu, 1992a, 1992b, Latil et aI., 1992a,

1992b), the POP method was regarded primarily as a technique for

constlUcting simple dynamical models, usually for forecasting or

diagnostic purposes. Howcver, the POP method is equally useful as an

approximate muili-variate spectral compression technique.

Tile POP method approximates the natural

here to the usual decomposed notation) as

damped oscillations 0:,

I 0: . I \Y lcrc Pi IS a constant comp ex pattern,

0: 0:( I) + i po:(2) Pi = Pi I

and the complex amplitude

satisfies the damped oscillator equation

28

variablility ~ = ~.(t) (retul11ing a 1

a supelPosition of anum ber of

(60)

(61 )

(62)

(63)

with eigcnrrcqucncy [.fl. ( > 0) nnd damping factor )/J. ( > 0); IP(t) is a

complex white-noise rorcing fUllction,

(64)

Decomposed into spectral comlxments <\J Y'(f) (to avoid notational

proliferation the Slune symbols will be used for lime and frequency-domain

functions) the solution of (63) is given by

(65)

where

(66)

is the POP transfer function, yielding for the POP oscillalion (60)

<jJ~(t) I TO:(r; lP'(f) P~ e,p (2rcirt) + complex conjugate

f

I ['1'0:(1) n''-(O py + T('-(-I)' n('-(-I/ prj exp (2rcifl) (67)

f

(nOie IIlat Ta(O '" TO:(_ f) *, n(I.(1) '" n a(t/ since both the POPs and the

noise forcing are complex).

'vl/hile

rotating

seiluence

consists

opposite

the free

clockwise po:(l) _>

I

POP solution consists of

in the _ p a(2)_>

I

complex (J.( I) >

-p i -

plane, po:(2)

I

generally

directions.

of a superposition or These arise from

frequencies ill the rcpreSCIll1l1 ion of

a single damped oscillation

patlel11 by the characterized _> P <?(l), the forced solution

two

the

n(I),

I

oscillations rotating in

positive

which

and

force

negativc

both the

basic POP pair and the complex conjugate pattern pair.

Assuming that the forcing components nU, nP for differcnt POPs a and pare

ullcorrelaled, thc complex cross SPCCtlUIll of the process

by

29

iii. is thcn given 1

F .. (JJ = <[$.(f)}* $.(/» IJ I J

O.

where

NO. = <,,rl(l) * net(lJ> = const

Met = <n(l.(O n(l.(.I» = consl

(69) (70)

The general expression (68) can be simplified by assuming that the

excitation of the conjugate POP pair, which occurs at negalive frequencies

rar removed rrom the resonant POP eigcn·rrequencies, is neglible. In this

case the second and third tcons in (68) are small for positive f (and

similarly thc first and third tenns I()r negative fl, so tilal for f > 0,

(68) reduces to

The corresponding

relation F.(!) IJ

example considered

exprcsssion * F.(I) .

IJ below, it

only the third telll1 in (68),

(71 )

for f < 0 follows from (71) and the symmeli),

(In applications, such as Ihe numcrical

will generally be more convenient 10 drop

as Ihis avoids a discoIHinuous change in thc

expressiDn for lile covariance spectrul11 which otherwise occurs at zero

frequency.)

The simplified POPs rcprcsc11lalion oj' thc Cl'Oss·specl'ul11 is seen 10 have

the same form as tile standard eom plcx EOF rcprcsc11lation

(72)

of thc cross·spcctrum in tcn1lS of complex EOFs e". The variance spectllJl11

FV (1) = < leV (f) 12> of the cocrricicnls c v (I) of the I complex EOF expansion

correspond to the spectrum N°'1 T(I.(O 12 of the POP repl'CSClllHlion while the

30

complex EOF CY itsclf corresponds to the Prillcipal Oscillation Pattcm p~, However. in contrast to the standard rcpresel1lalion, the cross-spectrum is

now no longer decomposed into a different SCI of EOFs for each spccll1Il

hand. but into a single set of complex POP pallerns applicable for the

cntire spcctrum,

The COllllibutioll of individual POPs to dillercnt spectral bands is

detem1ined by the weighting factor I T<\f) i 2 in (71). As pointed out,

individual POPs will contribute mainly to spectral bands in the

neighborhood of the POP eigcnfrcqucndcs {( the effective spectral

handwldth being proportional to the damping factor ,,0..

v In eontrasl to the twe EOF, C i' the complex POPs do nOl represent tile

eigenvectors of an Henllitian matrix ilnd will therelorc generally not be

(spatially) orthogonaL They can be readily orthogona!ized, however,

through a suitable (frequenc)' depcndenl) complex rOlmion p~' ->

f1.' I so.p p~ where So:p is unitary matrix: I (so:y{ sPy oo:P, The p, cO , a cO

I I

P y transformation preserves Ihe essenlial sl atistical orthogonality of the

POP coeflicients,

Thc detcl1llinmion of the approximate form (72) is normally carried out in

the time domain by filling n first order vcctor MlIl'kov process to the data

lime series (ef. v,Storch et ai, 1988), Howcvcr. a direct Ilt of the model

covariance spectrum to the observed covari,mcc spectnllll in the frequency

domain. using methods applied by Frankignoul and Hasselmann (J 972), Lemke

et aL (1980). HCl1crieh and Hassclmann (1982) and Dobrovolsky (1992) in

similar problems of stochastic modet fitting, should be feasible also in

lie POP-model case. Model filling in the spectral domain generally has the

advantage of providing better quantitative estimates of error hounds.

To illustrate tile impact of optimal filtering for a POP

consider the case of a signal lying in the pallcrn space

POPs, Since the di I'fcrent POP pairs are assumed 10

orthogonal, one can consider each POP signa! component

31

noise spectrum,

spanned by the

bc statislically

g.(t) = y(t) p. + complex conjugate 1 1

(73)

separately (y(t) denotes a complex time .. dependent coeJ'licicnt, and the

POP index a. has becn dropped).

Tile optimal fingcrprint

fiCt) = <\l(L) Pi + complex conjugatc (74)

can be rcpre~ented in this case in closed [01111. In the Fourier domain, the

complex fingerprint coernci~nt is givcll hy (Eqs.(14),(7J»

(75)

which yields in the time domain

The fingerprint is scen to havc the right structurc to reduce the POP

noisc conllibutions. For a modulation factor Y(1l cxp (2rcifot ),t)

eOlTcspondig to 11 pure POP oscillalion, Eq. (76) yields <\let) = 0, i.e.

pure POP oscillmions are rejected by tile fingerprint. (This result

appears paradoxical, since the fingerprint is defined in the Fourier

domain as the product of thc signal and the inverse noise SpeCtl1l111, which

cannot vanish identically. The explanation is that a pure POP oscillation

is not a permitted signal form, since it becomes infinitc lor t -> - ~, so

that its FOUlicr transform docs nnt exist. A signal which is zero [or t $;

o and represents a POP oscillation only for > 0 is not completely

removed by the fillgclprint.)

Fig.2

'\let) sholYs

for the

the POP spcctr;\

casc of a linear signal

100 lime incrcments, for taking again T

and 20 and the ), = 20

fingerplint modulation factors

modulnlion factor yet) thc frequency values r , 0

(measured in rrequency

(Eq.(58»,

(J, 10

increment . T- 1) unHS . For

damping

a linear

factor

signal (which appears as a periodic saw-tooth in

32

a)

1.0

0.8

E 2 t3 0.6 '" 0.

'" tI> 0.. 0 D..

0.4

0.2

-30 -10

-0.5

10 30 50 IT

. . . , . , . I 60 70 80 90 100

c)

oF=================~

-5

-15

-25 0 10 20 30 40 50 60 70 80 90 100 t

Fig. 2: Noise spectra (panel a) and real and imaginary components (panels, b, c, respectively) of optimal fingelprints for a POPs spectrum with damping factor 'J- = 20 and frequencies foT = 0, 10,20 (the imaginary fingerprint component vanishes for foT=O and is the same for fOT=10 as for fOT=20 to within a factor). Signal-to-noise enhancement factors for various values of I., and foT are given in Table 2. The length of the time series is 100 (05 t :s; 99; -50:S; IT :s; 49).

33

~ 0 10 20

I

10 3.5 11.21 1.16

I

20 1.63 1.12 1.08

40 1.14 1.06 1.04

i i

Table 2: Signal-to-noise enhancement factors E = lR2 (optimal fingerplint) I R2 (signal pattern)] for linear signal and POP variance spectra for variolls eigcnfrcqucncies fO and damping factors A.

the Fourier sum representation) the lirst derivative in Eq.(76) consists

of the sum of a constant tenn and a negative o-function (negative spike)

al each end of the lime illlcrval. The second derivative is given by the

derivative of a o-function, which in the discetc representation takes the

form of a posilivc and negative and spike at the hegirming and end,

respectively, of the time intclval.

The real pan of the I1ngerplill( modulation factor consists (hen of the

linear signal modulation racLOr

difference in the response at

latter term gaining more weight

unifonn white spectrum. This IS

for red power-law spectra in

separability model.

itseIr plus a term representing the

the endpoints of the tilne interval, the

the lllorc the spectrum differs from a

qualitatively similar to the result found

the previous example of a space-time

34

The imaginary component of thc footprint modulation factor consists of two

equal negative contri butions from thc endpoints of thc lime interval and

an equally weightcd positive contribution averaged over the full time

interval.

The enhanccmcnt of thc square signal-to-noise ratio R2 achieved

optim al fingerprint solution relative to thc reference case

using the

without

optimization is shown in Table 2. As in lire previous example, the largest

enhancement is achieved for noise spectra containing large variance

contributions at low frcqueneies.

The signal pallclll detection method developed by Ii for the time

independent problem can be rcadily extcndcd to the timc depcndent case.

The introduction of thc fingerprint concept leads to a significant

simplification of the thcory both conceptmilly and analytically. The

optimal fingerprint (Iilter) for the deteclion of an anticipated

space-time dependent signal pattern g in thc presence of natural climate

variability noise, characterized by the space-time dependent covariance

matrix ~, is given by I = ~-! g. Application of the optimal fingerprint to

an observed climate trajcctory ~ yields a detector d = (IT~) with maximal

signal-to-noise ratio. This result generalizes immediately to the

mulli-pattcm case: the set of fingerprints Iv= ~-I gv associated with a

set of p guess pallerns gv yields a set of detectors dv = (I~ ~s) for which

the relevant p-dimensional statistical-significance statistic p2

= L dvD~~d~l is maximized for any signal ~s lying in the space spann cd by

v, ~l the set of guess pallems gv' Here

covariance matrix of the dctector noise

variability ili.

The direction of the Jingerprint

direction of the associated

vector

signal.

interpretations of the CS(imaled signal

g~l) represcnts the

induced by the clim ate

normally di ffers from the

This permits alternative

inferred from the set of

dcteetors. In H, lire detcction and cslimaLion problems were regardcd as coupled parts of a single problem, and the estimatcd signal was defined to

35

lie in the space spanned by the sct of fingerprint veclors. In the present

analysis, the detection problem was solved first and the attribution of an

estimated signal iO the set of detectors was addressed subsequently as an

independent problem, From this viewpoint it appears more consistent to

regard the estimated signal as lying in the space spanned by the

prescdbed signal pallerns, given for eHller Arguments ean be

interpretation, The statistical eSlimated signal is significance of the

detcmlined in both cases dctectors and is thus by identical sets of

independent of thc interprctation.

Thc detection technique call be applied to any set of observed or

model-simulated data for which the second momcms can be adequately

estimated, independent of the completeness of the data set with rcgard to

the dynamical dcscliption of thc climate system,

Two practical difllculties are encountered in applying the technique,

First, a complete description of the space-time dependent covariance

matrix of the natural climate variability noise involves large quantities

of infollnation which cannot nonnally be effectively handled and also

cannot be inferred from a finite amount of observed or simulated data,

Thus some form of simpli ficd

expansion in POPs (Ptincipal

slatistical model must imroduced. An

Oscillation Pallcms) provides an

gencral reduction techniquc. In some cases a still simpler

separability model may be applicable. Examples given for both

effective

space-timc

types of

model dcmonstrMe that the optimal fingerprints can deviate significantly

from the

signal-to-noise

original

ratios

signal

compared

pattclll

with

and yield

straightforward

considerabl y

projection

enhanced

onto the

signal pallem, The strongest enhancement is obtained for red speetra with

high variance conlributions :11 vcry low frequencies,

Secondly, the optimally estinwtcd detectors dv have a known Gaussian

distribution only if the naturnl climate variability is Gaussian with

known (rather than estimated) covarinnee m:ltl1x. In the normal ease that

tile climate variability is nOll-Gaussian or the covariance matrix is

estimated from a limited data scI, the stntisticai significance of tlte

computed detectors must be cstimnlcd by Monte Carlo simulations or other

approximate techniques,

36

Not addressed in the pl'cscm paper were problems or data (un~Crlaintics were associated solely witil the natural

errors

climatc

patlCl1l,

allow for a priori probabilities of

variability) or qucstions related [0 tile

including extensions of the theory to

the anticpated signal distribution within

intended to pursue thcse qucstions

Oleo!'y.

Critlcal reviews of a first draft or

delinition of the signal

a prescribed signal space. It is

latcr in appliCiltions of the

this

helpful commelllS by Hans Vall Storch,

manuscript and a number of

Bcnjamin SanteI' and Wolfgang

Brilggcmann are gratefully acknowledged.

37

References

Bal11ett, T.P., 1983. Interaction of the Monsoon and Pacific Trade Wind System at Interannual

Time Scales. Part I: The Equatorial Zone. Mont. Weath. Rev., 111, 756 - 773.

Barnett, T.P., 1986. Detection of changes in global tropospheric temperature field induced by

greenhouse gases. 1. Geophys. Res., 91,6659 - 6667.

Barnett, T.P., 1991. An attempt to detect the greenhouse-gas signal in a transient GCM

simulation. In: Greenhouse-Gas-Induced Climatic Change: A Critical Appraisal of

simulations and Observations. M.E. Schlesinger (ed.), 559 - 568.

Barnett, T.P., and K. Hasselmann, 1979. Techniques of linear prediction, with application to

oceanic and atmospheric fields in the tropical Pacific. Revs Geophys. Space Phys.,

17,949 - 968.

Barnett, T.P., and M.E. Schlesinger, 1987. Detecting changes in global climate induced by

greenhouse gases. 1. Geophys. Res., 92, 14,772 - 14,780.

Barnett, T.P., M.E. Schlesinger, and X. Jiang, 1991. On greenhouse gas signal detection

strategies. In: Greenhouse-Gas-Induced Climatic Change: A Critical Appraisal of

simulations and Observations. M.E. Schlesinger (ed.), 537 - 558.

Bell, T.L., 1982. Optimal weighting of data to detect climatic change: Application to the

Carbon Dioxide problem. 1. Geoph}'s. Res., 87,11,161- 11,170.

Bell, T.L., 1986. Theory of optimal weighting of data to detect climatic change . .I. Atmos. Sci..

43, J 694 - 1710.

Bloomfield, P., and D. Nychka, 1992. Climate spectra and detecting climate change. Clim.

Change (in press).

Cubasch, U., K. Hasselmann, H. Hock, E. Maier-Reimer, U. Mikolajewicz, B.D. Santer, and

R. Sallsen, 1992. Time dependent greenhollse warming computations with a coupled

ocean-atmosphere model. Climate Dynam. (in press).

38

Dobrovolski, S.G., 1992. Global climate changes in water and heat transfer accumulation

processes. Del'S Atmos. Science, 21, Elsevier, 265 pp.

Frankignoul, C., and K. Hasselmann, 1977. Stochastic climate models, Part 2: Application to

sea-sUliace temperature anomalies and thermocline variability. Tellus, 29, 289 - 305.

Hannosch6ck, G., and C. Frankignoul, 1985. Multivariate statistical analysis of sea surface

temperature anomaly experiments with the GrSS general circulation model. f. Atmos.

Sci .. 42, 1430 - 1450.

Hasselmann, K., 1979. On the signal-to-noise problem in atmospheric response studies.

Meteorology of Tropical Oceans. Royal Met. Soc., 251 - 259.

Hasselmann, K., 1988, PIPs and POPs: The reduction of complex dynamical systems using

principal interaction and oscillation pattems. f. Geophys. Res., 93, 11,015 - 11,021.

Hayashi, Y, 1982. Confidence intervals of a climate signal. f. Atmos. Sci. 39, 1895 - 1905.

Hense, A., 1986. Multivariate statistical investigations of the northern hemisphere circulation

during the El Nino event 1982/83. Tellus, 38A, 189 - 204.

Hense, A., R. G1owienka-Hense, H. v. Storch, and U. Stahler, 1990. Northern Hemisphere

atmospheric response to changes of Atlantic Ocean SST on decadal time scales:

a GCM experiment. Climate DYllam., 4, 157 - 174.

HerteJich, K., and K. Hasselmann, 1987. Extraction of mixed layer advection velocities,

diffusion coefficients, feedback factors and atmospheric forcing parameters from the

statistical analysis of North Pacific SST anomaly fields. f. Phys. Oceallogr., 17,

2145 - 2156.

Karoly, DJ., 1987. Southern hemisphere temperature trends: A possible greenhouse gas effect?

Geophys. Res. Letts, 14, 1139 - 1141.

Karoly, D.l, 1989. Northern Hemisphere temperature trends: A possible greenhouse gas

effect? Geophys. Res. Letts, 16,465 - 468.

39

Latif, M., A. Sterl, E. Maier-Reimer, and M.M. Junge, 1992a. Climate variability in a coupled

general circulation model. Part I: The tropical Pacific. 1. Climate (in press).

Latif, M., A Sterl, E. Maier-Reimer, and M.M. Junge, 1992b. Structure and predictability of

the EI Nino/Southern Oscillation phenomenon. 1. Climate (submitted).

Lemke, P., E.W. Trink!, and K. Hasselmann, 1980. Stochastic dynamic analysis of polar sea

ice variability. 1. Phys. Oceanogr., 10,2100 - 2120.

MacCracken, M.e., and H. Moses, 1982. The first detection of carbon dioxide effects:

Workshop Summary, 8 - 10 June 1981, Harpers FelTY, West Virginia. Bull. Am.

Met. Soc., 63,1164 - 1178.

Madden, R.A, and V. Ramanathan, 1980. Detecting climate change due to increasing carbon

dioxide. Science, 209, 763 - 768.

Manabe, S., R.J. Stouffer, M.J. Spelman, and K. Bryan, 1991. Transient responses of a coupled

ocean-atmosphere model to gradual changes of atmospheric CO2, Part I: Annual

mean response. 1. Climate, 4, 785 - 818.

Mikolajewicz, U., and E. Maier-Reimer, 1990. Internal secular variability in an ocean general

circulation model. Climate Dynam., 4, 145 - 156.

Morrison, D.P., 1990. Multivariate Statistical Methods (3rd edition). McGraw Hill, 495 pp.

Munk, W.H., and AM.G. Forbes, 1989. Global ocean warming: An acoustic measure?

1. Phys. Oceanogr., 19, 1765 - 1778.

Preisendorfer, R.W., 1988. Principal component analysis in meteorology and oceanography.

Devs Almos. Science, 17, Elsevier, Amsterdam, 425 pp.

Solow, A., 1991. On the statistical compaIison of climate model output and climate data. In:

Greenhollse-Gas-Jnduced Climatic Change: A Critical Appraisal o/Simulations and

Observations. M.E. Schlesinger (ed.) 505 - 510.

Storch, H. v., and G. HannoschOck, 1985. Statistical aspects of estimated principal vectors

("EOFs") based on small sample sizes. 1. Climate Appl. Met., 24,716 - 724.

40

Storch, H. V., T. Bruns, I. Fischer-Bruns, and K. Hasselmanll, 1988. Principal oscillation

pattern analysis of the 30- to 60-day oscillation in general climate model equatorial

troposphere. J. Geophys. Res., 93, 11,022 - 1l,036.

Stouffer, R.I., S. Manabe, and K. Bryan: Interhemispheric asymmetry in climate response to a

gradual increase of atmospheric CO2, Nature, 342, 660 - 662, 1989.

Wainstein, L.A., and V.D. Zubakov, 1962. Extraction a/signals/rom noise (trans!. by R.A. Sil­

verman), Prentice-Hall, Inc., Englewood Cliffs., N.J., 362 pp.

Wallace, J.M., and R.E. Dickinson, 1972. Empirical orthogonal representation of time series in

the frequency domain, Part I, Theoretical considerations. J. Appl. Met., 11, 887 - 892.

Washington, W.M., and O.A. Meehl: Climate sensitivity due to increased CO2: Experiments

with a coupled atmosphere and ocean general circulation mode!. Climate Dynam., 4,

I - 38, 1989.

Wigley, T.M.L., and T.P. Barnett, 1990. Detection of the greenhouse effect in the observations.

In: Climate Change. The !PCC Scientific Assessment (Eds. LT. Houghton,

OJ. Jenkins and J.J. Ephraums). Cambridge University Press, Cambridge, pp. 239 -

256.

Wigley, T.M.L., and P.D. Jones, 1981. Detecting COTinduced climatic change. Nature, 292,

205 - 208.

Xu, .l.S., 1992a. On the relationship between the stratospheric quasi-biennial oscillation and

the U'opospheric southern oscillation. J. Atmos. Sci., 49,725 - 734.

Xu, J.-S., 1992b. The joint nonnal modes of the coupled ocean-atmosphere system observed

from 1967 - 1986. J. Climate (in press).

41