diagnosing sources of u.s. seasonal forecast skill · diagnosing sources of u.s. seasonal forecast...

Diagnosing Sources of U.S. Seasonal Forecast Skill

X. QUAN, M. HOERLING, J. WHITAKER, G. BATES, AND T. XU

NOAA/Earth System Research Laboratory, Boulder, Colorado

(Manuscript received 12 April 2005, in final form 29 September 2005)

ABSTRACT

In this study the authors diagnose the sources for the contiguous U.S. seasonal forecast skill that arerelated to sea surface temperature (SST) variations using a combination of dynamical and empirical meth-ods. The dynamical methods include ensemble simulations with four atmospheric general circulation models(AGCMs) forced by observed monthly global SSTs from 1950 to 1999, and ensemble AGCM experimentsforced by idealized SST anomalies. The empirical methods involve a suite of reductions of the AGCMsimulations. These include uni- and multivariate regression models that encapsulate the simultaneous andone-season lag linear connections between seasonal mean tropical SST anomalies and U.S. precipitationand surface air temperature. Nearly all of the AGCM skill in U.S. precipitation and surface air temperature,arising from global SST influences, can be explained by a single degree of freedom in the tropical SSTfield—that associated with the linear atmospheric signal of El Niño–Southern Oscillation (ENSO). Theresults support previous findings regarding the preeminence of ENSO as a U.S. skill source. The diagnosticmethods used here exposed another skill source that appeared to be of non-ENSO origins. In late autumn,when the AGCM simulation skill of U.S. temperatures peaked in absolute value and in spatial coverage, themajority of that originated from SST variability in the subtropical west Pacific Ocean and the South ChinaSea. Hindcast experiments were performed for 1950–99 that revealed most of the simulation skill of the U.S.seasonal climate to be recoverable at one-season lag. The skill attributable to the AGCMs was shown toachieve parity with that attributable to empirical models derived purely from observational data. Thediagnostics promote the interpretation that only limited advances in U.S. seasonal prediction skill should beexpected from methods seeking to capitalize on sea surface predictors alone, and that advances that mayoccur in future decades could be readily masked by inherent multidecadal fluctuations in skill of coupledocean–atmosphere systems.

1. Introduction

Foremost among U.S. seasonal forecast skill sourcesis the state of tropical sea surface temperatures (SSTs),a relation stemming from the sensitivity of atmosphericstationary waves and storm tracks to tropical forcing(e.g., Hoskins and Karoly 1981; Held et al. 1989). Todate, El Niño–Southern Oscillation (ENSO), which istypified by east tropical Pacific SST variations, is thesingle phenomena of the ocean–atmosphere systemdemonstrated to render U.S. seasonal forecasts skillful(e.g., Barnett and Priesendorfer 1987; Barnston 1994;Higgins et al. 2004). That skill is elevated in the winterand spring seasons, at which time it also exhibits a largespatial footprint over the United States because of theplanetary scale of ENSO-forced circulation variations.

The question we pose is whether ENSO constitutesthe sole tropical SST source of U.S. seasonal forecastskill. To be sure, empirical and simulation studies haveidentified sensitivities to SST forcings outside the easttropical Pacific. For example, the influence of SSTanomalies over a region north of the equator in thetropical west Pacific was highlighted by Palmer andOwen (1986) as contributing to the severe U.S. winterof 1976/77. Empirical support for such a connection wasprovided by studies of the relation between atmo-spheric circulation patterns and proxies for tropicalconvection (Livezey and Mo 1987). The influence ofSST anomalies over the warm pool regions of the In-dian and western Pacific Oceans have also been shownto play roles in the U.S. climate, such as during 1997and 1998 (Pegion et al. 2000), and during 1998–2002when the United States experienced protracted warmand dry conditions (Kumar et al. 2001; Hoerling andKumar 2003). Idealized modeling experiments, bothlinear dynamical models forced by tropical heating

Corresponding author address: X. Quan, NOAA/ESRLRIPSD1, 325 Broadway, Boulder, CO 80305-3328.E-mail: [email protected]

1 JULY 2006 Q U A N E T A L . 3279

© 2006 American Meteorological Society

JCLI3789

(e.g., Ting and Yu 1998; Chen and Newman 1998) andperpetual-mode general circulation models forced withidealized SSTs (Geisler et al. 1985; Barsugli andSardeshmukh 2002), have confirmed robust Pacific–North American responses to forcings located outsidethe classic ENSO region of the east tropical Pacific.

Historical simulations of twentieth-century climateusing atmospheric general circulation models (AGCMs)subjected to observed monthly evolving global SSTs arenow permitting diagnosis of sensitivity patterns underrealistic conditions. The availability of multi-AGCMs,each run numerous times over an historical period, per-mits a more robust and reliable identification of atmo-spheric response patterns associated with tropical forc-ing than hitherto possible. Using a 46-member en-semble of 50-yr experiments spanning 1950–99 fromfour different AGCMs, Hoerling and Kumar (2002)subjected the monthly 500-hPa height fields of the mul-timodel ensemble to an empirical orthogonal function(EOF) analysis. These yielded evidence for at least fourdifferent characteristic atmospheric patterns associatedwith tropical SST variability. The leading one was thewell-known ENSO teleconnection. However, the addi-tional patterns accounted for up to 50% of the localboundary-forced variance over parts of North America,and appeared to be of non-ENSO tropical origin. It isunknown whether these additional sensitivities contrib-ute to skill of U.S. surface temperature and precipita-tion predictions.

An ensemble of AGCM runs yields a wealth of in-formation about the sensitivity of the atmosphere toobserved SST anomalies. However, it is often not clearwhat the relationship between the prescribed SST andthe GCM ensemble mean atmospheric response is. Forexample, further diagnosis is necessary to determinehow many degrees of freedom in the SST field are ac-tually involved in producing the simulated atmosphericsignals. This is useful information, since it defines therelevant phase space of SST patterns that need to bepredicted accurately for a seasonal forecast. Even ifinformation can be obtained about the SST patternsfrom the Atmospheric Model Intercomparison Project(AMIP) runs (see, e.g., Hoerling and Kumar 2002), oneis still left wondering how predictable those patternsare. Goddard and Mason (2002) examined the predict-ability issue by running a complementary ensemble ofGCM runs, but instead of the traditional AMIP ap-proach, they use SSTs that persisted (as a simple pre-diction) for one season. In this way they were able toinvestigate the predictability of the seasonal mean at-mospheric state based on forecast SSTs rather than si-multaneous SSTs. Here we attempt to address both thequestion of how many degrees of freedom are needed

to reproduce the ensemble mean GCM response overthe United States, and the question of how predict-able that response is, using an empirical statisticalmodel trained on the 46-member multimodel GCM en-semble.

Our study combines AGCM and empirical methodsto diagnose the simulation and hindcast skill of seasonalsurface climate over the United States during 1950–99.The datasets and methods are presented in section 2.Section 3 presents results in which the seasonally vary-ing skill of surface temperature and precipitation aredescribed and diagnosed. In addition to the expectedENSO contribution, a non-ENSO source of skill is dis-covered. Section 4 focuses on a diagnosis of this addi-tional source of U.S. skill. A summary and concludingcomments are given in section 5.

2. Data and method

a. Observations

Monthly observed U.S. surface temperature andrainfall data for 1950–99 are available on a 2.5° gridbased on the Global Historical Climate Network.Monthly SST analyses for 1950–99 come from the MetOffice Hadley Center’s Global Sea Ice and SST dataset(Rayner et al. 2003). The global SSTs are created usingvarious techniques including reduced space optimal in-terpolation, and are available on a 1° grid.

b. Atmospheric climate simulations forced byglobal SSTs

The SST role in climate variability is assessed usingAGCMs forced with the specified, observed monthlyvariations in SSTs for 1950–99. Multiple integrationsare begun from different atmospheric initial conditions,but in which each ensemble member is subjected toidentically specified sea surface conditions.

Four different AGCMs are used, consisting of a totalof 46 simulations spanning the last half of the twentiethcentury. The models used are identical to those inHoerling and Kumar (2002). Table 1 summarizes theirspatial resolutions, and the reader is referred to theindicated references for details on different dynamicalcores and physical parameterizations used in eachmodel. The AGCMs include 12 simulations using theNational Center for Atmospheric Research (NCAR)Community Climate Model (CCM3; Kiehl et al. 1998),12 simulations using the National Centers for Environ-mental Prediction Medium-Range Forecast model(MRF9; Kumar et al. 1996), 10 simulations usingthe European Center–Hamburg Model (ECHAM3;

3280 J O U R N A L O F C L I M A T E VOLUME 19

Roeckner et al. 1992), and 12 simulations using the cli-mate model of the Geophysical Fluids Dynamics Labo-ratory (GFDL; Broccoli and Manabe 1992).

Our analysis of the AGCMs is based on the com-bined 46-member ensemble. The ensemble mean sea-sonal anomaly for each of the four models is calculatedrelative to the respective models’ climatology for eachof the 600 overlapping seasons during 1970–99. Theensemble mean anomaly of each model is standardizedby its SST boundary-forced (external) variance, as de-scribed in more detail in Hoerling and Kumar (2002).These standardized anomalies are then averaged acrossthe four models in order to produce the multimodel,standardized anomaly. Actual anomalies are recoveredby multiplying the standardized anomaly of each seasonby the four-model-averaged external variance. Hori-zontal and vertical resolutions of these models arelisted in Table 1. In an analysis examining the impact ofdifferent ensemble mean methods Y.-H. Byun (2002,personal communication) compared the skill of this 46-member ensemble mean and the skill of another en-semble mean of the same 46-member but used aweighted average in which the weights were differentfor each member and determined by minimizing thedifference between the model ensemble mean and theverification field. Byun’s results indicated that for the1950–99 50-yr period, the simulation skill of theweighted ensemble mean is virtually the same as theskill of the ensemble mean used in this study.

c. Atmospheric climate simulations forced byidealized SSTs

Simulations using a fixed idealized SST anomaly overthe subtropical western Pacific are conducted, the pat-tern of which is discussed in section 4. This anomaly isspecified and fixed throughout the seasonal cycle, andis added to the seasonally varying climatological SSTs.A total of 20 14-month simulations, beginning fromrandomly selected 1 November atmospheric initial con-ditions, were performed with NCAR’s CCM3, one ofthe models included in our multimodel suite forced byrealistic SSTs.

d. Empirical climate prediction model

The empirical model is based on uni- and multivari-ate linear regression that relates a set of seasonally av-eraged fields of tropical (20°S–20°N) SST (the predic-tors) to a set of seasonally averaged model-simulatedfields of U.S. surface temperature and precipitation(the predictands) (see Fig. 1). Regression models aredeveloped both for the simultaneous and one-seasonlag relationships between predictor–predictand pairs.This allows us to estimate the relevant SST subspaceboth for the zero-lag (contemporaneous) GCM simula-tion, and the one-season-lag GCM prediction. Unlikethe Goddard and Mason (2002) study, we do not rerunthe GCM with predicted SSTs, but instead use the lag-covariance relationships between the SST and the simu-lated atmospheric response from the AMIP integra-tions to estimate what the GCM prediction would be,given actual predicted SSTs. This approach assumesthat the relationship between SSTs and the atmosphericresponse over the United States is linear. This hypoth-esis is tested in the next section by verifying that theempirical model can reproduce the GCM simulationskill over the United States. The empirical statisticalmodel we use is very similar to others used for seasonalprediction (e.g., Barnston 1994), except that in our casethe model is trained on the 46-member ensembleaverage of multimodel simulations described in sec-tion 2b, instead of observed atmospheric states. Bytraining on the model ensemble mean, our tool de-scribes the relation between atmospheric signal andtropical SST forcing, whereas similar tools trained onthe brief, single realization of observed states predict ablend of signal and noise given a tropical SST state.

In the construction of the statistical model, the raw,seasonal predictor and predictand data are first prefil-tered using EOFs in a manner outlined by Barnett andPreisendorfer (1987) and as implemented by Barnston(1994). We retain N-EOFs in predictor and M-EOFs inpredictand fields, where the truncation is determinedby applying the formalism of North et al. (1982). Forthe tropical SSTs, N is chosen to be five for all 12 over-lapping seasons. For U.S. precipitation, M is chosen tobe six for all seasons, while M is chosen to be nine forU.S. surface temperature. The North et al. (1982)analysis suggested that the EOF modes higher than acritical order are statistically indistinguishable. There-fore, we do not include higher-order SST modes in ouranalysis and treat the higher-order SST modes as noise.The EOF expansion coefficients (principal compo-nents) are then normalized by the square root of theireigenvalues, and the EOF basis is calculated indepen-dently for each of the 12 overlapping three-month sea-

TABLE 1. Characteristics of the AGCMs used in the study.

ModelSpectral

resolutionSigmalayers

Ensemblesize Reference

CCM3 T42 18 12 Kiehl et al. (1998)ECHAM3 T42 18 10 Roeckner et al.

(1992)GFDL R30 18 12 Broccoli and

Manabe (1992)MRF9 T40 18 12 Kumar et al.

(1996)

1 JULY 2006 Q U A N E T A L . 3281

sons. The cross-covariance matrix is calculated in EOFspace, and the predicted EOF expansion coefficientsare transformed back to a 2.5° latitude–longitude grid.A parallel set of univariate regression models is con-structed to isolate the contribution of ENSO to thepredictands of interest. The univariate regression usesonly the first EOF (i.e., ENSO) mode of the tropicalSST as a predictor, and six (nine) EOF modes of theU.S. precipitation (surface temperature) as pre-dictands. The univariate regression is also trained onthe multimodel ensemble mean.

e. Verification methodology

We employ a deterministic measure of skill basedupon the temporal correlation. Monthly mean surfaceair temperature and precipitation anomalies are ob-tained by subtracting the 30-yr climatology of 1970–99for each calendar month from the monthly mean valuesat each grid point over our domain (see Fig. 2). Three-month mean anomalies are then calculated from themonthly mean anomalies. The same method is appliedto the output from the 46-member AGCM simulations.Verification is performed using the three-month meananomalies.

All the uni- and multivariate empirical model simu-

lations and hindcasts for each three-month season em-ploy a cross-validation procedure in which the model isconstructed from the covariance matrix of 49 yr ofseasonal data, excluding the target season for the simu-lation or hindcast. In this manner, a “new” empiricalmodel is constructed for each year, and also for eachseason. The 1951–99 skill of the empirical modelsimulations and hindcasts was then calculated in theway identical to that applied to the AGCM simula-tions. When calculating spatial averages of correlationskill for U.S. surface temperature, coastal grid points(indicated by circles in Fig. 2) are excluded because

FIG. 1. Schematic of the empirical model used to simulate and forecast U.S. seasonalprecipitation P and surface air temperature Ts. The indices i and j denote the seasons used ineach simulation and hindcast experiment. In the simulations, j is equal to i for each experi-ment. In the one-season-lead hindcasts, j is one-season lag to i (e.g., when i points to OND of1998, j is JFM of 1999). See text for further details.

FIG. 2. Grids included in the verification for precipitation (allcircle and plus sign grid points), surface temperature (only theplus sign grid points).


local temperature skill at those areas can be largelydetermined by the prescribed SST in AGCM simula-tions.

The field significance of the correlation skill maps foreach variable and for each of the 12 three-month sea-sons is tested in a manner similar to Livezey and Chen(1983). For a given variable and season, Monte Carloexperiments of 3000 samples are first performed by ran-domly ordering the 50-yr time sequence of observedfields and then calculating temporal correlations withthe true 1950–99 observed time sequence. From these,we compute the probability distribution of the numberof U.S. points in Fig. 2 having temporal correlationsexceeding 90% local significance (r � 0.25). Then, thenumber of U.S. grid points N0 at 95% of the accumu-lated obtained distribution constitutes the minimumnumber necessary for passing field significance. In or-der that a correlation skill pattern be field significant,the actual number of locally significant grid points Nmust exceed N0; that is, the ratio N/N0 must exceed 1.

3. Diagnosing SST skill sources

a. U.S. precipitation

The annual cycle of simulation skill, and its sources,averaged over the United States is summarized in Fig.3. Correlation skill of the AGCM simulations is low inall seasons (top), having a minor maximum during win-ter (thick solid line). Its primary source is tropical SSTs,as demonstrated by the ability of the multivariate re-gression model simulation skill (thin solid line) to fullyrecover that of the global SST-forced AGCMs. Further,a single pattern of the tropical SST variations is able toexplain this skill (dotted curve), namely the ENSO sig-nature of the tropical east Pacific (not shown). Thesimulation skills are field significant in winter andspring, but not so in summer and early autumn (bot-tom).

Inspection of the spatial patterns of skill reveals theunivariate regression model to be superior to theAGCM simulation in select regions. Figure 4 shows thetemporal anomaly correlation for the U.S. domain, andthe plotted values are the truncated correlation skillscores (�10) at all points exceeding 90% significance.The lower right corner of each panel plots the ratioN/N0, which denotes field significance for values �1.The six columns correspond to partially overlappingthree-month seasons, and the top three panels of eachcolumn display the AGCM simulation skill, the multi-variate regression model skill, and the univariate re-gression model skill, respectively.

During January–March (JFM), significant skill cov-

ers nearly half the domain in the univariate model, andmimics the spatial footprint of ENSO’s influence onU.S. precipitation (e.g., Ropelewski and Halpert 1989;Kiladis and Diaz 1989). This pattern is only weaklydiscernable in the AGCM simulation skill. Throughoutall remaining seasons, it is found that the AGCM simu-lation skill is generally surpassed by a reduced-spacerepresentation of the relation between SST and modelU.S. precipitation.

The lower panels of each column display the zero-lead hindcast skill using the multivariate model. Theserecover virtually all of the 50-yr-averaged AGCM simu-lation skill. The univariate model often performs betterthan the multivariate model. This is so because ENSOSSTs are the primary skill source, and the fact that suchan ENSO pattern can, itself, be skillfully predicted atshort lead times.

b. U.S. surface temperature

The SST source for U.S. surface temperature skill isfound to be determined not by ENSO alone, in contrast

FIG. 3. Spatial average of the 1951–99 temporal correlation forthe (top) U.S. precipitation and (bottom) field significance values.Points above the gray shading are statistically significant at 95%confidence level.

1 JULY 2006 Q U A N E T A L . 3283

to the pure ENSO source that was diagnosed for U.S.precipitation skill. Figure 5 summarizes the area-averaged skill of the three simulation datasets, fromwhich it is evident that the three simulation skills arecomparable during winter and spring, but that theAGCM skill is greater than either multi or univariateregressions in summer and autumn. Note that theAGCM skill is field significant in all seasons (bottom),contrary to the empirical model skill, which are notfield significant from September–November (SON)through November–January (NDJ).

There is a distinct NDJ seasonal peak in simulationskill, at which time the U.S.-averaged 50-yr mean skillexceeds 0.36. The field significance exceeding 99.9% ismore remarkable when compared with the lack of fieldsignificance by either empirical model simulation. Bycontrast, the coincidence in skill of the three simula-tions during winter and spring indicates that ENSO’slinear signal is the primary source. An ENSO SST

source appears to impart little skill during other seasonsas indicated by the large divergence between theAGCM and regression model skills at those times.These other SST sources could be of tropical origin, butare unaccounted for by our truncated multivariatemodel, or they could be of extratropical sea surfaceorigin. It is also possible that nonlinear processes asso-ciated with the atmospheric response to SST forcing areimportant contributors to the AGCM skill, relationsalso not represented in the regression models.

The spatial distribution of surface temperature skill,shown in Fig. 6, offers further insights on SST sources.There is little appreciable difference in skill patternsoccurring among the three simulations during winterand spring—the similarity with the univariate modelskill confirms once again that the linear component ofthe ENSO-forced atmospheric signal is of primary con-sequence. On the other hand, the AGCM simulationskill is greater in both its coverage and its absolute

FIG. 4. Temporal correlation between observed and simulated/hindcast three-month mean anomalies of precipitation. Small-fontnumbers are the first digit of the correlation coefficient value at each grid point having higher than 90% local significance. Large-fontnumbers shown at bottom right corner in each panel indicate field significance level (N/N0) of the corresponding correlation patterns.Ratios exceeding 1 indicate field significance at 95% confidence level. Note: the lag-0 in figure represents simulation, and lag-1 is usedfor zero-lead hindcast in text.


value than either regression model during summer andautumn. The footprint differs from the linear ENSOcontribution which, as illustrated by the univariate skillmap, is confined to the southern United States.

Shown in the lower panels of Fig. 6 are the hindcastskills for the multivariate model. These recover muchof the simulation skill of U.S. surface temperature dur-ing winter/spring, consistent with the fact that the linearENSO influence is of greatest relevance and that suchSSTs are themselves skillfully predicted at zero lead. Incontrast, they fail to reproduce the extensive spatialcoverage of the AGCM’s simulation skill during theNDJ season.

4. A subtropical west Pacific skill source

We first explore the SST structure associated withthe AGCM’s high skill over the northern United Statesduring NDJ. The black-shaded box in Fig. 7 keys on theregion of high AGCM skill, and is the domain overwhich surface air temperatures are averaged in order to

generate a 1950–99 time series. The correlation of thattime series with SSTs across the global oceans is shownby the shaded contours, and the analysis is performedusing both the AGCM (top) and observed (bottom)surface air temperatures. Maximum correlations ex-ceeding �0.5 occur with SSTs over the northwest Pa-cific Ocean and the South China Sea, and there is alsosome suggestion of an ENSO relation in so far as �0.3correlations occur with SSTs over the tropical east Pa-cific. However, the fact that an ENSO region correla-tion is very weak in the observations suggests it is un-likely an appreciable skill source. By contrast, the con-sistency between the simulated and observed highcorrelations over the western Pacific suggests that SSTvariations there could be an important skill source fornorthern U.S. surface air temperature.

The coherency of their interannual fluctuations lendssome support for a causal relation between subtropicalwest Pacific SSTs and the northern U.S. index region’stemperatures. Figure 8 overlays the times series of thenorthern U.S. region’s temperature index and an index

FIG. 4. (Continued)

1 JULY 2006 Q U A N E T A L . 3285

of SST averaged over the west Pacific box of Fig. 7. Theformer is calculated from both AGCM (Fig. 8, top) andobserved (Fig. 8, bottom) surface air temperatures. Forboth, the correlation with the west Pacific SST indexexceeds �0.5, with warm sea surface conditions accom-panying warm northern U.S. surface temperatures. Inaddition to the tendency for interannual swings of eachindex to be aligned, a post-1970 warming trend in bothocean and land time series also contributes to the posi-tive temporal correlations.

Figure 9 shows the temporal variation of our westPacific SST index with SST elsewhere. The coherencyof an SST signature that encompasses the South ChinaSea, Philippine Sea, and the subtropical west Pacific isevident. There is only a weak simultaneous relationwith SSTs over the ENSO region, again suggesting thatthis west Pacific source of U.S. simulation skill in theAGCM is not an obvious proxy for ENSO’s effect.

Independent evidence for a causal link betweenvariations in west Pacific SSTs and U.S. surface tem-peratures is provided by the additional AGCM experi-

FIG. 5. Same as Fig. 3, but for the U.S. surface temperature.

FIG. 6. Same as Fig. 4, but for the U.S. surface temperature.


ments forced exclusively with SSTs over that region.The idealized anomalous SST forcing is obtained froma composite of the SST pattern contrasting warm andcold events (when anomalous values exceeded onestandard deviation) in the northern U.S. temperature(Fig. 8, top), and has similar structure as outlined by theirregular contour in Fig. 9, and encompasses the spatialdomain of temporally coherent west Pacific SST varia-tions. A 20-member ensemble of CCM3 simulationswas conducted using both positive and negative polari-ties of the SST anomalies, with maximum amplitudes of1°C. The NDJ surface temperature anomalies of the(warm–cold) experiments are shown in the lower panelof Fig. 10. Widespread warming covers North America,with a maximum signal over the northern United Statesand southern Canada. This response pattern is remark-ably similar with the 1950–1999 correlation between anindex of the west Pacific SSTs and both multimodelAGCM (Fig. 10, top) and observed (Fig. 10, middle)surface temperatures. Indicated hereby is that the sta-tistical correlation does indeed describe a cause–effectrelationship.

The implications of the above analyses is that a bi-variate regression model, having west Pacific and tropi-cal east Pacific SST predictors, should be able to re-cover the AGCM simulation skill during the NDJ sea-son. The simulation skill of such a bivariate model isshown in Fig. 11 (middle panel). For comparison, thesimulation skill of only a univariate model using thewest Pacific SST as predictor is shown in the top panelof Fig. 11. Nearly all of the AGCM’s simulation skillover the northern United States during NDJ (see Fig. 6)is accounted for by a west Pacific influence. The resultsof the bivariate model, which also include the ENSOinfluence, almost completely recover the AGCM simu-lation skill. These calculations support the argumentthat the high AGCM simulation skill for NDJ surfacetemperature over the northern United States is attrib-uted to the ocean–atmosphere interactions in the west-ern Pacific Ocean.

As a final analysis, we reexamine the hindcast skill ofNDJ surface temperatures, but now use a one-season-lag bivariate regression model. The results are shown inthe lower panel of Fig. 11, and it is apparent that the

FIG. 6. (Continued)

1 JULY 2006 Q U A N E T A L . 3287

vast majority of simulation skill is retained in the pre-diction mode. This suggests that the west Pacific SSTpredictor is itself predictable and slowly evolving on theseasonal time scale. The new subtropical SST sourcebears some resemblance to the third canonical correla-tion analysis SST mode in Barnston and Smith (1996).However, our analysis revealed that the SST variationin the subtropical northwest Pacific Ocean is not a di-rect response to ENSO. In fact, the SST variation in thesubtropical NW Pacific Ocean leads an out-of-phaseENSO episode by about one year (not shown).

5. Summary and discussion

a. Summary

The SST origins for seasonal forecast skill of U.S.precipitation and surface air temperature have been di-agnosed. The principal tool consisted of ensembleAGCM simulations that were subjected to the knownmonthly varying global SSTs from 1950–99. The simu-lation skill of the resulting multimodel 46-member en-

FIG. 7. Spatial distribution of the correlation coefficient between SST and an index of regional meansurface temperature over the northern United States (42°–52°N, 120°–70°W) based on (top) AGCMsimulated and (bottom) observed NDJ surface temperatures. The contour interval is 0.1. The black-shaded box denotes the index region for constructing land temperature time series, and the white boxin the northwest Pacific denotes the index region for constructing SST time series.

FIG. 8. Time series of NDJ anomalies of northern U.S. surfacetemperature (solid line) based on (top) AGCM simulations and(bottom) observations. The dotted line denotes the time series ofNDJ SSTs averaged over the western Pacific region. Insertedboxes of Fig. 7 show the averaging domains for constructing eachtime series.


semble was evaluated, and then diagnosed using twoempirical reductions of the full AGCM data. The firstconsisted of a multivariate linear regression model re-lating the simultaneous states of five leading tropicalSST patterns to the AGCM’s U.S. seasonal climate.The second was a univariate linear regression modelwhose predictor consisted of the single leading tropicalSST pattern only—ENSO.

Our diagnosis revealed that the AGCM simulationskill could be explained largely from the linear influ-ence of tropical SST variations. Furthermore, one de-gree of freedom in those tropical SSTs, namely ENSO,was the virtually exclusive interannual skill sourceoriginating from tropical oceans. In the case of U.S.seasonal precipitation, the univariate regression modelthat encapsulated the linear ENSO signal producedsimulation skill that exceeded the AGCM simulationskill, though field significance was confined to the pe-riod of late autumn–late spring. One interpretation forthe superior precipitation skill of the univariate modelis that although sensitivity patterns to non-ENSO SSTsexisted in the AGCMs, they were seemingly erroneousand served to obscure the true skill source related toENSO.

A new SST skill source, of apparent non-ENSO ori-gins, was discovered for U.S. surface air temperature.Although the AGCM’s winter–spring skill source wasagain accounted for by a univariate linear influence ofENSO, the AGCM skill significantly exceeded that ofeither the linear models during the period from latesummer through autumn. During these latter seasons,both the AGCMs’ U.S.-averaged correlation skill andthe spatial extent of its significant local correlationsgreatly exceeded those occurring in the reduced phasespace linear model simulations based on tropical SSTs

alone. Our investigation focused on the NDJ season ofmaximum difference in skills, and a coherent sourcerelated to SST variability over the subtropical west Pa-cific Ocean and the South China Sea was found.

That this empirical relation constituted a cause–effect link was confirmed through analysis of an addi-tional suite of AGCM experiments that used an ideal-ization of SST forcing confined to the far west Pacific/South China Sea region. Those runs yielded a patternof U.S. surface warming in response to warm SST forc-ing that reproduced the characteristic correlation pat-terns derived from both the globally forced multimodelensemble and the observational datasets. A linear bi-variate regression model using only indices of Niño-3.4and west Pacific/South China Sea SSTs explained vir-tually all of the AGCM simulation skill during the NDJseason.

Hindcast experiments for 1951–99 were generated inorder to determine what elements of skillfully simu-lated U.S. responses to known SST forcing could bepredicted. These employed the multivariate regressionmodel using the one-season-lag relationships betweentropical SST predictors and the AGCM’s U.S. seasonalclimate. These are analogous to zero-lead predictions,and estimate what the AGCM predictions would havebeen given a forecast for the tropical SSTs. Virtually allof the simulation skill was recovered in the hindcastexperiments for the seasons and the variables for whichENSO was the dominate source of simulation skill. Thehindcasts failed to recover the AGCM simulation skillof U.S. surface air temperature during late summer–autumn. It was demonstrated, however, that a one-sea-son-lagged bivariate model could recover the AGCM’shigh simulation skill during autumn. One conclusiondrawn from the hindcasts is that the principal SST

FIG. 9. The 1950–99 temporal correlation between the NDJ SST anomalies and the NDJ index ofsubtropical west Pacific SSTs. The contour interval is 0.1. See Fig. 7 for the averaging domain forconstructing the index. The irregular white contour in the west Pacific encloses a region of SST variationscoherent with the index region, and is used as the forcing region for idealized AGCM simulations.

1 JULY 2006 Q U A N E T A L . 3289

sources of U.S. predictability can themselves be skill-fully forecast at zero lead.

b. Discussion

Dynamical methods for U.S. climate prediction arenow achieving parity with empirical approaches (e.g.,Peng et al. 2000). The key question is whether suchequivalence of capability testifies to the limitations inskill of seasonal climate prediction, as suggested byAnderson et al. (1999), or whether it speaks to the in-fancy of dynamical models. Our diagnosis of the skillsources offers the following insight on such questions.

FIG. 10. Spatial distribution of the correlation between the NDJSST anomalies in the western Pacific Ocean and the multi-AGCM(top) ensemble mean and (middle) observed surface temperatureanomalies. The contour interval is 0.1. (bottom) The CCM3 re-sponse to the idealized SST anomalies in the western PacificOcean. The contour interval is 1.0.

FIG. 11. Same as Fig. 6, but for the NDJ simulation skill basedon (top) a univariate regression model using only an index ofsubtropical west Pacific (WP) SSTs as predictor, (middle) a bi-variate regression model using the WP, and Nino-3.4 indices aspredictors, and (bottom) the hindcast skill based on a lagged bi-variate model relating ASO Niño-3.4 and WP SST anomalies toNDJ surface temperature.


The fact that the linear response to ENSO SST vari-ability accounts for much of the AGCM skill over theUnited States—particularly in winter/spring for bothprecipitation and temperature—implies that a simpleregression model trained from observations shouldhave comparable skill to the dynamical model givensufficient data. We confirm this to be the case in Fig. 12where the bar graphs compare the hindcast skill of ourprior multivariate model with an identical model inwhich the U.S. predictand data are derived from obser-vations. Their 50-yr hindcast skills are indistinguishablebecause both are originating from the ENSO influencewhose response is realistically modeled in the AGCMs.If there are other tropical SST skill sources in nature,then they are evidently too small (or they occur tooinfrequently) to be detected and have material effectson the 50-yr-averaged skill (this does not discount thepossible skillful contribution of such additional tropicalsources in individual years). The preeminence of ENSOas a U.S. skill source is further verified by the result ofour singular value decomposition (SVD) analysis. Wefound that the singular value of the leading SVD modeis much larger than other SVD modes. Our result isconsistent with that predicted by DelSole and Chang(2003). Our study did identify an additional non-ENSOskill contributor, but as was the case with ENSO, alinear empirical model could be used to capitalize uponthat source too.

In this study, our analysis has been focused on themultimodel ensemble mean. Treating predictions fromdifferent models as a single ensemble increases the en-semble size and reduces sampling biases. Furthermore,the skill of the ensemble mean is found to be higherthan the skill of individual AGCMs included (figure not

shown). In operational practice, the skill of seasonalforecasts may be further raised by applying more so-phisticated multimodel combination strategies (e.g.,Goddard et al. 2003; Barnston et al. 2003). The sourcesof skill identified in our analysis are limited only tothose that can be captured by the multimodel ensemblemean. Additional sources may be identified when op-timal ensemble average methods are applied to maxi-mize the skill. It is also possible that the seasonal fore-cast skill can be further improved, and additional non-ENSO skill sources identified, when newer generationmodels are available.

The possibility that extratropical SST variations con-tribute to skill cannot be discounted based upon ouranalysis. It is possible that the near equivalence in simu-lation skill between the AGCMs and the statisticalmodels using only tropical SST predictors (e.g., in allseasons for U.S. precipitation, and in winter/spring forU.S. surface air temperature) is symptomatic of AGCMbiases, including the two-tier design of experiments,rather than a lack of predictive impact of nontropicalSSTs. An important direction for future investigation isassessing how, if at all, this current diagnosis of U.S.seasonal forecast skill related to SST influences is modi-fied when using the next generation of AGCMs. Also,a comparative analysis of the AGCM skill against thatof coupled ocean–atmosphere models is needed to spe-cifically address whether the predictability estimates,such as presented herein and in previous studies usingtwo-tiered systems, are applicable to fully coupledearth system models. Regarding the skill attributes ofsuch fully coupled models, it should also be noted thatinitialization of observed land surface conditions maybe sources for U.S. skill, in addition to ocean surfaceconditions. While the AGCM studied herein do employcoupled soil moisture models of various complexity,none of the runs used observed soil moisture condi-tions.

A further open question in seasonal climate predict-ability concerns the expected value for skill itself. Is ananalysis of skill drawn from 50 years of data a reliableestimate of the true expected value? To what degreedoes such a 50-yr analysis speak to the system’s pre-dictability in general, and what are error bars on thoseestimates owing to sampling alone? Little is knownabout the manner in which skill can vary over extendedtime periods. Insofar as ENSO has been affirmed to bethe primary source of skill, it is reasonable to suspectthat periods of low ENSO variance would lead to lowerU.S. climate predictability. At the same time, ENSOitself accounts for only a small fraction of U.S. climatevariability (e.g., Hoerling and Kumar 2002), and thuspurely atmospheric intrinsic variations could also im-

FIG. 12. The U.S. spatially averaged multivariate model hind-cast skill for 1951–99 for observed January–May-averaged U.S.precipitation and surface temperature using identical empiricalmodels except that the predictand data are derived from theAGCMs [MR(G)] in one case and from the observations in theother case [MR(O)].

1 JULY 2006 Q U A N E T A L . 3291

pact skill fluctuations. As a further diagnosis of thestatistical robustness of our skill, we have used 650 yr ofoutput from the integration of an unforced coupledocean–atmosphere model [NCAR’s Community Cli-mate System Model, version 2 (CCSM2; Kiehl andGent 2004)]. This model possesses ENSO-related vari-ability that is sufficiently realistic for our purposes in-sofar as the model’s ENSO signal over the UnitedStates explains roughly the same fraction of interannualvariance as does the observed ENSO signal (notshown). Another empirical multivariate model was de-veloped for one-season-lag relationships between tropi-cal SSTs and U.S. surface air temperature based on thefirst 150 yr of the coupled model run. Hindcasts werethen made for the subsequent 500 yr of model data.Figure 13 shows the probability densities of indepen-dent 50-yr samples of hindcast skill of U.S. surface tem-perature for the seasons JFM through MAM. The 50-yrmean variations range from a low correlation skill of 0.1to a high value of 0.3, with a median value near 0.2.These variations arise purely due to intrinsic coupledmodel noise. This result suggests that dynamical modelsare still needed to fully harvest the skill source of sea-sonal forecasts even if the skill of dynamical modelscomes from just one degree of freedom in the SST,because the skills of empirical methods trained on 50 yrof data may have large decadal swings.

In conclusion, it is apparent that a sound appraisal ofthe future prospects for U.S. seasonal forecast skillmust be found upon an understanding of the sources forskill itself. Here we have explored the ocean’s role, andidentified a single degree of freedom in SST variationsas providing the bulk of U.S. seasonal skill. An outlookfor future capabilities must also seek to understand thesources for multidecadal variations in skill, which ouranalysis indicates could occur simply due to low-

frequency variations in ENSO and other intrinsic fluc-tuations of the coupled ocean–atmosphere system. Ifmultidecadal variations in seasonal forecast skill aresufficiently large, then they could mask technologicaladvances in seasonal prediction methodologies.

Acknowledgments. The authors thank Dr. Arun Ku-mar for discussions and suggestions during the formingof this paper. Thanks are also due to two anonymousreviewers and their critical reviews. Their commentsand suggestions improved the content of this paper.This study is supported by funds from the NOAAOGP’s CLIVAR Program.

REFERENCES

Anderson, J., H. Van den Dool, A. Barnston, W. Chen, W. Stern,and J. Ploshay, 1999: Present-day capabilities of numericaland statistical models for atmospheric extratropical seasonalsimulation and prediction. Bull. Amer. Meteor. Soc., 80,1349–1361.

Barnett, T. P., and R. W. Preisendorfer, 1987: Origins and levelsof monthly and seasonal forecast skill for North Americansurface temperature determined by canonical correlationanalysis. Mon. Wea. Rev., 115, 1825–1850.

Barnston, A. G., 1994: Linear statistical short-term climate pre-dictive skill in the Northern Hemisphere. J. Climate, 7, 1513–1564.

——, and T. M. Smith, 1996: Specification and prediction of globalsurface temperature and precipitation from global SST usingCCA. J. Climate, 9, 2660–2697.

——, S. J. Mason, L. Goddard, D. G. Dewitt, and S. E. Zebiak,2003: Multimodel ensembling in seasonal climate forecastingat IRI. Bull. Amer. Meteor. Soc., 84, 1783–1796.

Barsugli, J. J., and P. D. Sardeshmukh, 2002: Global atmosphericsensitivity to tropical SST anomalies throughout the Indo-Pacific basin. J. Climate, 15, 3427–3442.

Broccoli, A. J., and S. Manabe, 1992: The effects of orography onmidlatitude Northern Hemisphere dry climates. J. Climate, 5,1181–1201.

Chen, P., and M. Newman, 1998: Rossby wave propagation andthe rapid development of upper-level anomalous anticy-clones during the 1988 U.S. drought. J. Climate, 11, 2491–2504.

DelSole, T., and P. Chang, 2003: Predictable component analysis,canonical correlation analysis, and autoregressive models. J.Atmos. Sci., 60, 409–416.

Geisler, J. E., M. L. Blackmon, G. T. Bates, and S. Muñoz, 1985:Sensitivity of January climate response to the magnitude andposition of equatorial Pacific sea surface temperature anoma-lies. J. Atmos. Sci., 42, 1037–1049.

Goddard, L., and S. J. Mason, 2002: Sensitivity of seasonal climateforecasts to persisted SST anomalies. Climate Dyn., 19, 619–631.

——, A. G. Barnston, and S. J. Mason, 2003: Evaluation of theIRI’s “net assessment” seasonal climate forecasts 1997–2001.Bull. Amer. Meteor. Soc., 84, 1761–1780.

Held, I. M., S. W. Lyons, and S. Nigam, 1989: Transients and theextratropical response to El Niño. J. Atmos. Sci., 46, 163–174.

Higgins, R. W., H.-K. Kim, and D. Unger, 2004: Long-lead sea-

FIG. 13. Probability distribution of the 50-yr U.S. surface tem-perature skill obtained from the 500-yr simulation of the NCARcoupled climate system model.


sonal temperature and precipitation prediction using tropicalPacific SST consolidation forecasts. J. Climate, 17, 3398–3414.

Hoerling, M. P., and A. Kumar, 2002: Atmospheric response pat-terns associated with tropical forcing. J. Climate, 15, 2184–2203.

——, and ——, 2003: The perfect ocean for drought. Science, 299,691–694.

Hoskins, B. J., and D. J. Karoly, 1981: The steady linear responseof a spherical atmosphere to thermal and orographic forcing.J. Atmos. Sci., 38, 1179–1196.

Kiehl, J. T., and P. R. Gent, 2004: The Community Climate Sys-tem Model, version 2. J. Climate, 17, 3666–3682.

——, J. J. Hack, G. B. Bonan, B. A. Boville, D. L. Williamson,and P. J. Rasch, 1998: The National Center for AtmosphericResearch Community Climate Model: CCM3. J. Climate, 11,1131–1149.

Kiladis, G. N., and H. F. Diaz, 1989: Global climatic anomaliesassociated with extremes in the Southern Oscillation. J. Cli-mate, 2, 1069–1090.

Kumar, A., M. Hoerling, M. Ji, A. Leetmaa, and P. Sardeshmukh,1996: Assessing a GCM’s suitability for making seasonal pre-dictions. J. Climate, 9, 115–129.

——, W. Wang, M. P. Hoerling, A. Leetma, and M. Ji, 2001: Thesustained North American warming of 1997 and 1998. J. Cli-mate, 14, 345–353.

Livezey, R. E., and W. Y. Chen, 1983: Statistical field significanceand its determination by Monte Carlo techniques. Mon. Wea.Rev., 111, 46–59.

——, and K. C. Mo, 1987: Tropical–extratropical teleconnectionsduring the Northern Hemisphere winter. Part II: Relation-ships between the monthly mean Northern Hemisphere cir-

culation patterns and proxies for tropical convection. Mon.Wea. Rev., 115, 3115–3132.

North, G. R., T. L. Bell, R. F. Cahalan, and F. J. Moeng, 1982:Sampling errors in the estimation of empirical orthogonalfunctions. Mon. Wea. Rev., 110, 699–706.

Palmer, T. N., and J. A. Owen, 1986: A possible relationship be-tween some “severe” winters in North America and en-hanced convective activity over the tropical west Pacific.Mon. Wea. Rev., 114, 648–651.

Pegion, P., S. Schubert, and M. J. Suarez, 2000: An assessment ofthe predictability of northern winter seasonal means with theNSIPP1 AGCM. NASA Tech. Memo. 2000-104606, Vol. 18,100 pp.

Peng, P., A. Kumer, A. G. Barnston, and L. Goddard, 2000: Simu-lation skills of the SST-forced global climate variability of theNCEP-MRF9 and the Scrippts-MPI ECHAM3 models. J.Climate, 13, 3657–3679.

Rayner, N. A., D. E. Parker, E. B. Horton, C. K. Folland, L. V.Alexander, D. P. Rowell, E. C. Kent, and A. Kaplan, 2003:Global analyses of sea surface temperature, sea ice, and nightmarine air temperature since the late nineteenth century. J.Geophys. Res., 108, 4407, doi:10.1029/2002JD002670.

Roeckner, E., and Coauthors, 1992: Simulation of the present-dayclimate with the ECHAM model: Impact of model physicsand resolution. Max-Plank-Institut für Meteorologie Rep. 93,171 pp.

Ropelewski, C. F., and M. S. Halpert, 1989: Precipitation patternsassociated with the high index phase of the Southern Oscil-lation. J. Climate, 2, 268–284.

Ting, M. F., and L. H. Yu, 1998: Steady response to tropical heat-ing in wavy linear and nonlinear baroclinic models. J. Atmos.Sci., 55, 3565–3582.

1 JULY 2006 Q U A N E T A L . 3293

diagnosing sources of u.s. seasonal forecast skill · diagnosing sources of u.s. seasonal forecast...

Documents