S2D Prediction Activities at CCCma(p)

HFP2 (2-tier seasonal) basis of CMC operational forecasts Analyses Trend in SIP forecasts

CHFP1 and 2 (1-tier seasonal/interannual) initial approach and results analyses and initialization development path

DHFP (decadal) potential predictability results WGCM/WGSIP/CMIP5 and decadal prediction

Retrospective forecasts are crucial for - establishing forecast skill - providing forecast climatology for bias correction - guiding forecast calibration and post-processing

Why retrospective forecasts?

Current EC 2-tier operational system: - 4 AGCMs x 10-ensemble - validated by 2nd Historical Forecast Project (HFP2) - 4-month retrospective forecasts initialized each month 1969-2003

Validate also 1-tier coupled forecasts in CHFP

HFP2 CMC deterministic and probabilistic

seasonal forecasts are based on HFP2 HFP2 (2nd Historical Forecasting Project)

retrospective 1-season forecast experiment operational context – no information from the

future 2-tier - SST forecast provides boundary conditions

for AGCM forecasts multi-model, multi-realization – each of 4 models

produces 10-member ensemble forecasts we use results from 33-year period 1970-2002

Seasonal forecastsJan-Feb-Mar 2009

DeterministicProbabilistic

Skill assessment of HFP2 forecasts analysis by S. Kharin et al. (2008) deterministic forecasts

continuous valued categorical combining multi-model ensemble mean forecast

information unweighted variance weighted (2 methods) regression improved

scale unweighted ensemble mean (single parameter) scale each model mean (four parameter)

skill measures: correlation and MSSS probabilistic categorical

combining multi-model forecast information count value gaussian adjusted gaussian

skill measures: Brier skill score benefits of Multi-model approach

Correlation and MSSS for Sfc air Tmethods of combiningmulti-model forecasts

1 parameter regression weighting4 parameter regression weighting

Percent correct, 3-category forecasts

Seasonal variation of skillNo dependence on combination method

3-category probabilistic forecasts

Count methodsGaussian methods

Normal

Below

Adjustedgaussian

Brier skill score

Reliability

The virtues of the MME approach

For same overallensemble sizemore models give better scores

More modelsare better

Diminishingreturns

Four models isa “practical”choice

Summary

Deterministic forecasts not critical how MME formed scaled MME result (1 parameter) increases MSSS but

degrades correlation scaled MME result (4 parameters) degrades scores

Probabilistic forecasts gaussian method better than count adjusted gaussian only better over ocean (on account of

persisted SSTAs) Multi-model methods

for same ensemble size more models the better because of diminishing returns, 4 models (as HFP2) is

reasonable “practical” choice

Climate trends and seasonal forecasts

trend in HFP2 no GHG or other forcing in HFP2 implicit in persisted SSTAs and initial

conditions characterization of trend statistical correction

trend in CHFP1 GHG forcing effect in coupled forecast model

Boer (2008)

Background

Observational studies show long timescale trends thought to be due to GHG and other forcing - together with long timescale variability

Externally forced trends should provide an additional seasonal forecasting signal - if present and properly forecast

Do we capture these trends in the 33-years of retrospective forecast data of HFP2 (and does it matter)?

Linear trends in <T850> for DJF

For global means

<T> = <a> + <T’> - fit linear trends by least squares

forecast trend weaker than trend in NCEP reanalysis data

we also fit trends to each point

Forecast

NCEP

years

Units: oC

-17 17

Trends in SAT from NCEP and HFP2

DJFNCEP

DJFHFP2

JJANCEP

JJAHFP2

Units oC/decade

Trends in Z500 from NCEP and HFP2

DJFNCEP

DJFHFP2

JJANCEP

JJAHFP2

Units m/decade

Trends

trends are larger at higher latitudes, over land and in winter – the signature of global warming

trends in forecasts are weaker than in reanalysis data

suggests that lack of GHG forcing may degrade the forecasts

spatial average of MSE over globe

3 components error in trend error in non-trend component cross-product

Time evolution of error

Dealing with anomalies minimizes MSE in DJF T850

Anomalies

Trends setto zero at t=0

global means MSE over globe

trend

trend

total

total

non-trend

non-trend

cross-product

NCEP

HFP2

Mean error

Mean error

Mean square error

Mean square error

Can we improve skill by adjusting trend?

try simple statistical adjustments based on scaling or trend adjustment

only cases where single parameter is estimated to avoid over-fitting

evaluate in cross-validation mode use MSSS and Correlation scores

Statistical adjustments

Scaled

Trendadded

Trendreplaced

Rawforecast

Trendreplaced

Rawforecast

Scaled

Trendadded

Trendreplaced

Rawforecast

Scaled

Trendadded

Global

Dec Jan Feb Mar DJF JFM

0.10

0.15

0.20

0.25

0.30

0.35

0.40

BP01WS01GHGWS01GHGA

Land

Dec Jan Feb Mar DJF JFM

0.05

0.10

0.15

0.20

0.25

0.30

Ocean

Dec Jan Feb Mar DJF JFM

0.10

0.15

0.20

0.25

0.30

0.35

0.40

0.45

No radiative forcing trend in modelWeak radiative forcing trendStronger radiative forcing trend

Temperature at 850hPa (T850) Anomaly Correlation1 December Initialization

Preliminary evidence of impact of radiative forcing trend on coupled model forecast skill

Forcing can affect both land and ocean in this case

Summary Observed/reanalysis trends have GHG signature Trends in seasonal forecasts are weak Statistically “correcting trends” gives some

improvement in skill over Asia but not, unfortunately, over North America

Suggests seasonal forecasting models should include radiative forcing due to GHG and aerosols - physically realistic (if properly inserted)- some hope for improved forecasts

For coupled CHFP forecasts SSTs may also drift and we have some evidence GHG forcing is needed

The Coupled Historical Forecast Project, version 1: Formulation, results, and progress towards CHFP2

Bill Merryfield, George Boer, Greg Flato , Slava Kharin , Woo-Sung Lee , Badal Pal , John Scinocca

Canadian Centre for Climate Modelling and AnalysisEnvironment Canada

Pilot Project: CHFP1

Based on CGCM3.1/T63 (IPCC AR4)

Simple SST nudging initialization after Keenlyside et al. (Tellus 2005):

- Strongly relax SST to observed 1970-2001 time series

- Anomalous wind stress tends to set up correct equatorial thermocline configuration:

AsiaAsia South South AmericaAmerica

warmcool

Anomalous wind stress

1 Sep assimilation run

31302928272625

AGCM

OGCM

AugForecast 1

Forecast 2

Forecast 3

Forecast 10

…

12 mos

forecast runs

lead 0 lead 1 lead 2

1 Aug1 Jul1 Jun

10 combinations of ACGM + OGCM initial conditions

1 Oct1 May

Construct 10 initial conditions for 1 Sep (e.g.) by combining atm and ocn states from preceding week:

Launch forecasts 1 Feb, 1June, 1 Sep, 1 Dec 1971-2000 (10 ensemble members) x (4 initializations yr-1) x 30 yrs 1200 years of coupled model integration

CHFP1 Ensemble Generation

CHFP1

Persistence

Damped persistence

CHFP1 Results (Ensemble size=10)

Anomaly correlation Mean square skill score

NINO3.4 skills

• Not too bad for off-the shelf model, crude initialization• Much room for improvement, improvement being realized…

Coupled Forecast System Development Path

CHFP2

“Off the shelf” CGCM

Simple SST nudging

initialization

Ocean data assimilation

CHFP1

CGCM development

2D Var after Tang

et al. (JGR 2004)Improved error covariances

Atmospheric initializationInsertion of reanalyses Atm data assimilation

Land initialization

GHG forcing,

trend correction, etc

Off-line forcing by bias-correctedreanalysis

S assim after Troccoli et al. (MWR 2002)

AGCM, OGCM

Analysis and Verification

Obs SSTA Nov 1982 Deterministic forecast SSTA Nov 1982New AGCM, OGCM Lead=11 mo

• Much improved model ENSO with new AGCM, OGCM• Exemplified by “hit” for 11-month lead prediction of 1982/83 El Nino:

NINO3 Power Spectrum

CGCM4

CGCM3.8

CGCM3.1

OBS1970-99

OBS1940-69

PERIOD (Y)100 10 1 10-1

Ocean Data Assimilation

Initially use approach of Tang et al. (JGR 2004): - input ocean reanalysis in lieu of observations - simple variational assimilation level-by-level (2D

Var) - background error covariances of Derber &

Rosati (JPO, 1989)

Assimilate multiple ocean analyses

Explore methods to improve error covariances

MULTI-ANALYSIS EXP_ATMOS EXP_OCEAN

Ensemble member 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6

Atmosphere Initial State

8/318/31

8/30

8/29

8/28

8/27

8/26

8/31

Ocean Initial state

8/31 8/318/31

8/30

8/29

8/28

8/27

8/26

Used Reanalysis Data for ocean assimilation

GODAS ECMWF GFDL SODA INGV METUK GODAS GODAS

Ocean Initialization by multi-analysis assimilation

Experiment: compare NINO3.4 skill and ensemble spread for three

ensemble initialization strategies:

- Multi-analysis: off-line assimilation of 6 ocean analysis products (same atm)

- Exp_atmos: 6 AGCM states from consecutive days prior to forecast start (same ocn)

- Exp_ocean: 6 OGCM states from consecutive days prior to forecast start (same ocn)

1980-2001: 22 years of Sep 1–initialized forecasts

Lead Month

0 1 2 3 4 5 6 7 8 9 10 11

-0.2

0.0

0.2

0.4

0.6

0.8

1.0

multi_analysisexp_atmosexp_ocean

Cor

rela

tion

multi-reanalysisexp_atmosexp_ocean

0 1 2 3 4 5 6 7 8 9 10 11

0.0

0.2

0.4

0.6

0.8

1.0

Lead month

Ensemble Spread RMS Error

NINO3.4 skill and ensemble spread

Lead month

Lines : Ensemble mean

Symbols : Ensemble members

Improved skill at longer leads Larger ensemble spread in first two months

SST Forecast Skill

Multi-analysis ocean initialization leads to

Atmospheric Initialization

SST nudging informs AGCM of boundary forcing, but not correct synoptic configuration, i.e. weather

major loss of skill in first month of forecast

Two approaches are being pursued:

- Direct insertion of atmospheric analysis (cf. HFP2)

- Simple assimilation of analysis into AGCM

Land Initialization• CFCAS/GOAPP funded collaboration with A. Berg (Guelph)

• Force land surface model with bias-corrected reanalyses

after Berg et al. (Int J Clim 2005)

.95

.85

.75

.65

.55

>.5

Berg et al., 2003: 2005

Correlation of NCEP monthly precip with gauge-based measurements in USA:

before bias correction after bias correction

Coupled forecasts offer means for seasonal forecasting at longer leads, where future evolution of SSTA is critical

Prototype CHFP1 competitive with 4-model HFP2 at 1-month lead, but has only simplest initialization

CHFP1 provides a benchmark against which model and initialization improvements leading to CHFP2 can be assessed

CCCma participation in international CHFP through CLIVAR/WGSIP

Summary

Many scientific “opportunities”

improved models analysis of variability and of modes of variability improved analysis methods especially in the

ocean for model initialization for verification for model development

ensembles ensemble generation multi-model ensembles

prediction studies and the “DHFP” WGSIP/WGCM/CMIP5 coordinated project

Coupled Forecast System Development Path

CHFP2DHFP1

“Off the shelf” CGCM

Simple SST nudging

initialization

Ocean data assimilation

CHFP1

CGCM development

2D Var after Tang

et al. (JGR 2004)Improved error covariances

Atmospheric initializationInsertion of reanalyses Atm data assimilation

Land initialization

GHG forcing,

trend correction, etc

Off-line forcing by bias-correctedreanalysis

S assim after Troccoli et al. (MWR 2002)

AGCM, OGCM

Analysis and Verification

End of presentation