Validation of the FSU/COAPS Validation of the FSU/COAPS Climate ModelClimate Model

Mary Beth Engelman

[T. LaRow, D.W. Shin, S.Cocke, and M. Griffin]

CDPW Meeting – Tallahassee, FL

October 22, 2007

Outline:Outline:

I. Introduction to work

II. Background (Previous Studies)

III. Data

IV. Methodology

V. Results

VI. Summary/Conclusions

VII.Future Work

Introduction: The studyIntroduction: The study

• What I’m Doing…

– Scientific Objective:• Validating the FSU/COAPS Climate Model

– Variables:

» Surface Temperature (Maximum and Minimum)

» Precipitation

– Comparing:

» Model forecast vs. Observations

» Real-time runs (persisted SSTA) vs. Hindcast runs (observed weekly SSTA)

– Scientific Questions:1. How “well” does the FSU/COAPS climate model forecast

surface temperature and precipitation over the southeast United States?

2. How do forecasts made from persisted SSTA compare to hindcasts made from weekly updated prescribed SSTA?

Introduction: Reasons for studyIntroduction: Reasons for study

•And Why…

– Benefit to society:• Crop models can assist decision makers in minimizing loses and maximizing profits

in the agricultural community (Hansen et al., 1997)

– Benefit to the science:• Evaluation of the need for a coupled ocean-atmosphere climate model

• Validation potentially leads to improved climate models/forecasts

Previous Studies: Persisted vs. prescribed SSTAPrevious Studies: Persisted vs. prescribed SSTA

– Why use persisted anomalies for climate forecasts?• Currently, coupled models do not out perform predictions given by persisted SSTA at lead

times less than 3 months (exception: ENSO events)

– Why compare real-time runs to hindcasts runs?• Overestimates skill, but evaluates model’s possible potential

• Determines how much skill is lost from using persisted SSTA

– Performance of real-time vs. hindcasts runs?• Similar values of skill, unless large SSTA errors

(Goddard and Mason, 2002)

Previous Studies: SST and the southeast climatePrevious Studies: SST and the southeast climate

• SOI and Southeast United States El Nino Signals (Ropelewski and Halpert, 1986)

– Consistent signals:

• Above normal precipitation (October-March)

• Below normal temperature

– Physical mechanisms causing precipitation and temperature signals (teleconnections vs. direct link)

• Tropical Pacific SSTA and ENSO Signals (Montroy, 1997)

– Strongest signals: Precipitation in the Southeast US• Positive: November - March • Negative: July-August

• Teleconnections (Gershunov and Barnett, 1997)

– Strong teleconnection between tropical Pacific SSTs and wintertime precipitation over the coastal southeast US (negatively correlated in the interior states)

Previous Studies: The modelPrevious Studies: The model

The FSU/COAPS Climate Model:

• Large Scale Features:

– Capable of reproducing large scale features associated with ENSO (Cocke and LaRow, 2000)

– Capable of reproducing teleconnections response (Cocke et al., 2007)

• Temperature and Precipitation:

– Regional and global model are both in “reasonable” agreement with observations (Cocke and LaRow, 2000)

– Alterations to the physics of the model resulted in a strong wet bias and cold surface temperature bias, upgrading the model (inclusion of the NCAR CLM2 as land parameterization) reduced biases (Shin et al., 2004)

– Regional model rainfall events: • Over predicts small rainfall events

• “Accurately” produces higher magnitude rainfall events (Cocke et al., 2007)

– “Seasonal variations, anomalies, and changes in extreme daily events for maximum temperature are successfully forecasted” (Lim et al., 2007)

Previous Studies: SummaryPrevious Studies: Summary

•What to expect in my study (Hypothesis):

– Model Forecast vs. Observations:• Model forecasts will show a wet bias in precipitation and cold bias in maximum surface

temperature (unless bias corrected)

• Forecasts should show “reasonable” skill (anomalies, seasonal variations, various skill scores)

• Forecasts will tend to over predict the number of small rainfall events

– Real-Time Runs vs. Hindcast Runs (Persisted vs. Prescribed SSTA )• In general, hindcast runs will out perform real-time runs

• Real-time runs should be comparable to hindcasts runs when equatorial Pacific SSTA remain fairly constant throughout the duration of the forecast

• Loss of skill in the real-time runs will occur– When equatorial Pacific SSTA take a sharp change– Where areas are greatly influenced by SSTs (high teleconnection signal)

• Skill of real-time runs will decrease after 3 months

Data: Model Data: Model

•Details of model (Spectral Model):

– Resolution:• GSM: T63 (1.875˚ x 1.875˚)• NRSM: 20 x 20 km

– Bias: • Precipitation: wet• Max Temperature: cold

•Real time run:

– Persisted SSTA– Model Runs (5 ensembles):

• January 2005 and 2006 (3 month forecasts)

•Hindcast runs:

– Prescribed SSTA (updated weekly)– Model Runs (5 ensembles):

• October 2004 and 2005 (6 month forecasts)

– Model Climatology (20 ensembles):• 1987-2005

FSU/COAPS Climate Model

Model Details

Data: ObservationsData: Observations

COOP Observation Network:

• Daily/Monthly Temperature and Precipitation (January 1986-June 2007)

• Station to grid:

– 172 long term stations (1948-present)

– For each grid point…

• Finds the closest stations (using the great circle distance)

• Assigns the station’s temperature and precipitation to that grid

MethodologyMethodology

•In General:

– Looking for the ability to forecast:

• Correct anomaly• Extreme events

– Heavy Precipitation/Drought– Freezing Temperatures

•Today’s Talk (Temperature): ability to forecast:

– 1. Average monthly temperature anomaly

– 2. Max temperature distributions (bias corrected)*

– 3. Freeze events (bias corrected)*

*bias corrected:

where:

yclimatolog observed

oyclimatologmodel

etemperaturforecasted

o

m

f

T

T

T

Comparing JFM 05 vs JFM 06 SSTAComparing JFM 05 vs JFM 06 SSTA

2005 2006

2005: SSTA switch

from warm to cool

2006: SSTA remain

fairly constant

(figures provided by: http://www.pmel.noaa.gov/tao/jsdisplay/ )

Results: Ability to forecast anomaliesResults: Ability to forecast anomalies

Real-Time Hindcast Real-Time Hindcast

Jan.

Feb.

Mar.

2005 2006

Error in Monthly Temperature Anomalies (˚C):

Initial time: Jan. 05 Oct. 05 Jan. 06 Oct. 06

Error in Temperature

(ε = f ` - o`)

•2006 real-time runs

resulted in less error than

the 2005 real-time runs

•2005 and 2006 hindcasts

runs showed

approximately the same

magnitude of error

•In 2006 the real-time runs

and hindcast runs resulted

in “similar” forecast error

|ε| = 4.28

|ε| = 2.50

|ε| = 0.77|ε| = 3.14

|ε| = 4.86

|ε| = 2.72

|ε| = 0.87

|ε| = 1.96 |ε| = 2.79

|ε| = 0.95

|ε| = 1.40|ε| = 2.65

Results: Ability to forecast distributionsResults: Ability to forecast distributions

Daily Maximum Temperatures

(January-March)

•Observed distributions more

accurately captured by the

models in 2006

• Hindcasts vs. real-time runs

differ more in 2005 than in 2006

•In both years, model over

estimates tails of the distribution

Results: Ability to forecast extreme eventsResults: Ability to forecast extreme events

Deep Freeze Events

(January – March)

• Model tends to over predicts

number of freeze events

• Real-time and hindcasts

runs are more similar in 2006

Observed

Real-Time

Hindcast

2005 200620052006

Tmin < -2˚C Tmin < -4˚C

Summary/Conclusions:Summary/Conclusions:

– Model forecasts show a cold bias in surface temperature

– The model tends to over predict “extreme” events

– Real-time runs resembled hindcasts runs when equatorial Pacific

SSTA remain fairly constant throughout the duration of the forecast (as

in 2006)

– When SSTA changed during the time period of the forecast (as in

2005), hindcasts runs outperform real-time runs even at lead times of 3

months

Future Work:Future Work:

– Evaluate Real-Time October runs:• Currently comparing Januray real-time runs (0 month lead time) to October

hindcast runs (3 month lead time)

– Examine other verification methods/skill scores:• Anomaly correlations• Taylor diagrams• Relative Operating Characteristics (ROC)• Heidke Skill Score (HSS)

– Further investigate SSTA in the tropical Pacific and their effects on model forecast errors

Questions?Questions?

References:References:

Cocke, S., and LaRow, T.E., 2000: Seasonal predictions using a regional spectral model embedded within a coupled ocean-atmosphere

model. Mon. Wea. Rev., 128, 689-708.

Goddard, L, S.J. Mason, S.E. Zebiak, C.F. Ropelewski, R. Basher, and M.A. Cane. 2001. Current approaches to seasonal-to-interannual

climate predictions. International Journal of Climatology. 21: 1111-1152. DOI: 10.1002/joc.636

Goddard L, Mason SJ, 2002. Sensitivity of seasonal climate forecasts to persisted SST anomalies. Climate Dynamics 19: 619–631

Gershunov, A., 1998: ENSO influence on intraseasonal extreme rainfall and temperature frequencies in the contiguous United

States:Implications for long-range predictability. J. Climate, 11, 3192–3203..

Jolliffe, I.T., and D.B. Stephenson, 2003. Forecast Verification. Wiley, 240 pp.

Lim, Y.K., Shin, D.W., Cocke, S., LaRow, T.E., Schoof, J.T., O’Brien, J.J, and Chassignet, E., 2007: Dynamically and statistically

downscaled seasonal forecasts of maximum surface air temperature over the southeast United States.

Montroy, D., 1997: Linear relation of central and eastern North American precipitation to tropical Pacific sea surface temperature anomalies.

J. Climate, 10, 541–558..

Murphy, A.H., 1993. What is a good forecast? An essay on the nature of goodness in weather forecasting. Weather and Forecasting, 8,

281-293.

Ropelewski, C. F., and M. S. Halpert, 1986: North American precipitation and temperature patterns associated with the El Niño/Southern

Oscillation (ENSO). Mon. Wea. Rev., 114, 2352–2362.

Shin, D.W., S. Cocke, T. E. LaRow, and J. J. O’Brien (2005), Seasonal surface air temperature and precipitation in the FSU climate model

coupled to the CLM2, J. Clim., 18, 3217-3228.

Wilks, D.S., 1995. Statistical Methods in the Atmospheric Sciences. An Introduction. Sans Diego: Academic Press.

Wilks, D.S. and R.L. Wilby. 1999. The weather generation game: A review of stochastic weather models. Progress in Physical Geography

23:329-357.