observation system simulation experiments (osse) · pdf filefeb 14 2012 emc seminar wind lidar...

ECMWF Lars Isaksen, ECMWF 28th Nordic Meteorologists´ Meeting , June 2012 1

Observation System Simulation Experiments (OSSE) What is the experience of the meteorological NWP

community?

Lars Isaksen

Head of Data Assimilation Section at ECMWF [email protected]

Acknowledgements to a large number of OSSE scientists will be given on the slides

2

Internationally Collaborative Joint

OSSEs Progress At NOAA

Michiko Masutani[1,2,#], Lars Peter Riishojgaard [2,$], Zaizhong Ma[2,$],

Jack S. Woollen[1,+], Dave Emmitt[5], Sid Wood[5], Steve Greco[5],

Tong Zhu[3,@], Yuanfu Xie[4]

[1]NOAA/National Centers for Environmental Prediction (NCEP)

[2]Joint Center for Satellite and Data Assimilation (JCSDA)

[3]NOAA/ NESDIS/STAR,

[4]NOAA/Earth System Research Laboratory (ESRL)

[5]Simpson Weather Associates

# Wyle Information Systems, McLean, VA,

+IM Systems Group)IMSG), MD

$Earth System Science Interdisciplinary Center, Univ. of Maryland, College Park,,

@Cooperative Institute for Research in the Atmosphere (CIRA)/CSU, CO

OSSE:Observing Systems Simulation Experiments

http://www.emc.ncep.noaa.gov/research/JointOSSEs/

Acknowledgement. This presentation is based on material from:

Feb 14 2012 EMC Seminar Wind Lidar OSSEs at JCSDA

Observing System Simulation Experiments in the Joint Center for Satellite Data Assimilation

3

Lars Peter Riishojgaard1,2, Zaizhong Ma1,2,

Michiko Masutani3, Jack Woollen3,

Dave Emmitt4, Sid Wood4, Steve Greco4

1Joint Center for Satellite Data Assimilation 2University of Maryland College Park 3NCEP Environmental Modeling Center 4Simpson Weather Associates

Acknowledgement. This presentation is based on material from:

ECMWF The 5th WMO Symposium on Data Assimilation, Melbourne Oct 2009

Reference Result Verification NWP-System Observations


Observing System Experiment (OSE)

OSSE

Real atmosphere

Assimilation/ forecast


Compare to reference


Impact assessment

Nature run





Assimilation/ forecast Compare to reference

Calibrate

Impact assessment

Ack: Werner Wergen (DWD)


Data assimilation system(s) NCEP/EMC Global Forecast System (T-382) coupled with GSI

Nature run ECMWF T-511, commissioned for Joint OSSE

13 month period (May 2005-June 2006)

Validated extensively through Joint OSSE collaboration

Simulated observations Reference observations, including satellite radiance data (all observation)

Perturbation (“candidate”) observations

GWOS observations simulated by Simpson Weather Associates

Diagnostics capability NCEP operational verification system

5

Observing System Simulation Experiments

Data impact on analysis and forecast will be evaluated.

A Full OSSE can provide detailed quantitative evaluations of

the configuration of observing systems.

A Full OSSE can use an existing operational system and help

the development of an operational system

Advantages

Existing Data assimilation system

and verification method are used for

Full OSSEs. This will help

development of DAS and verification

tools.

OSSE Calibration

A Nature Run (NR, proxy true atmosphere) is produced from a

free forecast run using the highest resolution operational model

which is significantly different from the NWP model used in Data

Assimilation Systems.

Calibrations is performed to provide quantitative data impact

assessment.

Without calibration quantitative evaluation of data impact is not

possible.

Full OSSEs

There are many types of simulation experiments. Sometimes, we have to call

our OSSE a ‘Full OSSE’ to avoid confusion.

• Full OSSEs are expensive – Sharing one Nature Run and simulated observation saves costs – Sharing diverse resources

• OSSE-based decisions have international stakeholders – Decisions on major space systems have important scientific, technical, financial and political ramifications – Community ownership and oversight of OSSE capability is important for maintaining credibility

• Independent but related data assimilation systems allow us to test the robustness of answers

International Joint OSSE capability

Calibration of OSSEs verifies the simulated data impact by comparing it to real data impact. In order to conduct an OSSE calibration, the data impact of existing instruments has to be compared to their impact in the OSSE.

6


Role of a National OSSE Capability

Impact assessment for future missions Future operational observing systems (NOAA)

Decadal Survey and other science and/or technology demonstration missions (NASA)

Other agencies (e.g. DoD/DWSS)

Objective way of establishing scientifically sound and technically realistic user requirements

Tool for assessing performance impact of engineering decisions made throughout the development phases of a space program or system

Preparation/early learning pre-launch tool for assimilation users of data from new sensors

7


Joint OSSE history

NASA/NOAA collaboration started in 2007, involving NASA/GSFC, NOAA/NESDIS, NOAA/NWS, NOAA/OAR

Centered around common use of Nature Run provided free of charge by ECMWF

Coordinated through JCSDA

Informal, loosely structured nature, lack of common funding stream has presented challenges

Successful joint validation of ECMWF Nature Run

Some collaboration on simulation and calibration of observations

ADM experiments (GMAO)

GWOS experiments (JCSDA)

UAS experiments (OAR)

8


The realism of the forecast model is a very important for the quality of the Nature Run

Physical processes in the ECMWF model


The resolution of the forecast model is a very

important for the quality of the Nature Run


Checking the realism of the Nature Run is vital. Atlantic Hurricane in the nature run for the analysis period of 25 Sep to 10 Oct 2005

11

Based on figure produced by Joe Terry

Fig.1Average 500-hPa geopotential height anomaly correlation as a function of forecast range for the Northern (a) and Southern (b) Hemisphere. Tropical wind vector RMS errors (m/s) at 200 hPa (c) and 850 hPa (d) as a function of forecast range. CTRL shown in black, NOUV in red. All observations used were real. Lower plot of each panel shows difference between NOUV and CRTL with, error bars indicating differences that are significant at the 95% confidence level.

Fig. 2: As Figure 1, except that all observations were simulated based on T511NR.

Calibration experiments and DWL OSSE at JCSDA

Calibration and initial evaluation of DWL impact were conducted for the period 1st Jyly-15 August. Calibration experiments showed reasonable agreement in data impact of RAOB wind in real and simulated impact. (Fig.1 and Fig.2)

12


Simulated observation for Control experiments

- Entire Nature run Period - Michiko Masutani and Jack Woollen (NOAA/NCEP/EMC)

Simulated radiance data, Only Clear Sky radiance are posted (status Feb 2012) (Cloudy radiance are also simulated. Radiance with mask based on GSI usage is also simulated. But these data are not posted.)

BUFR format for entire Nature run period Type of radiance data and location used for reanalysis from May 2005-May2006 Simulated using CRTM1.2.2 No observational error added

Conventional data

Entire Nature run Period

Restricted data removed

Cloud track wind is based on real observation location

No observational error added

13


Simulation of radiance data at NOAA

Jack Woollen and Michiko Masutani

GOES and SBUV are simulated as they are missing from the GMAO

dataset.

PrepBUFR is simulated based on CDAS distribution and quality

controls.

Radiance data for instruments used in 2005-2006 are generated at

the foot print used by the NCEP reanalysis.

Simulation was conducted using CRTM 1.2.2.

Some calibration and validation will be conducted by NCEP and NESDIS.

Step 1. Thinning of radiance data based on real use

14

Step 2. Simulation of radiance data using cloudy radiance


Nature Run (grib1 reduced Gaussian) 91 level 3-D data (12 Variables) 2-D data (71 Variables) Climatological data

Observation template Geometry Location Mask

Decoding grib1 Horizontal Interpolation

DBL91

Running Simulation program (RTM)

Post Processing (Add mask for channel, Packing to BUFR)

Simulated Radiance Data

Need lots of cpu’s Need Radiation Experts

Need complete NR (3.5TB) Random access to grib1 data

Need Data Experts

Need Data Experts but this will be a small program

15


Th

Fig. 1 NOAA -15 AMSU-A Channel 1 brightness temperature at GSI analysis time 0000 UTC May 2, 2005, time window 6 hours from (left) observation, (right)) CRTM simulation with NR atmospheric profiles.

16

Evaluation of simulated GOES and AMSUA

at the 1st step (12hr fcst) of the Nature Run

simulated with 2005 template

Tong Zhu (NESDIS)

Observation and simulation of GOES-12

Sounder 18 IR channels at over North

Atlantic region at 1200 UTC October 1,

2005.

Tong Zhu (NESDIS)

Ob

se

rve

d

Sim

ula

ted

Ob

se

rve

d

Sim

ula

ted

These

figure

s do n

ot h

ave to

be sa

me

as w

eath

ers a

re d

iffere

nt , b

ut

simila

rities a

re o

bse

rved.

These

figure

s are

expecte

d to

be v

ery

simila

r.

Estimating the Optimal Number of GNSS

Radio Occultation Measurements for

Numerical Weather Prediction

Florian Harnisch, Sean Healy, Peter Bauer

Both studies funded by ESA (European Space Agency)

Use of an Ensemble of Data

Assimilations to estimate benefit of

Aeolus Doppler Wind Lidar data

David Tan, Erik Andersson, Mike Fisher, Lars Isaksen

Acknowledgement. The next part of my presentation is based on results from two scientific studies:

ECMWF Lars Isaksen, ECMWF, Oct 2012 Use and interpretation of ECMWF Products 18

Ensemble of Data Assimilations (EDA)

Run an ensemble of analyses with perturbed observations, model physics and Sea Surface Temperature fields.

10 EDA members plus a control at lower resolution.

Form differences between pairs of analyses (and short-range forecast) fields.

These differences will have the statistical characteristics of analysis (and short-range forecast) error.

L

40°N 40°N

50°N50°N

60°N 60°N

20°W

20°W 0°

0°

Model Level 58 **Temperature - Ensemble member number 1 of 11

Thursday 21 September 2006 06UTC ECMWF EPS Perturbed Forecast t+3 VT: Thursday 21 September 2006 09UTC

0.5

0.6

0.7

0.8

0.9

1

1.1

1.2

1.3

1.4

1.5

1.6

Yellow shading where the

short-range forecast is

uncertain: give observations

more weight in these regions.

EDA method (1)

• Cycle of N (=10) 4D-Vars in parallel using

- perturbed observations

- perturbed model physics (SPPT scheme)

- assuming that

→ ensemble estimate of the error statistics of the analysis

and short-range forecast

Slide 19

xan (tk) xb

n (tk+1)

Analysis

system

Forecast

xbn (tk)

ζmn (SPPT)

ζmn (SPPT)

y (tk) + ηn

SST’n

EDA method (2)

• Computed ensemble spread s (variance) for EDA experiments

→ estimates the analysis and forecast uncertainty (variance)

per for each time d

for a period D (Expectation)

• Investigate how the EDA spread s is changing when additional

Aeolus data or GNSS RO data are used → observation impact

Slide 20

ECMWF The 5th WMO Symposium on Data Assimilation, Melbourne Oct 2009

Reference Result An & Fc Diagnostics

NWP-System Ensemble Observations

Assimilation Ensemble

Ref. run



Ensemble spread

Ensemble spread

Assimilation/ forecast Ensemble spread

Calibrate

Impact assessment


OSSE

Nature run





Assimilation/ forecast Compare to reference

Calibrate

Impact assessment

ECMWF The 5th WMO Symposium on Data Assimilation, Melbourne Oct 2009 22

S.Hem

0.0 0.5 1.0 1.5

1000

100

Profiles of 12 hour forecast impact, Southern Hemisphere

Spread in zonal wind (U, m/s)

Scaling factor ~ 2 for wind error

Tropics, N. & S. Hem all similar

Simulated DWL adds value at all altitudes and in longer-range forecasts (T+48,T+120)

Differences significant (T-test)

Supported by information content diagnostics

Tan et al 2007, QJRMS 133:381–390

ADM-Aeolus

NoSondes

Pre

ssure

(hP

a)

Zonal wind (m/s)

p<0.001

Simulation of GNSS RO data

Slide 23

interpolate

2D

bending angle

operator

randomly distributed

observation time and location

Operational ECMWF analysis

→ proxy for the “truth”

realistic

observation

errors

simulated GNSS RO

bending angle

profiles

Daily EDA spread

• No initial background

perturbation

• Short-term ensemble forecasts

provide the next background

• Cycling produces background

perturbations

• Less than 7 days to reach

stable state

• Only minor fluctuations →

estimate of FC uncertainty

(stdev) not individual FC error!

Slide 24

+12 h forecast of temperature at 100 hPa

period

Averaged EDA spread - Analysis

Slide 25

T (K) at 100 hPa: Analysis ensemble spread for June 8 - 27, 0 / 12 UTC

EDA_2

EDA_16 EDA_64

EDA_8

Summary

• Full OSSEs are expensive

• Sharing one Nature Run and simulated observation saves costs

• Generation of observations and calibration is a major exercise

• OSSE-based decisions have international stakeholders and

OSSEs should be developed as joint global projects

• Community ownership and oversight of OSSE capability is also

important for maintaining credibility

• Independent but related data assimilation systems allow us to

test the robustness of answers

• The EDA is an independent and simpler OSSE method that has

been shown to be valuable, but it does not make OSSEs obsolete

Slide 26

observation system simulation experiments (osse) · pdf filefeb 14 2012 emc seminar wind lidar...

Documents