Progress in Joint OSSESimulation of observation for calibration

Preparation for DWL OSSE

Michiko MasutaniNOAA/NWS/NCEP/EMC

JCSDA , Wyle IS

1

Michiko Masutani[1,2,#], Jack S. Woollen[1,+], Zaizhong Ma[2,$], Lars Peter Riishojgaard, [2,$,5], Dave Emmitt[6], Sid Wood[6], Steve Greco[6],

Tong Zhu[3,@], Haibing Sun[3,%], Nikki Prive[4,@], Yuanfu Xie[4], Ellen Salmon[5] , Bill McHale[5], Thomas J. Kleespies[3], Guan Ping Lou[1+], FangLin Yang[1,+]

Ronald Vogel[3,+], Diane. Stokes[1], Kate. L. Howard[1,+], Fuzhong Weng[3], Yong Han[3]

[1]NOAA/National Centers for Environmental Prediction (NCEP)[2]Joint Center for Satellite and Data Assimilation (JCSDA)

[3]NOAA/ NESDIS/STAR,[4]NOAA/Earth System Research Laboratory (ESRL) [5] NASA/GSFC ,[6]Simpson Weather Associates

# Wyle Information Systems, McLean, VA,,+IM Sciance Group)IMSG), MD

$Goddard Earth Science and Technology Center, University of Maryland, Baltimore, MD, @Cooperative Institute for Research in the Atmosphere (CIRA)/CSU, CO

%Perot System Government Services, Virginia

OSSE:Observing Systems Simulation Experimentshttp://www.emc.ncep.noaa.gov/research/JointOSSEs/

Main contributors in 2010

2

Acknowledgement

The nature runs for Joint OSSEs were produced by Dr. Erik Andersson of ECMWF. We appreciate GMAO to provide initial satellite data for calibration at ESRL. GMAO also provided code to add random error to simulated data.

A Nature Run (NR, proxy true atmosphere) is produced from a free forecast run using the highest resolution operational model which is significantly different from NWP model used in DAS.

Data Assimilation Systems (DAS)Process initial condition from observational data

OSSEEvaluate observing systems and data assimilation systems using simulation experiments.

Observing Systems Simulation Experiments (OSSE)

DoD CG/AR at Colorado State University AMS Data Assimilation Education Forum January 21, 20084

Why do data assimilation?

1. I want better model initial conditions for better model forecasts

2. I want better calibration and validation (cal/val)

3. I want better acquisition guidance

4. I want better scientific understanding of Model errors (and their probability distributions)

Data errors (and their probability distributions)

Combined Model/Data correlations

DA methodologies (minimization, computational optimizations, representation methods, various method approximations)

Physical process interactions (i.e., sensitivities and feedbacks)Leads toward better future models

The Purpose of Data AssimilationAndrew S. Jones

http://www.emc.ncep.noaa.gov/research/JointOSSEs/references/Jones_Data_Assimilation_Ed_Forum.ppt

DoD CG/AR at Colorado State University AMS Data Assimilation Education Forum January 21, 20085

The Data Assimilation CommunityAndrew S. Jones

http://www.emc.ncep.noaa.gov/research/JointOSSEs/references/Jones_Data_Assimilation_Ed_Forum.ppt

What skills are needed by each involved group? NWP Data Assimilation Experts (DA system methodology) NWP Modelers (Model + Physics + DA system) Application and Observation Specialists (Instrument capabilities) Physical Scientists (Instrument + Physics + DA system) Radiative Transfer Specialists (Instrument config. specifications) Applied Mathematicians (Control theory methodology) Computer Scientists (DA system + OPS time requirements) Science Program Management (Everything + $$ + Good People) Forecasters (Everything + OPS time reqs. + Easy/fast access) Users and Customers (Could be a wide variety of responses)

e.g., NWS / Army / USAF / Navy / NASA / NSF / DOE / ECMWF

Full OSSEs

Data impact on analysis and forecast will be evaluated.

A Full OSSE can provide detailed quantitative evaluations of the configuration of observing systems.

A Full OSSE can use an existing operational system and help the development of an operational system

A Nature Run (NR, proxy true atmosphere) is produced from a long free forecast run using the highest resolution operational model which is significantly different from NWP model used in DAS.

Calibrations will be performed to provide quantitative data impact assessment.Without calibration quantitative evaluation of data impact is not possible.

International Joint OSSE capability

• Full OSSEs are expensive– Sharing one Nature Run and simulated observation save the

cost– Share diverse resources

• OSSE-based decisions have international stakeholders– Decisions on major space systems have important scientific,

technical, financial and political ramifications– Community ownership and oversight of OSSE capability is

important for maintaining credibility

• Independent but related data assimilation systems allow us to test robustness of answers

Nature Run Spectral resolution : T511 13 month long. Starting May1st,2005Vertical levels: L91, 3 hourly dumpDaily SST and ICE: provided by NCEP

Andersson, Erik and Michiko Masutani 2010: Collaboration on Observing System Simulation Experiments (Joint OSSE), ECMWF News Letter No. 123, Spring 2010, 14-16.

Complete Nature run data set is posted from NASA/NCCS portalhttp://portal.nccs.nasa.gov/osse/index.pl Password protected. Accounts are arranged by Ellen Salmon (Ellen.M.Salmon@NASA.gov)

Copies are available to designated users for research purpose& users known to ECMWF User list is maintained by Michiko Masutani (NOAA/NCEP)contact:michiko.masutani@noaa.gov

Joint OSSE Data

NASA/NCCS provided dis space for Joint OSSE data shirring

There is a entry created for Joint OSSEhttp://portal.nccs.nasa.gov/josse/index.pl

Make entry to each data set and generating institute, and contact person.People use these data must contact generating institutes.

Shared Joint OSSE data

Data posted by NCEP and NESDIS

NCEP will post simulated observation as progress. First we post from NCEP ftp site (rzdm) to be evaluated. After the data is evaluated it will be transferred to NASA NCCS portal. We appreciate any help in evaluation of the simulated observations.

Note.Radiance data are also being simulated by Environment of Canada using RTTOV. GMAO is working on calibrated radiance data

[Plans ]Complete 13month of radiance observation has been simulated and being evaluated.Ascii data will not be posted for new data but software to conert bufr to ascii data will be provided.Prepbufr for real data will be posted after sensitive data removedradiance data for 2009-2010 satellite.

[consideration]Sampling strategies for further radiance dataRTM used for simulation. RTTOV is available in reserch version of CRTM Observational error to be added Correlated observational error by Tom Kleespies Non Gaussian error distribution

Ozone may need to be completed for 13 month.

Software used to simulate radiance data posted from NCCS portalCRTM used for simulation. CRTM1.2.2 (Different from the version posted from JCSDA website)

Presentation for verification by Tong Zhu and Haibing SunLink to libraries requres Sample data set

Software to simulate IASI, Metop data has been tested and sample is posted

Radiance Simulation codes

Data posted from Joint OSSE Home page http://www.emc.ncep.noaa.gov/research/JointOSSEs/

Simulated TC vitalMichiko Masutani (NCEP/EMC

Guan Ping Lou (NCEP/EMC)

[Simulation of TC vital]TC vital was simulated using software originally written by Tim Marchock and currently developed by Guan Ping Lou of NCEP. 

Preliminary simulation which need to be verifiedDescription of data set posted from NASA/NCCS/portalSoftware used to simulation

TC-vital for 13 monthThe software are posted

13

OSSE Calibration

● In order to conduct calibration all major existing observation have to be simulated.

● The calibration includes adjusting observational error.

● If the difference is explained, we will be able to interpret the OSSE results as to real data impact.

● The results from calibration experiments provide guidelines for interpreting OSSE results on data impact in the real world. ● Without calibration, quantitative evaluation data impact using OSSE could mislead the meteorological community. In this OSSE, calibration was performed and presented.

15OAS-P199

14

Calibration at NCEP and ESRL

Calibration at NOAA/ESRL with NCEP

Using simulated data by GMAO and additional data from NCEP.Focused on July-August 2005.GSI version May 2007.

Some comparison between NCEP and GSO radiance data with and without observational error. NCEP radiance give about 0.5-1% higher AC in day 3. GMAO Obs error reduce AC for about 1-2% in day 3.

Calibration at NCEP

FY10 NCEP GDAS systemObsevational error is not added for simulated dataData denial experiment for RAOB wind showed simulated wind without observational error has about 1.5 more impact compared to real data.

Real Simulated

DWL data will be assimilated as Line Of Sight (LOS) wind component. Two LOS observation like U and V must produce very similar impact as one vector wind U and V. i.e. Two LOS observation at close location by GWOS will produce a vector wind like impact.

1515OAS-P199

Preparation toward OSSE for GWOS DWL

Clean up code and remove some mishandling of the data improve the performance at polar region.

Up grade code to handle various type of DWL data

Upgrade code are submitted as NCEP subversion to be reviewed for implementation

Variational quality control improve the performance in polar region. (The figure shows a zonally averaged improvement V on day three. Cint=0.05 and maximum is about 0.5m/s in both poles.)

0.5

Sensitivity to various constant to observational error is noted.Current assignment seems to be too small

Tuning factors Gross error, representativeness error Measurement error included the data Observational error assigned to data Response function in GSI

Evolution of GFS Forecast Skill

S. Lord and Fanglin Yang (NCEP/EMC)

500 hPa Anomaly Correlation

NCEP

Percent Poor Forecasts

NCEP

Does It Make a Difference toHow Forecasters Use Product?

NH

SH

Note on Data impact depend on forecast skill metric

Percent good Forecasts

1715OAS-P199

• “But these are just ‘width of the line’ improvements…”• Are these differences in scores important to users?• Can more sensitive measures of forecast performance

be developed?

Yang, Fanglin 2011: 24th Conference on Weather and Forecasting [Available online at http://ams.confex.com/ams/91Annual/webprogram/Paper182605.html]

Zonally and time averaged improvement by DWL compared to CTLReduction of the distance from the Nature run

V

Analysis

5 day fcst

3 day fcst

2 day fcst

1 day fcst

4 day fcst

U

1815OAS-P199

Countour interval is doubled for U

Averaged from forecast date July 7 to August 15.

The impact start from upper level tropics and spread to midlatitude

Some negative value in summer hemisphere is noted. However case study showed positive impact of DWL.

SH NH

Max distance from Nature run in 72 hr fcst Mean sea level pressure

Control With DWL

MSLP of 72 hour fcst. Coutour is NRFcst time 00Z August 3rd verified at 00Z August 6

Location of storm agree with Nature run when DWL added

NH AC drop off on 00z August 3rd forecast in control but not with DWL

Time series of 500hPa geopotential height ACZaizhong Ma et al (LWG Miami, February 2011)

NH SH

Day 1

Day 5

Day 3

Candidates for additional study

02/08/2011 20Working Group on Space-based Lidar Winds, Miami

00Z August 3rd

21

Summary

The Joint OSSE Nature runs have been evaluated, posted and made available to research communityNCEP data assimilation system is ready for simulation of DWL in different characteristics.Four month of simulated radiance (used in 2005), and ozone data at NCEP-NESDIS are posted from NASA/NCCS portal. Radiance data for complete 13 month period is being evaluated,The complete 13month long simulated conventional observation were been posted from NASA /NCCS portal.TC vital was simulated and posted from NCEP rzdm to be evaluated.Global OSSEs using Joint OSSE Nature run are conducted at NCEP-JCSDA-NESDIS-SWA (DWL) NASA/GMAO (ADM) NOAA/ESRL (UAS) Environment of Canada (PCW) MRI, Japan starting

15OAS-P199

22

Remark on Joint OSSEs

Using Full OSSE, various experiments can be performed and various verification metrics can be tested to evaluate data impact from future instruments and data distributions.

Theoretical predictions have to be confirmed by full OSSEs. The results are often unexpected. OSSE results also require theoretical back ups. Evaluation Data Assimilation System though OSSE is drawing many interest

Many interest from Regional OSSEs

Lack of appreciation of amount of work discourage the collaboration

15OAS-P199