TIGGE Data Archive at NCAR

9th GIFS-TIGGE Working Group

World Meteorological Organization

Geneva

31 Aug – 2 Sep, 2011

Doug Schuster

Steven Worley

Dave Stepaniak

Tom Cram

Others

Provider Summary

Archive Summary Service Improvements Special Projects Metrics Future Service Additions

Summary of Current Data Providers

•KMA and BoM stopped providing data•JMA now providing 00z analysis files

Archive Summary at NCAR

Online DataPeriod, most recent 3 months~ 50 TB , public products~ 10 TB, data preparation, subsetting~ 180GB Tropical Cyclone Track Data (CXML)

Offline DataFull period of record~ 600 TB, NCAR HPSS system

2010/2011 Service Improvements

Increased online data volume 5 TB -> 50 TB 2 weeks -> 3 months

Transitioned to new computing hardware Operates on longer data time-series (3 months) Faster request turn-around time

Transitioned to new HPSS tape archive Improved response time to access data

Strengthened data ingest processing New ldm server System monitoring (disk capacity, LDM status, etc) Failover mechanisms for LDM downtime

2010/2011 Special Projects

Developed and implemented first version of TIGGE model validation data portal Offers a wide variety of observational and analysis data

NCEP PREPBUFR data Used to initialize NCEP modelsSurface and Upper Air -T,U,V,Q,Z

Others to comeNCEP precipitation analysisTWG suggestions?

2010/2011 Special Projects

Developed and implemented first version of TIGGE model validation data portal Integrated with TIGGE model data subsetting portal

Check box to add model validation data based on model subset request parameters

NetCDF and ASCII output formatsCompatible with Mettools verification software

http://www.dtcenter.org/mrt/users Stand alone portal for independent requests Sends notification email with download instructions

TIGGE Model Validation Portal

TIGGE Model Validation Portal

NCAR TIGGE Model Data Archive Usage

YEAR GB AccessedGB

DownloadedActive Users / Unique Users

2006 5 5 0 / 5

2007 900 900 10 / 50

2008 1,300 1,100 10 / 20

2009 13,700 1,900 10 / 30

2010 48,200 2,200 12 / 32

2011 125,000 17,000 10 / 34

More data available for subsetting

NCAR TIGGE CXML Archive Usage

YEARGB

DownloadedFiles

Downloaded# Active Users /# Unique Users

2008 1 860 7 / 30

2009 26 12,300 17 / 35

2010 0.5 500 3 / 6

2011 147 19,500 6 / 21

NCAR TIGGE Model Validation Data Archive Usage

YEARGB

Downloaded# of Requests

# Active Users /# Unique Users

2011 200 140 6 / 23

Planned Service Improvements

Improved Online StorageHigh performance centralized disk to facilitate

subsetting services

Expanded/Updated ComputingTransition subsetting processing to HPC

environment ~late 2012 – 2013.Faster “turn-around” time on user sub-set requests

Potential for subscription services?Web services based requests?Special Projects Support -Hopson Meningitis

Proposal?Pending THORPEX Program Office Approval

Planned Service Improvements

Model Validation Data PortalAdd more data source options

Ocean based observations (ICOADS)Global tropical cyclone position and intensity dataAtmospheric Reanalysis (JRA, ERA, NNRR, CFSR)Global PCP datasetsSuggested TWG sets

Data sub-set and output format selection capability.Add output format options

http://tigge.ucar.edu/

Data Delivery (02/2010)

• Data exchanges using UNIDATA LDM, HTTP and FTP• 450 GB/day (ECMWF = 332 GB/day) • 1.7+ million 2-D gridded fields/day

PL = Pressure Level, PT = 320K θ Level, PV = ± 2 Potential Vorticity Level, SL = Single/Surface Level

Variable LvL ECWF UKMO JMA NCEP CMA CMC BOM MetF KMA CPTC

Geopotential Z PL

Specific H PL

T PL

U-velocity PL

V-velocity PL

Potential Vor PT

Potential T PV

U-velocity PV

V-Velocity PV

U 10m SL

V 10m SL

CAPE SL

Conv. Inhib. SL

Land-sea SL

Mean SLP SL

Orog. SL

Skin T SL

Snow D. H20 SL

Snow F. H20 SL

Parameter vs Provider Matrix

Variable LvL ECWF UKMO JMA NCEP CMA CMC BOM MetF KMA CPTC

Soil Moist. SL

Soil T SL

Sunshine D. SL

Surf. DPT SL

Surf. ATmax SL

Surf. ATmin SL

Surf. AT SL

Surf. P SL

LW Rad. Out SL

LH flux SL

Net Rad SL

Net Therm. Rad SL

Sensible Rad. SL

Cloud Cov SL

Column Water SL

Precipitation SL

Wilt. Point SL

Field Cap. SL

PL = Pressure Level, PT = 320K θ Level, PV = ± 2 Potential Vorticity Level, SL = Single/Surface Level

Parameter vs Provider Matrix

Archive Inhomogeneities

2010/2011 Supported Special ProjectsNCAR Research Applications Laboratory (RAL)

Google African Meningitis Forecast InitiativeDr. Thomas Hopson Integrate weather and health data to manage African

meningitis outbreaks (in Ghana as starting point).Looking for 2-week lead time to position vaccines.

Processing ECMWF data from present backwards through the archive for model calibration.Total Precipitation, 2m Temperature, 2m Dew Point.Global Subarea 40 deg N to 40 Deg S.0.5 x 0.5 degree resolution.Assist RAL and Unidata in providing “Real-time” access to

TIGGE based Multi-Model Ensemble Forecast data.

http://www.ral.ucar.edu/projects/ghana/http://www.google.org/predict.html