status of hdf-eos, related software, and tools

Status of HDF-EOS,Status of HDF-EOS,Related Software, and ToolsRelated Software, and Tools

Abe Taaheri, Raytheon IIS

HDF & HDF-EOS Workshp XIII Riverdale, MD

November 4, 2009

TOOLKIT / HDF-EOS SupportTOOLKIT / HDF-EOS Support

• Maintenance support is funded on EMD to EED Bridge (EEB) contract through Dec 2010

• Support will continue with EOSDIS Evolution & Development (EED) award in 2010

• Current support (One full-time developer) : HDF-EOS2 & 5 SDP/MTD Toolkit HDF-EOS plug-in for HDFView HDF-EOS2 to GeoTIFF Converter (HEG) Other Toolkit/HDF-EOS related software Toolkit/HDF-EOS/HEG helpdesk Related web sites maintenance

ECS Support of HDF-EOSECS Support of HDF-EOS

• Both HDF4 and HDF5 based flavors of HDF-EOS are part of ECS baseline.

• OS’s: Solaris (9, 10), Linux ( 32 and 64-bit ), Mac OS X (Power PC and Mac Intel),

Windows XP (MTD Toolkit, HDF-EOS2 & 5, HEG, HDFView)

• Added full Cygwin support in July 2009 release

• Compilers: Fortran 77/90 & g77/pgf90 , C, C++, gcc, g++,gfortran (Mac Intel)

Downloadable SoftwareDownloadable Software

• SDP/MTD Toolkit (Science Data Processing Tools)

• HDF-EOS2 (HDF4 based)• HDF-EOS5 (HDF5 based)

• heconvert (converts HDF4 - based Grid/Point/Swath structures to HDF-EOS 5 equivalents)

Downloadable ApplicationsDownloadable Applications

• HEGHEG - GeoTIFF conversion - Spatial subsetting, Field Subsetting - reprojection - stitching, sub-sampling and more ….

• HDFView Plug-inHDFView Plug-in, Java-based browser (HDF4, HDF5, HDF-EOS 2 and 5 access)

• HE5ViewHE5View* (HDF5 - based browser)• EOSViewEOSView* (HDF4 - based browser)

* Dropped support for OS updates on July 2006

Software/Tools in detailSoftware/Tools in detail

• What are: TOOLKIT, HDF-EOS ?

ToolkitToolkit

• Science Data Production (SDP) Toolkit :collection of tools used by

- Science software developers who producecode to process instrument data.

- HDF-EOS users who need to• Access metadata in HDF-EOS files• Perform Time/Date conversion• Access Digital Elevation Model Data• Access Ancillary Data, such as Digital Chart

of the World database (DCW), Olson World Ecosystem files, DEM data files, etc.

Toolkit (cont.)Toolkit (cont.)

• Tools provided in SDP Toolkit:- AA (Ancillary Data Access)- CBP(CELESTIAL Body Position)- CSC(Coordinate System Conversion)- CUC (Constant and Unit Conversions)- DEM (Digital Elevation Model Access)- EPH(Ephemeris Data Access)- GCT (Geo Coordinate Transformation)- IO (Input Output (File I/O))- MEM (Memory Management)- MET (Metadata Access)- PC (Process Control)- SMF (Status Message File (Error/Status))- TD (Time and Date Conversion)

Toolkit (cont.)Toolkit (cont.)

• Tools provided in MTD Toolkit:- MET (Metadata Access)- TD (Time and Date Conversion)

HDF-EOSHDF-EOS

• HDF-EOS is an extension to HDF which standardizes storage of and access to common Earth Sciences data structures

– By providing a single interface to data structures common to earth science

– By providing a container for EOS inventory, archive and product specific metadata

– By establishing a consistent relationship between geolocation and science data.

HDF-EOS 2HDF-EOS 2

• HDF4 - based, storage format for EOS standard products.

• Used operationally by MODIS, MISR, ASTER, Landsat, AIRS and other EOS instruments

• Support for Grid/Point/Swath structures

HDF-EOS 5HDF-EOS 5

• Based on HDF5, a complete rewrite of HDF-EOS2 with a different user interface.

– Used operationally by EOS Aura instruments

• Designed to ‘resemble’ HDF-EOS 2 to the maximum extent possible. – Supports same data structures (and more)– Added prefix ‘HE5_’ to HDF-EOS 2 functions

Software ReleasesSoftware Releases

Toolkit/HDF-EOS:Toolkit/HDF-EOS:

• Last release – July 2009– TOOLKIT 5.2.16 – HDF-EOS 2.16, HDF- EOS5.1.12

with HDF4.2r4, HDF5-1.8.3, Szip2.1

• Next release – June 2010– TOOLKIT 5.2.17 – HDF-EOS 2.17, HDF-EOS5.1.13

with new releases of HDF4, 5 (5-1.8.X)

Software Releases (Cont.)Software Releases (Cont.)

HEG (HDF-EOS2 to GeoTIFF Converter)HEG (HDF-EOS2 to GeoTIFF Converter)::

• Last release – August 2008– Version 2.9

• Next release – November 2009– Version 2.10


HDF-EOS Plug-in for HDFViewHDF-EOS Plug-in for HDFView::

• Last release – September 2008– Version 2.4

• Next release – – For HDFView Version 2.5, December 2009


HECONVERT (HDF-EOS2 to HDF-EOS5 HECONVERT (HDF-EOS2 to HDF-EOS5 Converter)Converter)::

• Early releases – 2001, 2004– Version ?

• Last release – Dec. 2008– Version 2.0

• Next release – Not Planned

TOOLKIT:TOOLKIT:

– XML metadata writing capabilty for inventory metadata in SDP toolkit (MLS)

– ODL fix (WriteLable) for potential core dump while writing long input pointer metadata attributes (AIRS)

– Support TOOLKIT for CYGWIN– pccheck utility fix for problems resulting with

empty space at a line beginning in PCF file

Major Development/FixesMajor Development/FixesLast Release (July 2009)

HDF-EOS5:HDF-EOS5:

– Support for reading HDF-EOS5 files with netCDF– Correcting wrong tiling and compression info

in the structure metadata– Use of unlimited dimension in HDF-EOS5

(Users Guide clarification)– Correction of output attribute data types in

attribute inquiry routine– Fix for writing int8 data correctly to output

hdf5 file with C calls. There was no such a problem with Fortran calls

Major Development/Fixes (cont.)Major Development/Fixes (cont.)

HDF-EOS2:HDF-EOS2:

– Fixed PTdeftimeperiod to return an error when stop time < start time

– Fix for correct internal compression method detection for the products that are compressed internally after the production, possibly using hrepack (structure metadata has no record of compression info)

– Numerous fixes for HEConvert (version 2.0 release)

Major Development/Fixes (cont.)Major Development/Fixes (cont.)

AvailabilityAvailability

• Access to libraries, applications, Documentation:

TOOLKITTOOLKIT:: http://newsroom.gsfc.nasa.gov/sdptoolkit/toolkit.htmlhttp://newsroom.gsfc.nasa.gov/sdptoolkit/toolkit.html

HDFViewHDFView::http://newsroom.gsfc.nasa.gov/sdptoolkit/HDFView/http://newsroom.gsfc.nasa.gov/sdptoolkit/HDFView/

HDFView_hdfeos_plugin.htmlHDFView_hdfeos_plugin.html

HEGHEG: : http://newsroom.gsfc.nasa.gov/sdptoolkit/HEG/HEGHome.htmlhttp://newsroom.gsfc.nasa.gov/sdptoolkit/HEG/HEGHome.html

AvailabilityAvailability

• Questions, Comments, RequestsQuestions, Comments, Requests::

[email protected]

[email protected]

http://www.hdfeos.org/ : : HDF-EOS ForumHDF-EOS Forum

Thank You!Thank You!

status of hdf-eos, related software, and tools

Documents