1

An Overview ofAn Overview ofHDF-EOSHDF-EOS

(Part II)(Part II)Doug IlgDoug Ilg

Raytheon STXRaytheon STX

Doug.Ilg@gsfc.nasa.govDoug.Ilg@gsfc.nasa.gov

(301) 441-4089(301) 441-4089

2

OutlineOutline

The Swath InterfaceThe Swath Interface What is an HDF-EOS File?What is an HDF-EOS File?

3

Swath Data SetsSwath Data Sets

Instrument

Scan Lines

Along Track

Inst

rum

ent P

ath

Instr

ument P

ath

Pro

file

s

Instrument

Along Track

4

DimensionName: GeotrackSize: 21

DimensionName: ScanSize: 16

DimensionName: TrackSize: 42

Map1DataDimension: “Track”

Geodimension: “Geotrack”Offset: 1

Increment: 2

“Temp” “Time”

A Swath StructureA Swath Structure

5

AccessAccess DefinitionDefinition Basic I/OBasic I/O InquiryInquiry UtilityUtility SubsetSubset

Components of the Swath Components of the Swath InterfaceInterface

6

Tips on Writing a SwathTips on Writing a Swath

Order of calls is significant:Order of calls is significant:– define dimensions before using them to define dimensions before using them to

define fields of mapsdefine fields of maps– setting a compression method affects all setting a compression method affects all

subsequently defined fieldssubsequently defined fields Unless a dimension map is defined, a Unless a dimension map is defined, a

1-to-1 mapping is assumed during 1-to-1 mapping is assumed during subsetting. This can produce strange subsetting. This can produce strange results.results.

7

Swath Subsetting FeaturesSwath Subsetting Features

By TimeBy Time– SWdeftimeperiod/SWextractperiodSWdeftimeperiod/SWextractperiod

By GeolocationBy Geolocation– SWdefboxregion/SWextractregionSWdefboxregion/SWextractregion

By “Vertical” fieldBy “Vertical” field– SWdefvrtregion/SWextractregionSWdefvrtregion/SWextractregion

Tip: use Time or Geolocation first, Tip: use Time or Geolocation first, then Verticalthen Vertical

8

Compression Methods for Compression Methods for SwathsSwaths

Run-Length EncodingRun-Length Encoding Adaptive HuffmanAdaptive Huffman GzipGzip

9

Question:Question:

What’s an HDF-EOS File?What’s an HDF-EOS File?

10

Some AnswersSome Answers

ECS: An HDF file with ECS ECS: An HDF file with ECS metadatametadata

Any file with an HDF-EOS object in Any file with an HDF-EOS object in itit

A file with A file with onlyonly HDF-EOS objects in HDF-EOS objects in itit

11

My AnswerMy Answer

You’ve asked the wrong question.You’ve asked the wrong question.

The difference between HDF and The difference between HDF and HDF-EOS is at the object level, not HDF-EOS is at the object level, not the file level.the file level.

12

ECSMetadata

HDF-EOS

HDF

An HDF fileAn HDF file

One or more HDF-EOS data objectsOne or more HDF-EOS data objectswith attendant structural metadatawith attendant structural metadata

Core and possibly Archive Core and possibly Archive metadatametadata

One or more HDF data objectsOne or more HDF data objects

KeyKey

13

ECSMetadata

HDF-EOS

HDF•PortabilityPortability•Elementary data accessElementary data access•Basic self-descriptionBasic self-description

•Geolocation information Geolocation information tightly coupled to datatightly coupled to data

•Fully standardized Fully standardized geographical subsettinggeographical subsetting

•Moderate self-descriptionModerate self-description

•ECS ingest, archiving, ECS ingest, archiving, search, and distributionsearch, and distribution

•Enhanced self-Enhanced self-descriptiondescription

14

An HDF file with a set ofAn HDF file with a set ofHDF data objectsHDF data objects

Services ProvidedServices Provided•PortabilityPortability•Elementary data Elementary data accessaccess•Basic self-descriptionBasic self-description

Services Services NotNot Provided Provided•Geolocation information Geolocation information tightly coupled to the tightly coupled to the datadata

•Fully standardized Fully standardized geographical subsettinggeographical subsetting

•ECS ingest, archiving, ECS ingest, archiving, search, and distributionsearch, and distribution

•Enhanced self-Enhanced self-descriptiondescription

HDF

15

HDF-EOS

An HDF file with a set ofAn HDF file with a set ofHDF-EOS data objectsHDF-EOS data objects

Services ProvidedServices Provided•PortabilityPortability•Elementary data accessElementary data access•Moderate self-Moderate self-descriptiondescription

•Geolocation Geolocation information tightly information tightly coupled to the datacoupled to the data

•Fully standardized Fully standardized geographical geographical subsettingsubsetting

Services Services NotNot Provided Provided•ECS ingest, archiving, ECS ingest, archiving, search, and distributionsearch, and distribution

•Enhanced self-Enhanced self-descriptiondescription

16

HDF-EOS

HDF

Services ProvidedServices Provided•PortabilityPortability•Elementary data Elementary data accessaccess

•Moderate self-Moderate self-descriptiondescription

•Geolocation Geolocation information tightly information tightly coupled to coupled to somesome of of the datathe data

•PartiallyPartially standardized standardized geographical geographical subsettingsubsettingServices Services NotNot Provided Provided

•ECS ingest, archiving, ECS ingest, archiving, search, and distributionsearch, and distribution

•Enhanced self-descriptionEnhanced self-description•FullyFully standardized standardized geographical subsettinggeographical subsetting

An HDF file with a mixedAn HDF file with a mixedset of HDF-EOS and HDFset of HDF-EOS and HDFdata objectsdata objects

17

Services ProvidedServices Provided•PortabilityPortability•Elementary data Elementary data accessaccess

•ECS ingest, archiving, ECS ingest, archiving, search, and search, and distributiondistribution

•Enhanced self-Enhanced self-descriptiondescription

Services Services NotNot Provided Provided•Geolocation information Geolocation information tightly coupled to the tightly coupled to the data data

•Fully standardized Fully standardized geographical subsettinggeographical subsetting

An HDF file with a set An HDF file with a set ofofHDF data objects HDF data objects with ECS metadatawith ECS metadata

HDF

ECSMetadata

18

Services ProvidedServices Provided•PortabilityPortability•Elementary data accessElementary data access•Geolocation information Geolocation information tightly coupled to the tightly coupled to the data data

•Fully standardized Fully standardized geographical subsettinggeographical subsetting

•ECS ingest, archiving, ECS ingest, archiving, search, and distributionsearch, and distribution

•Enhanced self-Enhanced self-descriptiondescription

Services Services NotNot ProvidedProvided

•NoneNone

An HDF file with a set An HDF file with a set ofofHDF-EOS data HDF-EOS data objects with ECS objects with ECS metadatametadata

ECSMetadata

HDF-EOS

19

Services ProvidedServices Provided•PortabilityPortability•Elementary data accessElementary data access•Geolocation information Geolocation information tightly coupled to tightly coupled to somesome of the data of the data

•PartiallyPartially standardized standardized geographical subsettinggeographical subsetting

•ECS ingest, archiving, ECS ingest, archiving, search, and distributionsearch, and distribution

•Enhanced self-Enhanced self-descriptiondescription

Services Services NotNot Provided Provided•FullyFully standardized standardized geographical subsettinggeographical subsetting

An HDF file with a An HDF file with a mixed set of HDF and mixed set of HDF and HDF-EOS data HDF-EOS data objects with ECS objects with ECS metadatametadata

ECSMetadata

HDF-EOS

HDF