documentation from ncml to iso ted habermann, noaa nesdis ngdc

Documentation from NcML to ISOTed Habermann, NOAA NESDIS NGDC

Layers of Access

HDF5 Data

HDF & NetCDF Library

Climate Forecast, Attribute Conventions

for Data Discovery

MatLab, IDL, IDV, Ferret, GMT

OPeNDAP

Geospatial Database

SQL

Community Data Models,Open GIS Specifications

ArcMap, ArcIMS,WMS, WFS, WCS

GML, KML, SimpleFeatures

HDF/netCDF GIS

Conventions

Discovery

Use / Mashup

Understanding

Climate-Forecast (CF) ConventionsStandard variable names and data organizations

Unidata Attribute Convention for Data Discovery

ISO Conventions

ISO Conventions

Containers

THREDDS Catalog

NcML In the file or the catalog

netCDF in the file

Purpose

Documentation Completeness

COMPLETNESS Use UnderstandingDiscovery

ClimateForecast ACDD ISO

NcML

19139

ncISO ncISO?

DocumentationRepositoryISO 19115,

19115-2, 19119 and extensions

THREDDS

HDF, netCDF, ACDD, CF

DIF, FGDC,Data.Gov

SensorML

WCS, WMS, WFS, SOS

KML

Open Provenance

Model, PROV

Documentation in Multiple Dialects

Although CF is specifically a netCDF standard, we feel that most of the ideas areof wider application. The metadata objects could be contained in file formats other than netCDF. Conversion of the metadata between files of different formats will be facilitated if conventions for all formats are based on similar ideas.

Spiral 1:Initial Content

Metadata ContentIndependent of standard

StandardGuidance /

Implementation

Check BackWith Users

Spiral 2-N: Scientific QuestionsNew RequirementsNew Use Cases

Check BackWith Data Collectors/

Providers

Spiral Development / Training

IdentificationIdTitleAbstractResource DateTopic CategoryTheme KeywordMetadata ContactScience Contact

ExtentGeospatial Bounding BoxTemporal Start/EndVertical Min/MaxPlace Keywords

DistributionDistributor ContactOnline ResourceDistribution FormatData Center KeywordsBrowse Graphic

Text SearchesPurposeExtent DescriptionLineage StatementProject Keywords

Quality/LineageSourcesProcess StepsQuality Reports / Coverages

Acquisition InformationInstrumentPlatformInstrument KeywordsPlatform Keywords

Content InformationAttribute TypeAttribute NamesAttribute DefinitionsAttribute Units

Discovery Understanding

ConnectionOnlineResource:Linkage (URL)NameDescriptionFunction

Spiral Development / Training: Potential Spirals

Climate-Forecasttitle

THREDDSdataset@idkeywordkeyword/@vocabularycreator/namecreator/contact@emailcreator/namedate/@type="created, issued, modified"]

ISOfileIdentifiertitleabstractresource date(s)topicCategorytheme keywordmetadata contactscience contact

Data Discoverynaming_authority:idtitlecreator_namecreator_emailcreator_urldate_createdinstitutionsummarykeywordskeywords_vocabulary

Identification and DiscoveryAs metadata are shared between national and international repositories it is becoming important to unambiguously identify specific records. This required an identifier in the metadata. Some mechanism must exist for ensuring that these identifiers are unique. This is accomplished by specifying the naming authority or namespace for the identifier.

Climate-Forecast THREDDS ISOOnlineResource:Linkage (URL)NameDescriptionFunction

Data Discovery

Connection

Climate-Forecastgeospatial_lat_mingeospatial_lat_maxgeospatial_lon_mingeospatial_lon_maxgeospatial_vertical_mingeospatial_vertical_maxtime_coverage_starttime_coverage_end

THREDDSgeospatialCoverage/northsouth/startgeospatialCoverage/northsouth/sizegeospatialCoverage/eastwest/startgeospatialCoverage/eastwest/sizetimeCoverage/starttimeCoverage/endgeospatialCoverage/updown/startgeospatialCoverage/updown/size

ISOGeospatial Bounding BoxTemporal Start/EndVertical Min/MaxPlace Keywords / Geographic Identifiers

Data Discoverygeospatial_lat_mingeospatial_lat_maxgeospatial_lon_mingeospatial_lon_maxgeospatial_vertical_mingeospatial_vertical_maxtime_coverage_starttime_coverage_end

Extent

This basic extent information supports spatial/temporal searches that are increasingly important as the number of map based search interfaces increases. Many of the attributes included in this spiral can be calculated from the data if the file is compliant with the NetCDF Climate and Forecast (CF) Metadata Convention.

Climate-Forecastgeospatial_lat_units &resgeospatial_lon_units & resgeospatial_vertical_units & restime_coverage_units & res

THREDDSgeospatialCoverage/northsouth/units &resgeospatialCoverage/eastwest/units &resgeospatialCoverage/updown/units &res

ISOData Discoverygeospatial_lat_units &resgeospatial_lon_units &resgeospatial_vertical_units &restime_coverage_durationtime_coverage_resolution

Other Extent

Climate-Forecast THREDDS/metadata/publisher/name/metadata/publisher/contact/@email/metadata/publisher/contact/@urlService URLs

ISODistributor ContactOnline ResourceDistribution FormatData Center KeywordsBrowse Graphic

Data Discoverypublisher_namepublisher_emailpublisher_url

Distribution

Climate-Forecastcommentreferences

THREDDSmetadata/documentationmetadata/projectmetadata/documentation[@type="funding"]

ISOPurposeExtent DescriptionLineage StatementProject Keywords

Data Discoverycontributorcontributor_roleacknowledgementproject

Text Searches

Text searches are a very important mechanism for data discovery. This group includes attributes that contain descriptive text that could be the target of these searches. Some of these attributes, for example title and summary, might also be displayed in the results of text searches.

Climate-Forecastsource

THREDDS ISOInstrumentPlatformInstrument KeywordsPlatform Keywords

Data Discovery

Acquisition Information

Climate-Forecastsourcehistoryflag_masksflag_valuesflag_meaningsancillary_variables

THREDDS ISOSourcesProcess StepsQuality Reports / CoveragesQuality Flags

Data Discovery

Quality Information

Climate-Forecastlong_namestandard_namestandard_name_vocabularyscale_factoradd_offsetvalid_min, max, rangeunits

THREDDSmetadata/variables@vocabulary

ISOAttribute TypeAttribute NamesAttribute DefinitionsAttribute Units

Data Discoverycdm_data_type

Content Information

Spiral Development / Training: Rubrics

THREDDS Data ServerMultiple Dialects:

THREDDS Catalog

file1.nc file2.nc file3.nc file4.nc file1.nc file.ncfile.nc file.ncfile2.nc file.nc

THREDDS Catalog

THREDDS Catalog

Extract Data

OPeNDAP WMS WCS

Extract Metadata (NcISO)

NcML ISO Rubric

THREDDS Metadata Server

THREDDS Metadata Server

Documentation in Three Dialects

ISO ACDDNcML

https://geo-ide.noaa.gov/wiki/index.php?title=NcISOhttp://groups.google.com/

group/ncisometadata

DocumentationRepositoryISO 19115,

19115-2, 19119 and extensions

THREDDS

HDF, netCDF, ACDD, CF

DIF, FGDC,Data.Gov

SensorML

WCS, WMS, WFS, SOS

KML

Open Provenance

Model, PROV

Documentation in Multiple Dialects

1010101010101010101111100100101101001001001001000100100010010101001001010010010101000100100010100100100101010101010101010101010100010010101000010100

Metadata

Where Are Citations?

XML

dataset /resource

source

keywordthesaurus

& ontology

evaluationprocedure

standardspecification

algorithm featurecatalog

applicationschema

alternatemetadata

metadata& servicestandard

associatedresource

name/metadata

additionaldocumentation

constraintsreference

sourcemetadata

processreference &

documentation

softwarereference

featurecatalog

formatspecification

Documentation

Questions?

ted.habermann@noaa.gov