the nodc glider technical specification tom ryan, dan seidov, john relph (nodc) and james bennett...
TRANSCRIPT
The NODC Glider Technical Specification
Tom Ryan, Dan Seidov, John Relph (NODC)and
James Bennett (University of Washington)
U.S. IOOS National Glider Strategy Development WorkshopAugust 1, 2012
2
NODC Helping Data Providers to Comply with Requirements to
Archive Data
NOAA Data Management Procedural Directive
National Science Foundation (NSF) Data Management Plan Requirements
Grant Requirements
3
Open Archival Information System (OAIS) Reference Model
Submission Information Package (SIP)– SIP is what a data collector/producer submits to
NODC Archival Information Package (AIP)
– AIP is what a collector/producer submits to NODC along with NODC generated files (browse graphic, transforms, metadata, etc.)
Dissemination Information Package (DIP)– DIP is what NODC provides to users through
discovery and access URLs
4
Glider Technical Specification Goals
Simplify the Procedure for Archiving Glider Data at NODC
Provide Framework For Glider Data Management Plans
Support Interoperability of Glider Data
5
NetCDF File Format
Data in netCDF is mandatory for NODC automation– Data are self documenting (metadata and data are maintained in
same file)– Improved data access and discovery (OPeDAP, THREDDS, WMS, WCS)
Climate and Forecast (CF) Metadata Conventions– Use CF Standard Name
Attribute Convention for Dataset Discovery (ACDD) NODC netCDF Templates
– Templates do not represent an attempt to create a new standard and are not required for archiving data at NODC.
http://www.nodc.noaa.gov/data/formats/netcdf/
6
File Naming Convention
PPPPP_GGG_DDDD_YYYYMMDD.nc
PPPPP – Prefix, Five Submitter defined alpha characters
GGGGG – Glider Serial Number, Five alpha numeric characters
DDDD – Deployment Dive Number, Four numeric characters
YYYYMMDD – Start date of deployment .nc –netCDF file extension
7
Metadata
Submitter generated collection level and/or deployment level FGDC/ISO metadata are encouraged but not required
NODC will create FGDC/ISO metadata record for SIP with discovery and access URLs to data – Ocean Archive System – Geoportal Server– HTTP and FTP – OPeNDAP’s Data Access Protocol via Hyrax– OGC’s WMS and WCS via THREDDS Data Server
Submission Manifest
• Contact information of submitter• List of all SIP files with cryptographic file
integrity validation (e.g., MD5 algorithm)- Ensures NODC received what was submitted
without any errors- Supports revisions of AIP for updated/corrected
data.
9
Submission Email
Email sent to [email protected] contains the following elements:– Data Provider– Contact Person– Collecting Institution– Submitting Institution – Transfer Protocol– Glider Type– Glider name– Manifest of files
Email is part of the SIP and provides provenance to the data.
10
File Transfer Methods Pull – Data is mirrored on NODC
servers (NODC preferred protocols)– RSYNC Service Protocol– FTP protocol
Push – Data are pushed to NODC Servers– Data must be uploaded to NODC FTP
servers in compressed form to allow for re-uploading if transmission errors occurred.
11
Platform Codes
Gliders are platforms and NODC will request platform codes from the International Council for Exploration of the Sea (ICES). To facilitate this request, the data submission must include: – Platform Name:– Country:– Platform Class : Autonomous underwater vehicle– Current length (meters):– Year Built:– Notes:
12
Cross-references
NODC encourages data submitters to provide all relevant documentation which will be published in Documentation-only Accessions. These accessions will provide persistent URLs to specific documents to be included in netCDF files.
Documents to include:– Quality Control Manual– Non-copyrighted journal articles– Technical Reports– Other documentation
13
Conclusion
The NODC guidelines for submitting glider data for automated accessioning are straightforward and easy to follow.
Following these guidelines by a collector/submitter will reduce the work needed to set up the automated process of archiving their data at NODC by up to 90% and will guarantee ease and smoothness of the archiving process.
14
Acknowledgement
The Group for High Resolution Sea Surface Temperature (GHRSST) Data Specification Version 2.0 (GHRSST Science Team, 2010)
served as a model for this Technical Specification.
15
Resources NODC Contacts
– [email protected]– [email protected]– [email protected] (add as a CC to all
correspondence) NODC netCDF templates
– http://www.nodc.noaa.gov/data/formats/netcdf/ CF Standard Names
– http://cf-pcmdi.llnl.gov/documents/cf-standard-names/ ACDD
– http://www.unidata.ucar.edu/software/netcdf-java/formats/DataDiscoveryAttConvention.html
NetCDF Compliance Checker– http://puma.nerc.ac.uk/cgi-bin/cf-checker.pl