metadata in the ioos community

QARTOD III November 2-4, 20QARTOD III November 2-4, 200505

MetadataMetadatain thein the

IOOS CommunityIOOS Community

Julie BoschJulie BoschNOAA Coastal Data Development NOAA Coastal Data Development

CenterCenter

QARTOD IIIQARTOD IIINovember 2–4, 2005November 2–4, 2005


OverviewOverview

• Definition of MetadataDefinition of Metadata• Community Metadata ActivitiesCommunity Metadata Activities

– MMIMMI– COTS/ONRCOTS/ONR– QARTODQARTOD– Discipline-specific workshopsDiscipline-specific workshops– NOAA/NAVY Interoperability NOAA/NAVY Interoperability

DemonstrationDemonstration

• IOOS DMAC Metadata Expert TeamIOOS DMAC Metadata Expert Team– StatusStatus– PlansPlans– Post-Katrina metadata modelPost-Katrina metadata model


A Definition of MetadataA Definition of Metadata

The information needed to The information needed to

identify identify assess assess

use use access access

exchangeexchangetransporttransport archivearchive

data for IOOSdata for IOOS


Marine Metadata Marine Metadata InteroperabilityInteroperabilityProject (MMI)Project (MMI)

• Contributions form the marine data communityContributions form the marine data community• Growing resource for marine metadata Growing resource for marine metadata

informationinformation– Content Standards and ProfilesContent Standards and Profiles– Vocabularies and OntologiesVocabularies and Ontologies– Transport ProtocolsTransport Protocols– GuidesGuides– ExamplesExamples– ToolsTools

• Submitted vocabularies include parameter use Submitted vocabularies include parameter use and parameter discovery vocabulariesand parameter discovery vocabularies

• www.marinemetadata.orgwww.marinemetadata.org


Marine Metadata Marine Metadata InteroperabilityInteroperability

MMI Workshop on Advancing Domain MMI Workshop on Advancing Domain Vocabularies (August 2005)Vocabularies (August 2005)

• OWL files generated for over 40 existing vocabulariesOWL files generated for over 40 existing vocabularies

• Breakout groups initiated development of domain Breakout groups initiated development of domain ontologies (relating vocabularies) using VINE toolontologies (relating vocabularies) using VINE tool

WavesWaves Sensors and Sensors and InstrumentsInstruments

ChlorophyllChlorophyll UnitsUnitsCTD MeasurementsCTD Measurements Benthic Habitat ClassificationBenthic Habitat Classification

• Demonstration of interoperable data access using Demonstration of interoperable data access using metadata and a vocabulary mapping ontologymetadata and a vocabulary mapping ontology


COTS/ONR ProjectCOTS/ONR Project

• Metadata Working GroupMetadata Working Group– Developed a work plan to develop metadata Developed a work plan to develop metadata

to assist in enabling the IOOS to assist in enabling the IOOS Interoperability Demonstration (part 2)Interoperability Demonstration (part 2)

– Plan focused on determining/developing Plan focused on determining/developing controlled vocabularies and providing controlled vocabularies and providing metadata guidance for data including winds, metadata guidance for data including winds, SST, waves, surface current and chlorophyllSST, waves, surface current and chlorophyll

– Aggressive time-line for completionAggressive time-line for completion• Metadata work plan segment of Metadata work plan segment of

COTS/ONR Project not implementedCOTS/ONR Project not implemented


Quality Assurance Real Time Quality Assurance Real Time Ocean Data (QARTOD)Ocean Data (QARTOD)

QARTOD I – December 2003QARTOD I – December 2003– Metadata requirementsMetadata requirements

• General QA/QC flags be definedGeneral QA/QC flags be defined• Specific test used for QA/QC be definedSpecific test used for QA/QC be defined• Pros and cons of existing standards discussedPros and cons of existing standards discussed

QARTOD II – February/March 2005QARTOD II – February/March 2005– Discipline specific metadata needs identifiedDiscipline specific metadata needs identified– Waves, in-situ currents, remote currentsWaves, in-situ currents, remote currents

Post QARTOD IIPost QARTOD II– Progress on waves metadataProgress on waves metadata– NDBC and CDIP - FGDC wave metadata NDBC and CDIP - FGDC wave metadata

generationgeneration– Additional work to be done on incorporating the Additional work to be done on incorporating the

How and What of QC into metadataHow and What of QC into metadata


WorkshopsWorkshops

Discipline specific workshops (CSC and NDBC)Discipline specific workshops (CSC and NDBC)– Salinity (August 05)Salinity (August 05)

• QC for real-time salinity measurementsQC for real-time salinity measurements• Drafted metadata record example of salinity data attributesDrafted metadata record example of salinity data attributes

– Waves (November 05)Waves (November 05)• Significant advancements on QC requirements and Significant advancements on QC requirements and

recommendationsrecommendations• Identification of details that need to be captured in Identification of details that need to be captured in

metadatametadata

Results include identifying best practices for Results include identifying best practices for continuing the discipline specific approachcontinuing the discipline specific approach

What’s next? CTD, HF Radar, Chlorophyll,…?What’s next? CTD, HF Radar, Chlorophyll,…?


NOAA/Navy IOOS Interoperability NOAA/Navy IOOS Interoperability DemonstrationDemonstration

Data Discovery/Access Lessons LearnedData Discovery/Access Lessons Learned

• Determining the best (reasonable) Determining the best (reasonable) sources of data for the demonstration sources of data for the demonstration was a time consuming process and was a time consuming process and required extensive domain expertise required extensive domain expertise and contacts with individual data and contacts with individual data providers, pointing to the lack of providers, pointing to the lack of accessible metadata.accessible metadata.


NOAA/Navy IOOS Interoperability NOAA/Navy IOOS Interoperability DemonstrationDemonstration

Metadata RecommendationsMetadata Recommendations

• Standards must be established that will Standards must be established that will define the minimum set of common define the minimum set of common information that must be included with information that must be included with data submitted for use in the IOOS data submitted for use in the IOOS community.community.

• FGDC compliance is required but present FGDC compliance is required but present standards do not address most IOOS data standards do not address most IOOS data to allow functional discovery and access.to allow functional discovery and access.


IOOS DMAC Metadata Expert IOOS DMAC Metadata Expert Team Team

Co-Chairs: Anne Ball (CSC) and Julie Bosch (NCDDC) Co-Chairs: Anne Ball (CSC) and Julie Bosch (NCDDC)

Team MembersTeam Members– Plan for 10-12 members representing various Plan for 10-12 members representing various

organizations and disciplinesorganizations and disciplines– 8 team members currently nominated 8 team members currently nominated

OrganizationsOrganizations– NASANASA - NFRA - NFRA

(IOOS)(IOOS)– OBISOBIS - NOAA- NOAA– MMIMMI - ORION- ORION– InternationalInternational - EPA- EPA– Research Institutes (WHOI, UA)Research Institutes (WHOI, UA) - USGS- USGS– FGDC, ISOFGDC, ISO - Private Industry- Private Industry


GoalsGoals

Liaison and coordinate with related organizations and Liaison and coordinate with related organizations and activities in order to determine:activities in order to determine:– Content requirementsContent requirements– Appropriate standard(s)Appropriate standard(s)– Implementation of the standard(s)Implementation of the standard(s)– Metadata exchangeMetadata exchange– Data discovery toolsData discovery tools

Make recommendations to the steering team based on Make recommendations to the steering team based on these findingsthese findings

Provide a publicly accessible forum with information Provide a publicly accessible forum with information including metadata examples, templates, and tools including metadata examples, templates, and tools for IOOS participationfor IOOS participation


Key Team ActivitiesKey Team Activities

• Research metadata standards being used Research metadata standards being used for coastal and ocean observationsfor coastal and ocean observations

• Engage data collectors, users, and Engage data collectors, users, and modelers to identify the information modelers to identify the information needed to discover, use, understand, needed to discover, use, understand, access, transport, exchange, and integrate access, transport, exchange, and integrate observational dataobservational data

• Develop templates and profiles of Develop templates and profiles of metadatametadata


Ontologies

QARTODQC Flags

ControlledVocabularies

MetadataStandards

FormatSpecifications

MetadataTransport

DataTypes

DiscoverySystems

Post-Katrina Model of Metadata


Parting ThoughtsParting Thoughts

Examples of basic metadata for Examples of basic metadata for CDT, HF Radar, and Wave data CDT, HF Radar, and Wave data are availableare available

During QARTODDuring QARTOD– Identify and note specific QC Identify and note specific QC

details that need to be captured details that need to be captured as part of the metadataas part of the metadata


Julie BoschJulie Bosch

NOAA Coastal Data Development NOAA Coastal Data Development CenterCenter

Stennis Space Center, MSStennis Space Center, MS

228-688-3841228-688-3841

[email protected]@noaa.gov

metadata in the ioos community

Documents

metadata guidance

discipline specific

lack of accessible metadata

individual data providers

advancing domain vocabularies

qc requirements

extensive domain expertise

time consuming process