SIMORC Workshop March 2007
SeaDataNet
A Pan-European Infrastructure for Ocean and Marine Data
Management
SIMORC Workshop March 2007
Background
• Data availability of vital importance for marine research
• European data set is fragmented
• Not always quality controlled
• Not always easily accessible
• In 35 countries bordering European seas• more than 600 scientific laboratories from government
organizations and private industry collect data
• use various sensors on board of research vessels, submarines, fixed and drifting platforms, airplanes and satellites
• measure physical, geophysical, geological, biological and chemical parameters, biological species, etc.
SIMORC Workshop March 2007
What is SeaDataNet?
• EU Sixth Framework Programme • Integrated Infrastructure Initiative (I3)• 1 April 2006 to 31 March 2011• Objectives
• develop an efficient distributed pan-European marine data management Infrastructure for managing large and diverse data sets
• network existing professional data centres of 35 countries, active in data collection
• provide integrated databases of standardised quality on-line
SIMORC Workshop March 2007
SeaDataNet partners
49 partners including:• Major oceanographic
institutes of 35 countries
• National Oceanographic Data Centres (NODCs)
• Divisions of major national marine research institutes
• 3 satellite data centres
• 3 scientific modelling centres
• 2 SMEs experts in software development
• 3 International organisations
SIMORC Workshop March 2007
Building on experience
• Data management of scientific projects, e.g. MTPII-MATER, CANIGO, OMEX, MERSEA, etc.
• Leaders of Data Management Concerted Actions• MEDAR/MEDATLAS II - Mediterranean Data Archaeology
and Rescue, aiming to make available a comprehensive data product of temperature, salinity and bio-chemical data in the Mediterranean and Black Sea through a wide international cooperation
• SEA-SEARCH – The gateway to Oceanographic and Marine Data & Information in Europe, to , to provide an effective navigation tool to oceanographic and marine data and information sources for European marine waters and to centres in Europe with expertise and skills in oceanographic and marine data & information management.
• Bringing in new partners – satellite data centres
SIMORC Workshop March 2007
SeaDataNet components
SIMORC Workshop March 2007
Infrastructure, versions & planning
• Version 0Continuation and maintenance of existing Sea-Search systems with minor modifications
• Version 1All data centres involved in upgraded meta-databases
Technical Task Team partners involved in transparent data access, security, monitoring, viewing services = trans-national access
Pre-operational by October 2007 and operational by February 2008
• Version 2All data centres involved in trans-national access and access upgraded
Operational by 2010
SIMORC Workshop March 2007
SeaDataNet data policy
• Metadata• free and open access, no registration required• each data centre should provide the meta-data in standardized
format to populate the catalogue services
• Data and products services • the general case if free and open access/mandatory registration• a “SeaDataNet role” (partner, academic, commercial etc.) is
attributed to each user • Each NODC attributes the roles to the users of its of country• Out of the partnership, the roles are assigned by SeaDataNet user-
desk
• each data centre node delivers data according to the user’s role and its local regulation
• each data centre should provide freely the data sets necessary to develop the common products
SIMORC Workshop March 2007
Discovery services
• Marine environmental data sets (EDMED)
• Marine environmental research projects (EDMERP)
• Observing systems (EDIOS)• Research Cruise Summary
Reports (CSR) • Common Data Index (CDI)• Directory of marine
organisations (EDMO)
SIMORC Workshop March 2007
Cruise Summary Reports
• Conceived by IOC• ICES acted as focal point • Currently over than 37000
cruises• Documenting data
collected on scientific research cruises operated by European vessels
• Cruise track charts included
• Online entry form• Online searchable
database
SIMORC Workshop March 2007
European Directory of the Ocean-observing System (EDIOS)
• Initiative of the European Global Ocean Observing System (EuroGOOS)
• Regular repeated measurements
• On-line searchable directory of observing, measuring, and monitoring systems
• Over 10,000 observation entries
• Links to on-line real-time and archive data
• Tool to aid design of observing systems
SIMORC Workshop March 2007
Common Data Index
• Index to individual data sets
• Approx. 200000 entries from 10 partners
• ISO19115 metadata standard has been adopted
• On-line searchable directory
• Direct access to on-line data sets, catalogues or e-mail request
SIMORC Workshop March 2007
Technical developments
• Establishment of common vocabularies
• Upgrading formats for interrelationships between these directories
• ISO 19115 content model is the basis for the XML formats and exchange schema (XSD)
• Innovating maintenance modalities:• XML exchange formats upgrading
• XML validation services
• Online Content Management System (CMS)
• Introducing Web services• Vocabularies and Directory of Marine Organisations (EDMO)
SIMORC Workshop March 2007
SeaDataNet standards
• Interoperability – key to distributed data management system success• Includes establishing common understanding of terms
used• Achieved through use of controlled vocabularies
containing keys, terms, abbreviations and essential (but often missing) definitions
• Schemes for handling duplicate data• Common quality control protocols• Ocean Data View software for visualisation
SIMORC Workshop March 2007
SeaDataNet : Delivery Services V1
Primary function• Users will be able to download
selected data sets CDI User Interface• Searching and browsing of
metadata of data sets
Shopping Basket in combination with AAA services• Preparing a user request of multiple data sets, handling
validation of users, and routing requests to the data centres
SIMORC Workshop March 2007
Authentication, Authorization and Administration (AAA)
• Single Sign On system required for access to distributed system
• User’s authentication information based on personal login / password
• User must register in order to get login• Web form to provide necessary information • Online user agreement “SeaDataNet General Licence”• After processing, login/password sent by e-mail (e-mail check)
• Users’ directory management (registration processing)• Centralised vs Decentralised
• Choice in V1 for CAS system (= centralised) • SeaDataNet Data Policy • Authorisation based on “Roles”
SIMORC Workshop March 2007
SeaDataNet : Delivery Services V1
Download manager• process of requests by each data centre to deliver one
comprehensive data set (one zip file) per data centre
Request status manager• provide users overview of request status of requests and
download functions• provide data centres with overview of all requests and
data delivery
Data delivery formats
• CF compliant NetCDF format for gridded and multi-dimensional data types (e.g. ADCP)
• Modified ODV spreadsheet format for other data types (e.g. time-series, profiles)
SIMORC Workshop March 2007
SeaDataNet User’s portal schema V1
IFREMER Database
BODC Database
BSH Database
...
Download managers in data centres
CSR EDIOS
EDMED
CDI
Shopping basket
Requestsstatus
manager
datametadata
CSR
CDI
Project info
Metadata & Data catalogues
Data request Status of request
General request Metadata request
Organisation +data source id
Data download
at BODCat BSH at MARIS+ EDMERP
services for metadata
Data downloading services
Visualisation services (WMS) for V2
Entry point for access hits
EDMO
EDIOS EDMED
EDMERP EDMO
Software
Vocabularies
Standards Cross search
AAA
Registr.
My transact.
User Register
Userregistration
SIMORC Workshop March 2007
Data products
• SeaDataNet will provided added-value products to be created by regional centres• Mediterranean Sea• Black Sea• Baltic Sea• Arctic and North Sea• Atlantic and global ocean
• First products => gridded climatologies • based on already available data-sets (mostly hydrographic data)• updated during the project life
• To make products comparable, all regional data centers will use a common tool for spatial mapping
• For this a new version of DIVA (Data Inverse Variational Analysis) was developed (University of Liege)
SIMORC Workshop March 2007
SeaDataNet capacity building
The training program has two main objectives:
•to ensure that the data and metadata to be integrated in the SeaDataNet system will be formatted, checked for quality and disseminated according to the common protocols developed during the project
•to transfer expertise and to train IT experts of the SeaDataNet data centers in the basics, installation and operation of the SeaDataNet technical components
SIMORC Workshop March 2007
SeaDataNet system components
• Discovery services = Metadata directories
• Vocabulary services = Common vocabularies
• Auxiliary tables = Organisations, Research Vessels
• Security services = Authentication, Authorization & Administration (AAA)
• Monitoring and statistics = usage & system performance
• Delivery services = Data access & downloading of data sets (ASCII, NetCDF, …, data virtualisation)
• Viewing services = GIS, ODV analysis software, Google Earth, ..
• Product services = Generic and standard products
SIMORC Workshop March 2007
International Conference IMDIS 20081st Announcement
• Second international conference “International Marine Data Information Systems” (IMDIS)
• March 31 - April 1, 2008, Athens, Greece • Joint organisers: SeaDataNet, Hellenic Centre for
Marine Research, IOC/IODE • Four sessions:
• Marine environmental data bases: infrastructures and data access systems
• Data Services in ocean science: standards and interoperability
• Visualisation tools• Services for Users and Education
SIMORC Workshop March 2007
www.seadatanet.org