Download - Alastair Gemmell Jon Blower Keith Haines
The use of standard OGC web services in integrating distributed model, satellite and in-situ
datasets
Alastair GemmellJon BlowerKeith HainesEnvironmental Systems Science Centre & Reading e-Science Centre, University of Reading, UK
Martin PriceUK Met Office
Keiran MillardQuillon HarphamHR Wallingford, UK AGU Fall Meeting December AGU Fall Meeting December
20082008
Talk Outline
Context to the work
Potential problems
Useful tools and technologies
Our solution
Summary
Context to the work
Part of ECOOP project: European Coastal Operational Oceanography. 71 partners across most European countries.
Task was to integrate model, in-situ, and satellite data feeds from various project partners into one portal.
Users should be able to compare observed data with the model outputs.
Initial focus was on Ecosystem-relevant data in the North Sea. This has been expanded to other data.
Potential problems
Lack of interoperability
Heterogeneity in data formats
Data dispersed around Europe
Useful tools and technologies
OGC standards (www.opengeospatial.org)
Web Map Service (WMS)
Web Feature Service (WFS)
Climate Science Modelling Language (CSML)
THREDDS / OPeNDAP
Useful tools and technologies – OGC standards
WMS = Web Map Service. Serve geo-referenced images. Ideal for model output (also satellite data).
WFS = Web Feature Service. Serve geo-referenced points, lines, polygons. Ideal for in-situ observations, trajectories etc.
OGC has specifications for these services, allowing data to be served in a consistent manner.
Applications know what data format to expect and how to ask for it.
ncWMS: OGC-compliant WMS for NetCDF developed at Reading e-Science Centre (ncwms.sf.net).
Connects to Godiva2 web client which uses OpenLayers to display data.
Useful tools and technologies – CSML
Climate Science Modelling Language (csml.badc.rl.ac.uk)
A standards-based way of representing data features pertinent to the Climate Sciences.
13 main feature types including profiles, trajectories, swaths, timeseries.
Provides a common view onto datasets, independent of their storage format or physical location. Ideal for integrating diverse data products
We are developing a set of reusable Java libraries that embody the CSML concepts - can then apply these techniques to a number of other projects.
Useful tools and technologies – THREDDS
THematic Realtime Environmental Distributed Data Services (www.unidata.ucar.edu/projects/THREDDS/)
Enables data providers to serve NetCDF and similar data easily online via OPeNDAP protocol. Subsetting of data can be built into request
THREDDS now contains ncWMS bundled in (http://www.unidata.ucar.edu/projects/THREDDS/tech/TDS.html).
Data can be served by THREDDS either via OPeNDAP (e.g. obs data), or WMS (e.g. model images).
Diverse datasets held in different places can read in via THREDDS servers at each institute.
Our solution
ESSCWeb Portal
ESSCncWMS
web app.
CSML
ESSCECOOP Obs
web app.
ESSCPOLCOMS
MRCSmodel
(Physical)
CEFASSmartBuoys
SMHISEPRISE In-situ data
ESSCFerrybox (NetCDF)
HR WallingfordWFS
ASCII
LOCAL
REMOTE
PMLTHREDDS Server
CF-NetCDF
PMLTHREDDS Server
PMLPOLCOMS-
ERSEM MRCS model(Biological)
Comparing / co-plotting datasets can catch errors!
This looks like a suspiciously large and constant difference between obs and model
Summary
Historically data providers in different disciplines used different formats. This created barriers to multidisciplinary work.
Increasingly there are common standards and conventions for formatting and serving data. There is gradual uptake of these by providers.
We have found OGC WMS and WFS effective solutions for working with model and observed data respectively.
CSML was useful as a means of representing the diverse data as common feature types.
These tools and technologies allows diverse data to be brought together for the first time – this can reveal new trends in the data, as well as helping with quality control.