ESIP Air Quality Jan 2006 1
Air Quality ClusterTechnology Track
Earth Science Information PartnersPartners
• NASA• NOAA• EPA
• (?)• USGS• DOE• NSF• Industry…
Data Flow & Interoperabilityin DataFed
Service-based AQ Analysis System
R. B. Husar, S. R. Falke and K. Höijärvi
Washington University, St. Louis, MO
ESIP Federation Winter Meeting 2006Washington, DC, January 4, 2006
ESIP Air Quality Jan 2006 2
Federated Network for Air Quality Data and Processing ServicesProject Team:
Software Architecture: R. HusarSoftware Implementation: K. Höijärvi
Data and Applications: S. Falke, R. Husar
The DataFed Project
ESIP Air Quality Jan 2006 3
DataFed in a Nutshell
DataFed VisionAid air quality analysis by effective use of relevant data
DataFed Goals
Facilitate access and flow of AQ data from provider to users Support user-driven data processing value chains
Participate in specific application projects
Approach: Mediation Between Users and Data ProvidersDataFed assumes spontaneous, autonomous data providersNon-intrusively wraps datasets for access by web servicesMediates, homogenizes data views. e.g. geo-spatial, time...
Applications
Browsers and analysis tools for distributed monitoring data Serve as data gateway for user programs; GIS, science tools
DataFed is focused on the mediation of air quality data
ESIP Air Quality Jan 2006 4
Typical DataFed AQ Analysis Tools
Consoles: Data from diverse sources are displayed to create a rich context for exploration and analysis
CATT: Combined Aerosol Trajectory Tool for the browsing backtrajectories for specified chemical conditions
Viewer: General purpose spatio-temporal data browser and view editor applicable for all DataFed datasets
ESIP Air Quality Jan 2006 5
Web Services: Building Blocks of DataFed
Programming Access, Process, Render Data by Service Chaining
[Station]
[Monitor]Clickable monitor locations layer
Map boundary layer
ESIP Air Quality Jan 2006 6
Single Data Model for All AQ Data
Most Views are slices through a cube of data organized by lat, lon, altitude, and time (X,Y,Z,T)
Multidimensional Data Cube
ESIP Air Quality Jan 2006 7
OGC Abstract Specification – Coverage (00-106)
• Abstract Spec.: Coverages model and visualize spatial relationships between, and the spatial distribution of, earth phenomena
• Coverage subtypes of Air Quality interest are Image, Grid and Discrete Point
• Has a property (Coverage_Function) that has a spatial domain, and a value set.
• For Image and Grid, the Coverage_Function is an array…Discrete Point coverage is described by a PointC_Function where the spatial domain consists of a collection of points
Observation:The Abstract Spec. foresaw the need for Point Coverage, i.e. fixed monitoring Stations
ESIP Air Quality Jan 2006 8
Earth Science – GIS InteroperabilityShared Data Model
ESIP Air Quality Jan 2006 9
OGC Web Coverage Service (WCS) Specification
• HTTP GET/POST based interfaces• Services have XML service descriptions (“Capabilities”, “Description”)• Filter parameters allow selection of subsets of source data• Output formats advertised by each service instance
OGC WCS getCoverage SchemaSuitable for wrapping with SOAP envelope, WSDL access, loose coupling
WCS is for "coverages" – information representing space-time-varying phenomena
WCS describes, requests and delivers coverages in spatio-temporal domain
WCS version 1.1 is limited to grids/"simple” coverages with homogeneous range sets
ESIP Air Quality Jan 2006 10
through
Data Access through Adapters
DataFed SOAP,HTTP Get
OGC WCS HTTP Get, Post
OGC WMS HTTP Get
Station-Point SQL Server,
Files…
Sequence Image, file
nDim Grid OpenDAP NetCDF,
…
Other Traject., Event, Pic
Sources Diverse formats
Many data models
Data Wrapper Data into geo-cubes
Queries to views
Virtual Data Cube Global geo-cube data model Makes queries data-neutral
Others? e.g.
OpenDAP
Output Protocol dependent
User specified
GeoTable CSV,XLS,GML
GeoGrid GML,NetCDF..
GeoImage GeoTIFF,
PNG..
Other MS Dataset..
Query Adapter Maps query to protocol
User selects protocols
ESIP Air Quality Jan 2006 11
GALEON Interoperability Experiment
GALEON Geo-interface for Atmosphere, Land, Earth,
and Ocean netCDF
Lead by UNIDATA & OGC
Participants: academia, industry, international
Unify Earth Science & GIS Data Flows
B. DomenicoB. Domenico
GALEON UNIDATA
ESIP Air Quality Jan 2006 12
OGC WCS Demonstration: THREDDS_GFS 4Dim Dataset
Lat/Lon Box Elev Range Time RangeMap: BBOX=-180,-90,180,90, 1350,1350& TIME=2005-12-06/2005-12-06/PT3HTime: BBOX=-34,49.05,-34,49.05, 1350,1350& TIME=2005-12-05/2005-12-08/PT3HElev: BBOX=-34,49.05,-34,49.05, 0,18000 & TIME=2005-12-06/2005-12-06/PT3H
The form of the WCS query is the same for all slices through the data cube (views) The only difference in the views is the thickness of the slices in each dimension Return grid is in multiple formats (NetCDF, CSV, GML, PNG, … )
Map View Services WCS Query
Time View Services WCS Query
Elevation View Services WCS Query
ESIP Air Quality Jan 2006 13
OGC WCS Demonstration: AirNOW Station Dataset
Map View Services WCS Query
http://webapps.datafed.net/dvoy_services/ogc.wsfl?SERVICE=wcs&REQUEST=GetCoverage&VERSION=1.0.0&CRS=EPSG:4326&COVERAGE=AIRNOW.pmfine&FORMAT=CSV&
BBOX=-125.3889,22.55111,-61.40667,51.20889,0,0&TIME=2005-06-
26T15:00:00Z&WIDTH=999&HEIGHT=999&DEPTH=999
Formats: CSV, NetCDF-table, etc
Time View Services
http://webapps.datafed.net/dvoy_services/ogc.wsfl?SERVICE=wcs&REQUEST=GetCoverage&VERSION=1.0.0&CRS=EPSG:4326&COVERAGE=AIRNOW.pmfine&FORMAT
=CSV&BBOX=-90.16028,38.61222,-90.16028,38.61222,0,0&TIME=2005-06-20T00:00:00Z/2005-06-30T00:00:00Z&WIDTH=999&HEIGHT=999&DEPTH=999
Formats: CSV, NetCDF-table, etc
ESIP Air Quality Jan 2006 14
OGC WCS Demonstration: Grid, Image, Station Data Types
Coverage=THEEDDS.T& BBOX=-126,24,-65,52,0,0 &TIME=2002-07-07/2002-07-07&FORMAT=NetCDFCoverage=SURF.Bext& BBOX=-126,24,-65,52,0,0 &TIME=2002-07-07/2002-07-07&FORMAT=NetCDF-tableCoverage=SEAW.Refl& BBOX=-126,24,-65,52,0,0 &TIME=2002-07-07/2002-07-07&FORMAT=GeoTIFFCOVERAGE=sst& BBOX=-126,24,-65,52,0,0 &TIME=2001-01-01,2001-01-01&FORMAT=NetCDF
UNIDATA – THREDDS/GALEON WCS
DataFed GALEON WCS
U Florence, It GALEON WCS
DataFed GALEON WCS
Grid
Grid
Image
Station
Services WCS Query
Services WCS Query
Services WCS Query
Services WCS Query
ESIP Air Quality Jan 2006 15
Benefits of WMS Compatibility: Geospatial One-Stop
Provides Access toDataFed spatial layers
through WMS
Problem: No Time Dim!
http://www.datafed.net/DataLinks/OGC/OGC.htm
DataFed/VIEWS Asian Dust
Concentration (circles) S. Falke
ESIP Air Quality Jan 2006 16
Summary
OGC WCS protocol is suitable for delivering most air quality data:
Suitable for most data view queries
BBOX, TIME
Station-Point SQL Server,
Files…
Sequence Image, file
nDim Grid OpenDAP NetCDF,
…
Applicable to most data types
Station, Image, Grid
For Air Quality data needs: • WCS: Add Point coverage to WCS (in addition to grid, polygon) • WCS: Strongly typed (XML Schema-verifiable) return data types• Testing: Much fixing and testing for server-client compatibility