GRID-ENABLED GRID-ENABLED MEDIATION SERVICES MEDIATION SERVICES

FOR GEOSPATIAL FOR GEOSPATIAL INFORMATIONINFORMATION

Ilya Zaslavsky, Chaitan BaruIlya Zaslavsky, Chaitan Baru

San Diego Supercomputer CenterSan Diego Supercomputer CenterUniversity of California San DiegoUniversity of California San Diego

OutlineOutline

The Geosciences Network Project The Geosciences Network Project (GEON)(GEON)

Grid-enabling GEON servicesGrid-enabling GEON servicesWeb services, grid services, OGSA, OGCWeb services, grid services, OGSA, OGC

Running example: geologic map Running example: geologic map integrationintegration

Ontology servicesOntology servicesMap assembly servicesMap assembly servicesDemoDemo

GEON: The Geosciences GEON: The Geosciences NetworkNetwork

IT: SDSC, Penn State, San Diego State UniversityIT: SDSC, Penn State, San Diego State University Geosciences: Arizona State University, Bryn Mawr College, Geosciences: Arizona State University, Bryn Mawr College,

Cornell University, Rice University, UNAVCO, University of Cornell University, Rice University, UNAVCO, University of Arizona, University of Idaho, University of Missouri, University Arizona, University of Idaho, University of Missouri, University of Texas El Paso, University of Utah, Virginia Techof Texas El Paso, University of Utah, Virginia Tech

Education and Outreach: DLESE, Cornell, UNAVCOEducation and Outreach: DLESE, Cornell, UNAVCO Agency Partners: USGS, NASAAgency Partners: USGS, NASA Industry partners: IBM, ESRI, GeoFusionIndustry partners: IBM, ESRI, GeoFusion

• Two testbeds• Broad range of geoscience data sets

GEON IT IssuesGEON IT Issues Prototyping a national information Prototyping a national information

infrastructure for Geosciencesinfrastructure for Geosciences Close collaboration between geoscientists and Close collaboration between geoscientists and

IT to interlink databases and Grid-enable IT to interlink databases and Grid-enable applicationsapplications

The GEON Grid: heterogeneous networks, The GEON Grid: heterogeneous networks, compute nodes, storage capabilities (SRB, compute nodes, storage capabilities (SRB, Rocks, Globus)Rocks, Globus)

““Deep” data modeling of 4D dataDeep” data modeling of 4D data Situating 4D data in context—spatial, temporal, Situating 4D data in context—spatial, temporal,

topic, processtopic, process XML-based standards for data exchangeXML-based standards for data exchange Semantic integrationSemantic integration of Geosciences data of Geosciences data

Logic-based formalisms to represent knowledge and map Logic-based formalisms to represent knowledge and map between ontologiesbetween ontologies

GEMS= Grid-Enabled Mediation ServicesGEMS= Grid-Enabled Mediation Services

Grid ServicesGrid Services

OGSA: architectureOGSA: architectureGlobus: software tools (Globus: software tools (www.globus.orgwww.globus.org))Global Grid Forum: standards (Global Grid Forum: standards (

www.ggf.orgwww.ggf.org))Features:Features:

WSDL/SOAP-basedWSDL/SOAP-basedSecurity and authenticationSecurity and authenticationPersistent state mechanismsPersistent state mechanisms

SDSC GEMS (and GeoGEMS)SDSC GEMS (and GeoGEMS)

SDSCSDSC Grid-Enabled Grid-Enabled Mediation Services (GEMS)Mediation Services (GEMS)

Based on XML, XQuery (next generation of MIX—Based on XML, XQuery (next generation of MIX—Mediation of Information using XML)Mediation of Information using XML)

Defined in terms of a set of services that are used at:Defined in terms of a set of services that are used at: ““Registration time”Registration time”

Dataset registration, schema registration, ontology registrationDataset registration, schema registration, ontology registration Source content and capability related services: e.g., “term Source content and capability related services: e.g., “term

resolution” service, capability description service, …resolution” service, capability description service, … ““View definition time”View definition time”

Data Integration Services, Discovery servicesData Integration Services, Discovery services Query runtimeQuery runtime

Dynamic binding of logical to physical resourcesDynamic binding of logical to physical resources Administrative ServicesAdministrative Services

Services to manage access controls, control replication, …Services to manage access controls, control replication, …

GEMS Components

Query Optimization & Plan Generation

Verification, AccessControl, and Query

Rewrite

Result Assembly(e.g. map

generation)

Ontology Service

CommunityAuthorization

Service

Monitoring& Discovery

Service

NetworkWeather Service

ReplicaLocation Service

Client

ComputeResources

Distributed Compute and Storage Resources

DatabasesDatabases

Databases

File systemFile

systemFile system

ComputeResourcesCompute

Resources

Registration Services

Metadata Registry

Deployment Services

Data Integration Services

Querying and MappingGUI (vector)

. . .

WSDL WSDL

WSWSDL

WSWSDL

Mediator

SOAPserver

SOAPserver

SOAPserver

SOAPserver

WSWSDL

Querying and MappingGUI (raster)

Spatial results assembly

. . . Global legend generation Integrated View

ArcXML exchanges

ArcIMS

WS HTTP GET

WMS

WS

Ontology-basedrewriting

OGSA AXIS

Tomcat Apache

STACK

OGSA

Example: Geologic Map Example: Geologic Map IntegrationIntegration

State Name “Geologic age” Lithography Map Symbol Descriptive Other Arizona Period - Abbrev Descr D_Symbol,

P_Symbol, Geo Idaho ERA, System,

Series, Period, also: Age=Era& System(Period)& Series

Rock_Type, LITH1..LITH8, LITHA, LITH_FORM, IDLITH=LITHA& LITH_FORM LITHOLOGY= LITH_MAJOR=LITH1& LITH2&LITH3

Formation Unit_Name D_Symbol, P_Symbol, P1, HP

Nevada Time_Unit - Fmatn - D_Symbol, P_Symbol, COLR, COLR2, COLR3

Virginia - - Geology - Pennsylvania Age (Devonian,

Silurian, Quaternary, etc.)

Lith1, Lith2, Lith3 Map_Symbol Name

Delaware Geo_Unit_A (Holocene, Pleistocene, Pliocene, etc.)

- Geo_Unit_S Geo_Unit_N

GeoGEMS: RegistrationGeoGEMS: Registration Source schema, with user-defined types Source schema, with user-defined types

and operationsand operations Exposed functions (in ArcIMS lingo)Exposed functions (in ArcIMS lingo)

Get_service_infoGet_service_info Get_feature_countGet_feature_count Get_imageGet_image png png Get_FeaturesGet_Features XML string XML string Get_ExtractGet_Extract zipped shapefile zipped shapefile

Semantics registration (*.owl)Semantics registration (*.owl) Data quality informationData quality information + standard items: narrative, spatial + standard items: narrative, spatial

extent…extent…

<?xml version="1.0" encoding="ISO-8859-1" ?> <geological_ages>

<era name="Cenozoic"><period name="Quaternary">

. . .</period><period name="Tertiary">

<subperiod name="Neogene" my="22"><epoch name="Pliocene" my="3.6">

<age name="Piacenzian" ma="3.4"/><age name="Zanclian" ma="5.2"/>

</epoch><epoch name="Miocene">

<intervals my="5.2"><age name="Messinian" ma="6.7"/><age name="Tortonian" ma="10.4"/>

</intervals><intervals my="5.9">

<age name="Serravallian" ma="14.2"/><age name="Langhian" ma="16.3"/>

</intervals><intervals my="7">

<age name="Burdigalian" ma="21.5"/><age name="Aquitanian" ma="23.3"/>

</intervals></epoch>

</subperiod><subperiod name="Paleogene" my="42">

<epoch name="Oligocene"><age name="Chattian" ma="29.3" my="6"/><age name="Rupelian" ma="35.4" my="6.1"/>

</epoch><epoch name="Eocene">

<age name="Priabonian" ma="38.6" my="3.2"/><intervals my="11.4">

<age name="Bartonian" ma="42.1"/><age name="Lutetian" ma="50"/>

</intervals><age name="Ypresian" ma="56.5" my="6.5"/>

</epoch><epoch name="Paleocene">

<age name="Thanetian" ma="60.5" my="4"/><age name="Danian" ma="65" my="4.5"/>

</epoch></subperiod>

</period></era>

+ <era name="Mesozoic">+ <era name="Paleozoic">+ <era name="Sinian">+ <era name="Riphean">

<era name="Animikean" /> <era name="Huronian" /> <era name="Randian" /> <era name="Swazian" /> <era name="Isuan" />

+ <era name="Hadean"></geological_ages>

OntologyOntology

Services Services for Map for Map IntegrationIntegration

A. Concept Expansion Extracting all sub-concepts

of the queried term (WHERE period=“Tertiary”) from global ontology

B. Concept Resolution Each source returns unique

values for each query term (stored in mapping.xml) rewriting query in terms of local source

Global ontology

<?xml version="1.0" encoding="UTF-8"?><concepts> <list><concept>Tertiary</concept></list> <list><concept>Quaternary/Tertiary</concept></list> <list><concept>Tertiary/Jurassic</concept></list> <list><concept>Tertiary/Cretaceous</concept></list></concepts>

WHERETIME_UNIT = "Tertiary" OR TIME_UNIT = "Quaternary/Tertiary" OR TIME_UNIT = "Tertiary/Jurassic" OR TIME_UNIT = "Tertiary/Cretaceous" OR TIME_UNIT = "Tertiary"

Operational uses of Operational uses of ontology in GISontology in GIS

Many operations in GIS are based on Many operations in GIS are based on establishing object identityestablishing object identityEdge-matchingEdge-matchingPlanar enforcementPlanar enforcementGeneralization Generalization ……

Associating spatial objects with Associating spatial objects with concepts in domain ontology provides concepts in domain ontology provides for a richer set of object identity rulesfor a richer set of object identity rules

AutomaticAutomatic labeling results…labeling results…

12

3

4

Special Cases: 1Special Cases: 1

B: basal nucleus of Meynert B: basal nucleus of Meynert (C0004788)(C0004788)

LGP: lateral globus pallidus, LGP: lateral globus pallidus, C0262267C0262267

Basal nucleus cells (B) are Basal nucleus cells (B) are within LGP, but their precise within LGP, but their precise locations not known locations not known polygon is coded LGP, B is a polygon is coded LGP, B is a secondary descriptorsecondary descriptor

SpecialSpecial Cases: 3Cases: 3

DG: dentate gyrus, C0152314DG: dentate gyrus, C0152314 PoDG: polymorph layer of the dentate PoDG: polymorph layer of the dentate

gyrusgyrus CA1: field CA1 of hippocampus CA1: field CA1 of hippocampus

(C0019564)(C0019564) All of them have a common parent: All of them have a common parent:

hippocampus hippocampus a common parent is a common parent is used to label polygon; polylines are used to label polygon; polylines are labeled separatelylabeled separately

Mediator

LegendGenerator

MapAssembler

Ontology

…

GRID SERVICESFOR MAP INTEGRATION

ArcIMS and WMS Services wrapped in WSDL/SOAP

Querying multiple map servicesIn GEON (www.geongrid.org)

SpatialSpatial Results AssemblyResults Assembly

Previously (in MIX): XMAS query specifies how Previously (in MIX): XMAS query specifies how result fragments are merged into XML treeresult fragments are merged into XML tree

Now:Now: XML fragments contain or reference fragments of XML fragments contain or reference fragments of

different typesdifferent types Format of the output map is determined at run timeFormat of the output map is determined at run time Query results must be placed in spatial context, Query results must be placed in spatial context,

which is not explicitly requested in a querywhich is not explicitly requested in a query The composite map should support further queriesThe composite map should support further queries

MapMap Assembly Scenarios - Assembly Scenarios - 11

Client-side overlay of map images from individual Client-side overlay of map images from individual sourcessources Grid service interface:Grid service interface:

GifService:getGif (String imageService, String layerID, String GifService:getGif (String imageService, String layerID, String highlightColor, String envelope, String imageSize, String highlightColor, String envelope, String imageSize, String expression)expression)

Example (“Tertiary” in Nevada):Example (“Tertiary” in Nevada): GifService:getGif GifService:getGif ("nevada", ("nevada",

"2", "2", “255,255,0”,“255,255,0”,”-122.523,51.09,-99.361,29.10315875613748”,”-122.523,51.09,-99.361,29.10315875613748”, “611,580 “ “611,580 “ “(TIME_UNIT = &apos;Tertiary&apos; OR TIME_UNIT = “(TIME_UNIT = &apos;Tertiary&apos; OR TIME_UNIT = &apos;Quaternary/Tertiary&apos; OR TIME_UNIT = &apos;Quaternary/Tertiary&apos; OR TIME_UNIT = &apos;Tertiary/Jurassic&apos; OR TIME_UNIT = &apos;Tertiary/Jurassic&apos; OR TIME_UNIT = &apos;Tertiary/Cretaceous&apos; OR TIME_UNIT = &apos;Tertiary/Cretaceous&apos; OR TIME_UNIT = &apos;Tertiary&apos;)”&apos;Tertiary&apos;)”) )

These are converted to GET_IMAGE or Getmap requests These are converted to GET_IMAGE or Getmap requests at source wrappers at source wrappers

Variants: using ImageFusion service at the middleware to Variants: using ImageFusion service at the middleware to merge individual images into onemerge individual images into one

MapMap Assembly Scenarios - Assembly Scenarios - 22

Results image is generated at the Results image is generated at the mediator by merging coordinate mediator by merging coordinate information from information from ArcIMS/WFS/Oracle sourcesArcIMS/WFS/Oracle sources Each source returns XML-formatted Each source returns XML-formatted

coordinate stringscoordinate strings Only image of query results get Only image of query results get

generatedgenerated Original images don’t have to be Original images don’t have to be

updatedupdatedVariants:Variants:

Passing coordinate data on to Passing coordinate data on to client as SVG/VMLclient as SVG/VML

Xml coordinates

MapMap Assembly Scenarios - Assembly Scenarios - 33

New image service New image service dynamically generated at the dynamically generated at the mediator from raster and/or mediator from raster and/or vector fragmentsvector fragments More flexibleMore flexible Services: GridFTP, (un)compress, Services: GridFTP, (un)compress,

ImageAssembly, ImageFusion, ImageAssembly, ImageFusion, Query, DataConversionQuery, DataConversion

Further user interaction - with Further user interaction - with the new servicethe new service

Convenient session managementConvenient session management

Raster/vector fragments

ArcIMS Image Service

Wrappers

Raster Mapping

Client

Vector Mapping

Client

Java Application

Command Module

Result Assembly

Service

Command Module

Wrappers

Raster Mapping

Client

Vector Mapping

Client

Java Application

Mediator

Source Wrappers

Raster Mapping

Client

Vector Mapping

Client

Java Application

getExtractgetFeaturesgetImagegetFeatureCountgetServiceInfocompress

WSDL:

OrganizationOrganization of map assembly of map assembly servicesservices File

Transfer service

Uncom-press

service

Image Assembly service

Image Fusion service

Query service

Data Conversion service

ArcIMSImageservice

Ontology and other services

Command.xml

Request=extent&query&clienttype

Request fragmentsResponse: type&extent&handle

Request:type&clienttype&handle&extent

MapMap Assembly As Grid Assembly As Grid ServiceService

Creation – via Creation – via factoryfactory interface: interface: Creates a new grid service instance, as ArcIMS Creates a new grid service instance, as ArcIMS

serviceservice CreateCreate grid service handle (unique URL) grid service handle (unique URL) Grid Service Reference (GSR): a WSDL Grid Service Reference (GSR): a WSDL

description of the service instance (returned by description of the service instance (returned by Http GET on GSH)Http GET on GSH)

Lifetime management:Lifetime management: SoftStateDestruction and ExplicitDestruction SoftStateDestruction and ExplicitDestruction

interfacesinterfaces Notification and authentication interfacesNotification and authentication interfaces

Mediator

LegendGenerator

MapAssembler

Ontology

…

GRID SERVICESFOR MAP INTEGRATION

ArcIMS and WMS Services wrapped in WSDL/SOAP

Querying multiple map servicesIn GEON (www.geongrid.org)

DEMO

Questions??Questions??