workshop using streaming geospatial data resources workshop : using streaming geospatial data...
Post on 18-Dec-2015
218 views
TRANSCRIPT
WorkshopWorkshop:
Using Streaming Geospatial Data Using Streaming Geospatial Data ResourcesResources
IASSIST / IFDO 2005May 24, 2005Edinburgh, ScotlandUK
Workshop: Objectives
Objectives Introduction to some key Open Geospatial Consortium (OGC)
technologies Hands-on use of some OGC services and clients Highlight problems related to integration of these services with
data services operations & libraries Not Objectives
Learn all the relevant technologies in detail Learn how to use specific proprietary desktop GIS software or
how to “do GIS” Provide real concrete answers about how to integrate these
services with data services operations & libraries
Proprietary web map services ArcIMS, etc. -- map servers delivering mapping functionality to
Web browser clients or to desktop GIS software
Proprietary spatial database services ArcSDE, Oracle spatial, etc. – database servers delivering data to
desktop GIS software … or to map server applications
Open Geospatial Consortium-compliant services Web Map Services, Web Feature Services, etc. Focus on data and service interoperability OGC specifications also cover other services besides data delivery
Overview: What is streaming geospatial data?
Why Streaming Data?
Size of geospatial data resources Size of aerial photography, OSMM GML
Frequency of data update Seconds, hourly, daily, weekly, monthly, annual …
Flexible access to data Low (er) bandwidth access Subset, analyse, inspect data
New licensing methods BUT – DRM issues (see later)
Why Interoperability?
Integrate existing systems
Create distributed network of interoperable data stores and application services
Distributed multiagency/ multiorganizational/ multidepartmental sharing (ideally)
Application fusion, bring together multiple GIS applications using geography as the integrated framework (ideally)
Non OGC-compliant map servers are data islands Data from different servers cannot be combined
Data available in many vendor or non-proprietary formats Conversion time consuming, confusing
Data available in different coordinate systems Conversion time consuming, confusing
Service chaining Make services that add value to other services
Why Interoperability?
Web Services Architecture
ServiceRegistry
ServiceProvider
ServiceConsumer
PublishPublish FindFind
BindBind
WebWebBrowserBrowser
UDDIUDDIWSDLWSDL
WebWebServerServer
How?
OGC and Web Services
Open Geospatial Consortium was working on developing geospatial Web services before “Web Services” (SOAP/WSDL/UDDI/…) existed – focus on modular, reusable components
Because of early start OGC is now working to adapt OGC specifications to mainstream Web Services architecture (or vice versa), i.e. making OGC services build on top of standard Web Services frameworks
OGC use of HTTP GET & POST for invocations is a constraint on acceptance of OGC specs in Web Services community
See OGC Web Services, Phase 2 (OWS-2) & Phase 3 (OWS-3):
http://www.opengis.org/initiatives/?iid=7
OGC Technology Overview/Timeline
Early 1990’s: “Sticky tape and pipe cleaners” map server implementations—homegrown
Circa 1995-1996: Commercial software products become available—no interoperability
Circa 1998-2000: Commercial map server products stream data to workstation software (no interoperability) e.g. ArcIMS
Circa 2000-2001: WMS 1.x implementations, first home-grown, later commercial and open source
Circa 2002-2003: WFS 1.0 implementations
Open Geospatial Consortium Overview
The Open Geospatial Consortium (OGC) is a not-for-profit, international consortium whose many members work to make GIS an integral part of information systems of all kinds.
Operates a Specification Development Program that is similar to other Industry consortia (W3C, OMG, etc.).
Also operates an Interoperability Program (IP), a partnership-driven engineering and testing program designed to deliver proven specifications into the Specification Development Program.
OGC used to talk about “web-enabling GIS”, now they talk about “geo-enabling the web.”
Advantages of OGC Spec Adoption
OGC interface specifications are open, freely downloadable documents that can be applied and tested widely.
Commercial and non-commercial software can support these interfaces for interoperability and information exchange (and many now do).
Adopting common service interfaces lets solutions use the same software to connect to systems back-ended by different GIS vendor solutions (write once use many times).
Significant vendor participation in specification development: they get to help drive direction of specification development—and interoperability helps them to vertically integrate in industry i.e. good for vendors and good for consumers.
OGC Specifications (some!!)
Web Map Service (WMS) Specifies the behavior of services that return georeferenced
maps Web Feature Service (WFS)
Interface that supports query level access to vector data repositories
Web Coverage Service (WCS) Supports networked interchange of geospatial data as
‘coverages’ containing values or properties of geographic locations
Styled Layer Descriptor (SLD) Styling language for WMS map servers, allow for pre-defined
and user defined styles
OGC Technology Overview
Catalog Services (CAT) Support the ability to publish and search collections of metadata
for data, services, and related information objects Web Registry Services (WRS)
Defines a common mechanism to classify, register, describe, search, maintain and access information about OGC Web resources
Web Map Context (WMC) (Ver 1.1 now an adopted specification) Like a ‘spatial bookmark’, allows recreation of a map using OGC
services More (lots!)… see: http://www.opengis.org/specs/
Web Map Server Specification 1.1.1
GetCapabilitiesReturns server-level metadata, description of services and content,
acceptable request parameters GetMap
Returns map image whose geospatial and dimensional parameters are well-defined
GetFeatureInfoReturns information about particular features shown on map (optional)
Specification defines syntax for URLs to invoke map operations. XML encoding defined for service
level metadata.
Rendered Map Image
Returns info about feature
Capabilities XML Document with
Layer Information
Legend Roads Water BuildingsMap Layers
Web Map Service
GetCapabilities
GetMap
WMS Interface: Operation
Client
Adapted from Web Map Server Demonstration presentation © 2000 OGC
GetFeatureInfo
WMS Map Requests
Layers
Styles (possibly)
Bounding Box
Projection or geographic coordinate reference system
Desired output format
Output size (width, height)
Background transparency or color
Sample WMS Query
http://anyoldserver.com/WebMapService?version=1.1.1&
layers=Raster50K&styles=default&
srs=EPSG:27700&bbox=287000,692001,290000,695001&
width=400&height=400&
format=image/gif&
request=GetMap
Asks the mapserver what functionalities a particular WMS map services is capable of providing, such as:
– supported interfaces (does it support GetFeatureInfo?)– map layers (what are they?)– symbolization (are there named styles available?)– geographic extent (what part of the world is this?)– Spatial Reference System (SRS) (which are supported?)– keywords (for topic searches)– supported image formats (supports transparency?)
The Capabilities Interface is required.
WMS GetCapabilities
Capabilities function as “service” metadata, providing information about what the WMS map service offers.
Clients determine whether or not they can work with that server based on its capabilities. WMS servers should be compatible with all versions of the standard earlier than the one it advertises.
The standard supports a lot of optional tags that enhance the usefulness of a Capabilities file - they are not all mandatory.
http://anyoldserver.com/WebMapService?request=GetCapabilitiesrequest=GetCapabilities&version=1.1.1&service=WMS
WMS GetCapabilities
Support of this interface is optional
Returns information about a point on the map, similar to an “identify” operation in GIS
Layer must be set to queryable in Capabilities XML
Default format of request returned should be the MIME format for GML – application/vnd.ogc.gml
WMS is stateless so original GetMap request parameters are included
WMS service provider has some flexibility in how to return results – lots of variation here
WMS GetFeatureInfo
URL Prefix VERSION REQUEST Map-Request-Copy Query_Layers X Y
(Info_Format) (Feature_Count) (EXCEPTIONS) (Vendor Specific Parameters - VSP)
GetFeatureInfo Request Parameters
(Optional parameters in parentheses)
GetFeatureInfo Request
http://anyoldserver.com/WebMapService?version=1.1.1&
request=GetFeatureInfo&layers=cs_ng_10560_1&
styles=&srs=EPSG:27700&
bbox=332319,502462,333656,503799&width=600&height=600&
format=image/gif&query_layers=cs_ng_10560_1&
info_format=text/xml&feature_count=999&
x=258&y=270&
http://dearg.ucs.ed.ac.uk:8081/opengis/wms/historic/WebMapService?version=1.1.1&request=GetFeatureInfo&layers=cs_ng_10560_1&styles=&srs=EPSG:27700&bbox=332319.067428162
85,502462.06742816285,333656.93257183715,503799.93257183715&width=600&height=600&format=image/gif&query_layers=cs_ng_10560_1&info_format=text/
xml&feature_count=999&x=258&y=270&exceptions=application/vnd.ogc.se_xml
The optional DescribeLayer operation applies only to a WMS that supports Styled Layer Descriptors (SLD)
It allows the user to request an XML description of one or more map layers
DescribeLayer Request
Image Services vs. Feature Services
Image Services Deliver image only (snapshot of the data), not actual feature (vector)
data More limited functionality—can’t export locally; changing symbology
more limited Lower bandwidth requirement
Feature Services Streams actual feature data over the Internet, available as individual
feature classes Allows extra functionality of changing symbology or exporting the data
locally; cleaner maps Higher bandwidth requirement Filtering; more sophisticated linking
Geography Markup Language 2.x-3.x
XML encoding for transport and storage of geographic information, including spatial and non-spatial properties of features
Defines the XML schema syntax, mechanisms, and conventions that provide open, vendor-neutral framework for the definition of geospatial application schemas and objects
Allows profiles that support proper subsets of GML framework descriptive capabilities
Support schemas for specialized domains
Support storage and transport of application schemas and datasets
Increase ability to share schemas and information
3.0 latest; 2.1.2 used with WFS (and in OSMM)
Web Feature Service 1.0
A WFS allows a client to retrieve actual geospatial data (vectors, not just an image) encoded in Geography Markup Language (GML 2.1) from multiple Web Feature Services.
A WFS request consists of a description of query or data transformation operations that are to be applied to one or more features.
The request is generated on the client and is posted to a WFS server using HTTP.
The WFS server then reads and (in a sense) executes the request and returns the resulting dataset in GML format.
Basic WFS servers provide Read-Only access to the data.
Transactional WFS servers actually allow clients to manipulate the data, including:
Create a new feature instance Delete a feature instance Update a feature instance Get or Query features based on spatial and non-spatial constraints
Web Feature Service 1.0
Differences between WFS and WMS capabilities (Nuke Goldstein Oct 2004)
http://www.directionsmag.com/article.php?article_id=686&trv=1
Conventional Library Geospatial Information Services
MapCollections
DataCollections
MapServers
MapPortals
• Map Collections– Paper Maps – Digital maps (e.g. Digimap)
• Data Collections– CD-ROMs, File server & FTP access– Web based (e.g. Census Dissemination Unit)
• Map Servers– Integrate collected data, Web-based mapping (e.g. EDINA)
• Coming: Map Portals– ‘Discovery’ portals (e.g. Go-Geo!)– Front end to distributed, streaming data (OpenGIS)
User Considerations for OGC Services
• Advantages– Time of need access– Streaming access (needed subset only, by location)– Most current data– Combine data from multiple sources
• Disadvantages– Reliability?– Effect of ongoing changes in underlying data on project work?– Image services (WMS) lack data intelligence, use as background
but not for analysis– Directory and catalog services not yet mature (services are often
moving targets)
When are streaming data services useful?
• By content type– Very large data files (e.g., ortho-imagery)– Very quickly changing data (e.g. ward boundaries)– Data not typically acquired by affiliated institution (other
regions/countries)• By user role
– Use of data as background or reference layer (e.g., ortho imagery, digital topo / historic maps)
– Preliminary analysis of data prior to acquisition or download– Casual user access and vertical integration– Geospatial web services as part of service chain (e.g., use to
geocode/geoparse documents)
Problem: Service Discoverability/Selection
Geospatial Web Services are in a “Wild West” stage• Difficult to locate them• Difficult to assess content• Services come and go (demonstration services)• Variable adherence to standard; proprietary hooks
Disconnect between object metadata (e.g., FGDC metadata for dataset) and service metadata (e.g. OGC Capabilities file)-can have unidirectional link from service metadata to object
Catalog Services ( CAT )
Support the ability to publish and search collections of metadata for data, services, and related information objects
Discovery Services allows clients to locate metadata that describes GIS data
Access Services provides the client with methods to request services on the data, such as get a map
or download data, etc
Management Services Management Service defines methods for a client to change the metadata held by a
catalog.
Catalog Interface enables diverse but conformant applications to perform discovery, browse and query
operations against distributed and potentially heterogeneous catalog servers.
Problem: Numeric Data Integration
Early OGC specifications and services are relatively map- and feature-centric—service content relatively “dumb” from the point of view of attribute/tabular data integration
How can geospatial web services be integrated with numeric data stores in a web services environment?
Discussion docs: Geo-linked Data Access Service (GDAS) (http://portal.opengis.org/files/?artifact_id=5858)
& associated Geo-Linking Service (GLS) (http://portal.opengis.org/files/?artifact_id=5859) Demo implementation at: http://wms1.agr.gc.ca/gdas/gdas.phtml
Problem: Who Will Preserve the Data?
History of lack of interest in long term access and data preservation on the part of data producers
Market for older data versions too small to be driving force in time series development (they don’t ask for it until its too late)
NC OneMap: “Historic and temporal data will be maintained and accessible”—not yet clear how that will be done
Context of current JISC repository call and ongoing Go-Geo! Phase 4b work
Preservation
Secondary data archives are often developed as a by-product of providing data access (needed to get the data to give the data)—and secondary archives are often the ones that survive over time
If we no longer need to get the data to provide access to the data, is there as much incentive to build secondary archives
… on the other hand, data acquisition is time consuming, especially for local data. Can we use emerging Web Services technologies to automate the process of secondary archive construction? (e.g. agent-based trolling of capabilities files for new/updated data, then agent-negotiated transfer)
An Opportunity: Cascading Map Servers
Web Mapping Service that behaves like a client of other WMS servers and behaves like a server to other clients
When two or more maps are produced with same bounding box, spatial reference system, and output size, results can be accurately layered to produce a composite map
Can perform additional functions such as output format conversion or coordinate conversion for other servers
Enables creation of distributed map servers from which clients can build customized maps
Cascading Map ServersCascading Map Server
Web Browser
MapServer
Cascading Map Server MapServer
MapServer
MapServer
CapabilitiesIntegrator
MapIntegrator
JPEGWGS 84
GIFNAD 83
GIFWGS 84
Different viewer client in browser!
inte
rnet
inte
rnet
Adapted from Web Map Server Demonstration presentation © 2000 OGC
Why Cascading Map Servers?
Running a map server that locally hosts other peoples data is a recipe for stale data
Cascading map server leaves data management to data producer—deliver most current data
Library or data service defined cascading map servers might allow us to do with we do best: build a window to the world of information where the window is configured to best meet the needs of our target community
Cascading Map Servers: Software
Emerging software products from companies like Cubewerx, Ionic, Compusult (limited cascading support in open source Minnesota Map server)
Software not yet mature; niche companies leading the way on strictly OGC-compliant software (e.g. Cubewerx CubeSERV) http://www.cubewerx.com/main/products/CubeSERV_WMS.html
Proprietary hooks in some major vendor OGC connectors might complicate consumption of services by cascading map server
There are technical reservations however, e.g. how to specify plot order and reliably fuse 3rd party servers (SLAs/SLDs?)
Community Building: NC OneMap
… As informally discerned from NC OneMap training materials Agreements
• WMS version: 1.1.0 (for now)• Background color and transparency handling• Image preference: GIF followed by PNG• Make data available in geographic coordinates (most originates in some flavor of State
Plane)• Agree to call parcels “Parcels” and to use specific symbology• Registration process: Manual, with Capabilities files sent to USGS
Guidance• 1:12,000 to 1:15,000 scale cutoff for orthos and parcels (in process of working out
detailed guidance)• Encouragement to include layer metadata• Single word layer names• Load Capabilities DTD locally (service chain failure issues)• Security recommendations• Working on procedure for creating legend image swatches and method for making
legends available (no SLD support in phase 1)
Building WMS Servers
Add-ons to commercial map server software (e.g. ArcIMS from ESRI)
Dedicated commercial software (e.g. Cubewerx)
Open source/freeware (e.g University of Minnesota Map Server)
Write your own (we did!!)
Follow the yellow brick road… (http://www.intl-interfaces.com/cookbook/WMS/)
+See Selected Resource handout for resources/links
OGC Directions
OGC Web Services 2 Initiative Web Services integration - SOAP/WSDL,UDDI, etc Technical baseline maturation
OGC Web Services 3 Initiative ‘Threads’ OGC Location Services (Open LS) OGC Web Sensor Enablement (SWE) OGC Common Architecture Geo-Decision Support (GeoDSS) Geo-Digital Rights Management (GeoDRM)
geoDRM – conceptual deployment
WMS
I&A
WFSClient
GetMap
GetFeature
Register
Login
Accept License
Accept License
GetMap
WMS
GetIdentity
VerifyIdentity
GetFeature
LicenseManagement
LicenseManagement
Conclusions: Technology
OGC technologies quickly evolving and gaining wide vendor acceptance (if sometimes inconsistently)
Reliability is still a question, especially in service chaining
With regard to service discovery and selection its still sort of like the Wild West—need to see Catalog Services and Web Registry Services fleshed out and widely adopted—need to flesh out library/data services roles in technical developments
Integration of OGC technologies with mainstream Web Services an important step
Conclusions: Data Services Implications
To what extent are streaming data services useful to users? Service reliability is a question Access to very large image resources as background is convenient—
in general convenient for background
Geospatial web services links with numeric data not well developed Opportunities exist to help make services more discoverable
(registries; catalog services) Need to integrate services with collections How do we link objects with services that act upon them? (parallels
in numeric data community with SDA’s etc.?) Cascading map servers as new service opportunity
OGC Client Software – ESRI-centric list
OGC Interoperability Add-on for ArcGIS
http://www.esri.com/software/standards/interopdownload.html Cubewerx Extensions for ArcGIS and MapInfo
http://www.cubewerx.com/main/about_us/news/WMS_ext.html
ARC3WMS (ArcView 3.x WMS Client)http://www.refractions.net/arc3wms/
Demo EDINA adapter
ArcExplorer Webhttp://www.esri.com/software/standards/interopdownload.html#aeweb
CadCorp MapBrowserhttp://www.cadcorp.com/products_geographical_information_systems/map_browser.htm
UDIG (User-friendly Desktop Internet GIS)
http://udig.refractions.net/downloads/udig1.0.RC4.exe
Gaiahttp://www.thecarbonproject.com/products/gaia.html
MapInfo (7.8)http://extranet.mapinfo.com/products/Overview.cfm?productid=1044
AutoCAD
http://www.ime.co.uk/WMS/
OGC Client Software – some others
Catalog Services: Products
Red Spider Catalog2 (Ionic)http://www.ionicsoft.com/products/main_catalog.jsp
deegree (open source)http://deegree.sourceforge.net/
terraCatalog (Con terra)http://www.conterra.de/software/terraCatalog/index_engl.shtm
Indicio (Galdos)Http://www.galdosinc.com/index.html
Registries / Catalogs
Cascading Map Servers: Examples
USGS National Map
http://nationalmap.usgs.gov
NC OneMap
http://www.nconemap.com
FEMA Hazardmaps.gov
http://hazardmaps.gov
OGC Server Software -- Commercial
Cubeserve WMS Cascading Map Server (Cubewerx)
http://www.cubewerx.com/main/products/CubeSERV_WMS.html
Cartalinea (Galdos)
http://www.galdosinc.com/index.html
Red Spider (Ionic)
http://www.ionicsoft.com/products/specRedSpiderEnterprise.jsp
Moximedia Internet Mapping Framework
http://www.moximedia.com/ows.html
Cadcorp SIS
http://www.cadcorp.com/index.htm
Intergraph WMS Clientwww.wmsviewer.com
OGC Server Software -- Connectors
WMS Bridge – ArcIMS WMS Connector (Compusult)
http://www.compusult.net/cslt_prod_dm_wmsb.html
ArcIMS WMS and WFS Connectors (ESRI)http://www.esri.com/software/standards/product-support.html
OGC Server Software – Open Source
Minnesota Map Server
http://mapserver.gis.umn.edu
Geoserver
http://geoserver.sourceforge.net/html/index.php
deegree
http://deegree.sourceforge.net/index.html
For a list of products compliant to or implementing OGC specs or interfaces see:
http://www.opengis.org/resources/?page=products
Example Server Implementations
Federal• National Map http://www.nconemap.com
• FEMA HazardMaps http://hazardmaps.gov
State and Local• NC OneMap http://www.nconemap.com
• NCMapNet http://www.ncmapnet.com
• Louisiana Atlashttp://atlas.lsu.edu/central/displayOGCWFSFAQ.htm
University• Harvard University WMS (Distributed) Servers http://portal.opengis.org/files/files/?artifact_id=5908