from oracle to the web - automating spatial data updates
DESCRIPTION
TRANSCRIPT
From Oracle to the Web - Automating Spatial Data Updates David Smith, FRS Program Manager, EPAAmy Ramsdell, GIS Manager, Blue Raster
April 8
Overview
The Need: Automate spatial data refresh The Solution: Technologies used ArcGIS Server (AGS) map service final output FME workspaces that automate refresh process
File geodatabase (fgdb) creation Metadata refresh
Python shutdown script in workspace FME Server web services used EPA future plans for FME Questions
The Need:
Automate refresh of map service serving spatial data
Operational database system The Facility Registry Service (FRS) is a centrally managed
database that identifies facilities, sites or places subject to environmental regulations or of environmental interest.
Disconnected static file for map service Fgdb faster performance in AGS Lessen activity on operational database
Monthly manual refresh
The Solution:
ESRI ArcGIS Server Final Output
Web server hosting GIS web services
REST endpoint for map servicehttp://geodata.epa.gov/ArcGIS/rest/services/OEI/FRS_INTERESTS/MapServer
Underlying data: file gdb 28 layers (tables in fgdb)
Data Refresh FME Workspace
Oracle Spatial reader
ESRI File Geodatabase writer
Whole bunch of filtering in-between to create separate tables in fgdb Could build the
filtering in Oracle before it reaches FME
Oracle Spatial Type Reader
Required for spatial object: Table, materialized view, or view
Shape field is SDO_GEOMETRYCREATE TYPE sdo_geometry AS OBJECT ( SDO_GTYPE NUMBER, SDO_SRID NUMBER, SDO_POINT SDO_POINT_TYPE, SDO_ELEM_INFO SDO_ELEM_INFO_ARRAY, SDO_ORDINATES SDO_ORDINATE_ARRAY);
USER_SDO_GEOM_METADATA table entry Metadata for spatial tables owned by schema
ESRI File Geodatabase API Writer
Non-ArcObjects access to fgdb
C++ API
Does not require an ArcGIS license on FME Server or Desktop
Works in ArcGIS Server 10x
Python Shutdown Script in Workspace
In Workspace Parameters – Advanced option
Shutdown Script in 3 Steps
1. Package the output zipfgdb(geodatabase, path)
2. Refresh the map service updateArcGISServer(server, services, _ZIP,
_ZIPDESTINATION)
3. Notify dependent subscription workspace NotifySuccess()
Shutdown Script: 1. Package the Output
Zip the file geodatabase
Shutdown Script: 2. Refresh Map Service
1. Stop map service
2. Delete current fgdb
3. Copy to shared network drive
4. Extract new fgdb
5. Start map service
1
2
4
3
5
Refresh Map Service: Stop/Start Services
Stopping and starting AGS map service
Refresh Map Service: AGS Token Access
Generate token for admin access
Refresh Map Service: Replace Fgdb
Delete existing fgdb and extract zip
Shutdown Script: 3. Notify Subscription
FME Server
FME Notification Service - Topics
Create a Success and a Fail notification topic to perform some action based on the data refresh workspace execution status
FME Notification Service - Subscriptions 2 subscribers
Email service to notify when the workspace fails Metadata workspace runs if data refresh workspace completes
successfully
Metadata about Map Service
FGDC – Compliant metadata required in XML format XML file updated for date of fgdb refresh
https://edg.epa.gov/
Subscriber: Metadata Refresh Workspace
EPA Future Plans for FME
Kick off workspace from Oracle instead of manually in FME Server’s Services interface – good for more frequent refresh cycles than monthly using FME’s Job Submitter service http://
servername.com/fmejobsubmitter/Samples/workspacename.fmw?FMEServerHost=http://servername.com&SuccessTopic=InternetFGDB&Token=4fab5b76c6yufed2c4731s3d0387bfb366569949&FailTopic=InternetFGDBFail&UserName=admin&Password=password&opt_showresult=false&opt_servicemode=sync
Possibly enhance FRS Query Tools by adding spatial selections and making more file formats available
Integrate into other EPA program map service refresh flows
Research FME’s ability to translate Resource Description Framework (RDF), a semantic web data format
EPA Facility Registry Services (FRS)
Thank You!
Questions? For more information:
David Smith, EPA Facility Registry Service [email protected]
@DruidSmith
Judith Doherty, FRS Project Manager, Indus Corporation [email protected]
Amy Ramsdell, Blue Raster [email protected]
www.blueraster.com/blog @AmyRams