a geospatial approach to address data etl ... - esri canada · a geospatial approach to address...
TRANSCRIPT
![Page 1: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition](https://reader031.vdocument.in/reader031/viewer/2022022011/5b150a987f8b9ab6778d5288/html5/thumbnails/1.jpg)
P r e s e n t e d B y : D i l i p S e b a s t i a n
N o v e m b e r 1 5 , 2 0 1 6
A Geospatial approach to
Address Data ETL and Address
Data Management
![Page 2: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition](https://reader031.vdocument.in/reader031/viewer/2022022011/5b150a987f8b9ab6778d5288/html5/thumbnails/2.jpg)
ABOUT SASKTEL
• SaskTel is the leading Information and Communications
Technology (ICT) provider in Saskatchewan
• Our Mission:
• To provide the best customer experience through our
superior networks, exceptional service, advanced
solutions, and applications
• Headquarters in Regina, SK
• Esri user since 2006
![Page 3: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition](https://reader031.vdocument.in/reader031/viewer/2022022011/5b150a987f8b9ab6778d5288/html5/thumbnails/3.jpg)
WHY A NEW DATA MODEL?
• No single comprehensive source of Address data available to satisfy
SaskTel’s business needs.
• Drawbacks of current dataset:
• Lacked information about address type
• Civic Address and Rural Land location information went into the same fields
• Overlapping Address Points for multi-dwelling units
• Non relational
• No data validation
![Page 4: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition](https://reader031.vdocument.in/reader031/viewer/2022022011/5b150a987f8b9ab6778d5288/html5/thumbnails/4.jpg)
PROJECT OVERVIEW
• GIS data modelling using ESRI ArcGIS
• From a non-relational data set to relational data modelData Modelling
• ETL Workflows using FME ESRI Edition
• Create Junction tables through spatial analysisETL Process
• ESRI Add-In for Address Creation
• Python scripts for process automationTools & Scripts
• RESTful Services for search & geocoding
• Continual Improvement of internal address datasets
GOLD Source of Address
![Page 5: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition](https://reader031.vdocument.in/reader031/viewer/2022022011/5b150a987f8b9ab6778d5288/html5/thumbnails/5.jpg)
ADDRESS AT SASKTEL
• Address is defined by the following:
1. Urban, Rural or First Nations Address
2. Legal Parcel
3. Legal Land Description
Quarter Section
Legal Subdivision
Section 21
Township 16-20
West of 2nd Meridian
![Page 6: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition](https://reader031.vdocument.in/reader031/viewer/2022022011/5b150a987f8b9ab6778d5288/html5/thumbnails/6.jpg)
ADDRESS
Addre
ss
Urban
Single Unit
Multi Unit
Single Building
Multi Building
Lot-Block
Rural
Rural Route
First Nations
Quarter Section
Address Data Model.gdb
Address
Buildings
Communities
LegalParcels
LandLocation
SaskTelAttributes
Streets
BlocksHasLots
Community_Domain
LotsHasBuildings
Province_Domain
SubAddress
StreetType_Domain
![Page 7: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition](https://reader031.vdocument.in/reader031/viewer/2022022011/5b150a987f8b9ab6778d5288/html5/thumbnails/7.jpg)
COMMUNITY DATASET
• Over 3000 Saskatchewan Communities Geocoded
• Big leap in E911 process
• Unregistered and Registered Communities
co-exist with proper qualifiers.
Google Maps
SaskTel Data
![Page 8: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition](https://reader031.vdocument.in/reader031/viewer/2022022011/5b150a987f8b9ab6778d5288/html5/thumbnails/8.jpg)
ETL PROCESS
• Source Data: Old Address Dataset
• Regex Analysis
• Extract and copy Address data into proper fields
• Break down composite data into simple data
• Spatial Analysis
• Generate Business Attributes
• E911 Community
• Land base information
• Related tables for Address and Sub-Address data
• Data Cleansing
![Page 9: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition](https://reader031.vdocument.in/reader031/viewer/2022022011/5b150a987f8b9ab6778d5288/html5/thumbnails/9.jpg)
TOOLS & SCRIPTS
• Esri Add-In Tool for Address Creation
• Real-time data validation
• On-the-fly Spatial Analysis
• E911 information
• Land base
• Business Attributes
• Relationship between Address & SubAddress
• Geoprocessing Tools/Scripts
• Batch Geocoding
• Data Formatting
• Process Automation
![Page 10: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition](https://reader031.vdocument.in/reader031/viewer/2022022011/5b150a987f8b9ab6778d5288/html5/thumbnails/10.jpg)
ADDRESS CREATION TOOL
• Civic Address
• 1919 Saskatchewan Dr, Regina, SK
• 1060 Dorothy St, BLDG-A, Regina, SK
• Lot-Block Address
• BLK 5 LT 10 Fraser Ave, Craven, SK
• Legal Land Description
• SW15 13-23-W2
• First Nations Address
• HSE-1, Cumberland House IR, SK
![Page 11: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition](https://reader031.vdocument.in/reader031/viewer/2022022011/5b150a987f8b9ab6778d5288/html5/thumbnails/11.jpg)
GOLD SOURCE OF ADDRESS
• Decentralized Inventory system to Centralized GIS system
• RESTful services for field service management software
• E911
• Referenced to cleanse and improve address and address related data
• Geocoding Subscriber address data
![Page 12: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition](https://reader031.vdocument.in/reader031/viewer/2022022011/5b150a987f8b9ab6778d5288/html5/thumbnails/12.jpg)
ADDRESS & ARCGIS
Address Management
ArcGIS for
Desktop
Custom
Add Ins
Geoprocessing
Scripts
ArcGIS for
Server
Geocoding
Service
Routing
Service
Inventory
Management
Workforce
Management
![Page 13: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition](https://reader031.vdocument.in/reader031/viewer/2022022011/5b150a987f8b9ab6778d5288/html5/thumbnails/13.jpg)
SYSTEM ENVIRONMENT
Oracle
Geodatabase
ArcGIS for
Desktop 10.2.1
Address Data
ArcGIS for
Server 10.4
FME ESRI
Edition 2016
![Page 14: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition](https://reader031.vdocument.in/reader031/viewer/2022022011/5b150a987f8b9ab6778d5288/html5/thumbnails/14.jpg)
BENEFITS
• Data Quality Improvement
• Improved Productivity
• Increased Market Penetration
• Confident in addressing Customer needs
• Faster Service Provisioning
• Better Customer Service
• Centralized Address Data Management
• Better Analysis Capability
![Page 15: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition](https://reader031.vdocument.in/reader031/viewer/2022022011/5b150a987f8b9ab6778d5288/html5/thumbnails/15.jpg)
LESSON LEARNED
• Initially underestimated the complexity of Address Data
• Underestimated the time and effort required for Address Maintenance
• Recommendations:
• Do not delay your Address Management
• Know the advantages that GIS Address Management can provide
![Page 16: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition](https://reader031.vdocument.in/reader031/viewer/2022022011/5b150a987f8b9ab6778d5288/html5/thumbnails/16.jpg)
FUTURE PLANS
• Reconciling with legacy internal systems and drive continual
improvement within the corporation
• Educating internal groups about the advantages of GIS technology