esri uc 2014 | technical workshop | arcgis data reviewer edwin waite & shankar chandrasekaran...
TRANSCRIPT
Esri UC 2014 | Technical Workshop |
ArcGIS Data Reviewer
Edwin Waite & Shankar Chandrasekaran
Planning and Deploying Data Quality Services
Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |
A few examples of why data validation matters
Planning and Deploying Data Quality Services
Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |
Leaning Tower of Pisa
Planning and Deploying Data Quality Services
Esri UC 2014 | Technical Workshop |
Mars Climate Orbiter (1999)
Esri UC 2014 | Technical Workshop |
Garbage in Garbage Out (GIGO)
Type Presentation Title Here
Buildings
Roads
Water Bodies
Esri UC 2014 | Technical Workshop |
Garbage in Quality Out
Type Presentation Title Here
Buildings
Roads
Water Bodies
Esri UC 2014 | Technical Workshop |
What is ArcGIS Data Reviewer for Server?
• System to support data quality management for ArcGIS
• Automates quality control process
• Monitor and Measure data quality
ReviewValidate Summarize
Esri UC 2014 | Technical Workshop |
Enterprise Quality Management System
Type Presentation Title Here
GIS Data Server
Data Reviewer
Esri UC 2014 | Technical Workshop |
Benefits of Data Reviewer for Server
• Provides accurate data for your applications
• Measure and report data quality
• Eliminate costs associated with poor data quality
Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |
Monitoring your data with validation
Planning and Deploying Data Quality Services
Validate
Esri UC 2014 | Technical Workshop |
Automated ValidationDifferent Type of Issues in GIS Data
Type Presentation Title Here
Esri UC 2014 | Technical Workshop |
Automated Validation
• Checks validate data based on conditions you define.
• Results are written to the Reviewer workspace.
Esri UC 2014 | Technical Workshop |
Automated Validation
• Over 40 rules out of the box
• Ability to extend validation with custom rules
• Rules are authored using ArcGIS Data Reviewer for Desktop
Attribute/Table Feature integrity Spatial Relationships
Item Metadata
Esri UC 2014 | Technical Workshop |
• Group checks into batch jobs
Automated validation
Planning and Deploying Data Quality Services
Esri UC 2014 | Technical Workshop |
ArcGIS Server
Automated Validation Architecture
Type Presentation Title Here
Data Reviewer
Web API
REST
API
Reviewer Service
Scheduler
Validation Service
Data Workspace
Reviewer Workspace
Execution Details
Esri UC 2014 | Technical Workshop |
Automated Review Demo
Planning and Deploying Data Quality Services
Contributor
Organization
Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |
Questions ?
Planning and Deploying Data Quality Services
Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |
Analyzing Validation Results
Planning and Deploying Data Quality Services
Esri UC 2014 | Technical Workshop |
• Dashboard- Provides transparency to stakeholders on data quality
- Ability for stakeholders to make informed and accurate decisions
- Interactively summarize results based on different perspectives
- Summaries are derived from results
Summarize data quality
Planning and Deploying Data Quality Services
Esri UC 2014 | Technical Workshop |
Summarize data qualityDashboard - Perspectives
Planning and Deploying Data Quality Services
What are the issues on a geographic extent?
How severe are the issues with my data?
How many types of issues do I have in my data?
How many issues have been reviewed and fixed?
How many issues do I have by feature class?
Esri UC 2014 | Technical Workshop |
Dashboard
Planning and Deploying Data Quality Services
Data Reviewer
Web API
ArcGIS Server
Reviewer Workspace
Dashboard
Reviewer Service
REST
API
Esri UC 2014 | Technical Workshop |
• Results are records that are stored in Reviewer workspace
• Contains information about a feature or a row which failed QC standard
• Query Results- Apply filters
- Sort/Page
Results
Planning and Deploying Data Quality Services
ResultsAutomated Validation
Esri UC 2014 | Technical Workshop |
Results
Planning and Deploying Data Quality Services
• Spatial and non-spatial
• Attributes describe check, problem, and validation information
• Object ID provides link back to feature that failed validation
Esri UC 2014 | Technical Workshop |
ArcGIS Server
Results
Planning and Deploying Data Quality Services
Data Reviewer
Web API
REST
API
Reviewer Service
Results
Reviewer Workspace
Esri UC 2014 | Technical Workshop |
Analyze Results demo
Planning and Deploying Data Quality Services
Contributor
Data Stewards
Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |
Questions ?
Planning and Deploying Data Quality Services
Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |
Visual Quality Control
Esri UC 2014 | Technical Workshop |
Visual Review Process
Planning and Deploying Data Quality Services
Reviewer Service
Write Results
Editor
ArcGIS Server
Reviewer
Geodatabase
Esri UC 2014 | Technical Workshop |
• Get feedback from Subject Matter Experts (SME)
• Identify missing or incomplete data
• Track issues reported
Benefits of Visual Review
Type Presentation Title Here
Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |
Demo
Visual Quality Control
Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |
Questions ?
Planning and Deploying Data Quality Services
Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |
Deploying Data Quality Services
Planning and Deploying Data Quality Services
Esri UC 2014 | Technical Workshop |
Deploying Data Reviewer ServicesSingle machine deployment
Planning and Deploying Data Quality Services
Server Data WorkspaceReviewer Workspace
Desktop Web Device
Esri UC 2014 | Technical Workshop |
Deploying Data Reviewer ServicesSingle machine Deployment
Planning and Deploying Data Quality Services
ArcGIS Server Single Machine
Inspect and Analyze Map Service
Validate GP Service
Inspect and Analyze Map Service
Inspect and Analyze Map Service
Validate GP Service
Validate GP Service
Esri UC 2014 | Technical Workshop |
Deploying Data Reviewer ServicesMulti-machine deployment
Planning and Deploying Data Quality Services
Data WorkspaceReviewer Workspace
Desktop Web Device
Servers
Esri UC 2014 | Technical Workshop |
Deploying Data Reviewer ServicesMulti-machine deployment
Planning and Deploying Data Quality Services
ArcGIS Servers
Inspect and Analyze Map Service
Validate GP Service
Inspect and Analyze Map Service
Validate GP Service
Inspect and Analyze Map Service
Validate GP Service
Esri UC 2014 | Technical Workshop |
Deploying Data Reviewer ServicesAmazon Elastic Cloud Deployment
Planning and Deploying Data Quality Services
Desktop Web Device
Data Workspace
Reviewer Workspace
Esri UC 2014 | Technical Workshop |
Deploying Data Reviewer ServicesAmazon Elastic Cloud Deployment
Planning and Deploying Data Quality Services
Inspect and Analyze Map Service
Validate GP Service
Inspect and Analyze Map Service
Validate GP Service
AGS - Amazon EC
AGS - Amazon EC
Esri UC 2014 | Technical Workshop |
Data Reviewer ServerOut of the Box
• Server components- Execute Batch Job GP Service
- Reviewer Service
- REST API
• Web components - Client API Reference (JavaScript & Flex)
- Starter Applications (Batch Validation Manager)
- Widgets (Flex & JavaScript)
Planning and Deploying Data Quality Services
Execute BatchJob GP Service
Reviewer Service
Esri UC 2014 | Technical Workshop |
Data Reviewer Platform
Planning and Deploying Data Quality Services
• ArcGIS Data Reviewer for Desktop- Define and author Quality Control Standards- Configure Reviewer Workspace to store results
• ArcGIS Data Reviewer for Server- Schedule and run automated validations- Analyze result stored in Reviewer workspace - Submit SME feedback for Reviewer workspace for visual review
• Client API’s - Develop applications to access data quality services
Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |
Widgets and Starter Apps - Demo
Planning and Deploying Data Quality Services
Esri UC 2014 | Technical Workshop |
ArcGIS Data Reviewer ServerSystem Architecture
Planning and Deploying Data Quality Services
Data WorkspaceBatch Validation
Dashboard
Visual Review
HTTP/REST
Reviewer Workspace
Results
Validation Service
Esri UC 2014 | Technical Workshop |
• Product page- www.esri.com/datareviewer
• Resource Center- http://resources.arcgis.com/en/communities/data-reviewer/
• Training- www.esri.com/training
• Questions & comments- [email protected]
Resources
Planning and Deploying Data Quality Services
Esri UC 2014 | Technical Workshop |
Resource Center – Data Reviewer
• Gallery - Apps
- Templates
- Tutorials
• API Documentation
• Live Samples
Day and Description Type Time Location
Tuesday July 15
ArcGIS Data Reviewer: An IntroductionTechnical Workshop
8:30 – 9:45am Room 14A
ArcGIS Data Reviewer: Extending the Validation Framework Demo Theater 9:30 – 10:00amGeodata
Mgmt Theater
ArcGIS Data Reviewer: Leveraging Geoprocessing for Data Validation Demo Theater 10:30 – 11:00amGeodata
Mgmt Theater
ArcGIS Data Reviewer: Assessing Positional Accuracy Demo Theater 1:00 – 1:30pmGeodata
Mgmt Theater
ArcGIS Data Reviewer: Plans for Quality Assessment of Raster Datasets
Demo Theater 5:30 – 6:00pmGeodata
Mgmt Theater
Wednesday July 16
ArcGIS Data Reviewer: An IntroductionTechnical Workshop
8:30 – 9:45am Room 14A
ArcGIS Data Reviewer: Integrating ArcGIS Data Reviewer and ArcGIS Workflow Manager to Automate Data Quality Control Workflows
Demo Theater 11:30 – 12:00amGeodata
Mgmt Theater
ArcGIS Data Reviewer: Special Interest Group Meeting SIG 12:00 – 1:00pm Room 15B
ArcGIS Data Reviewer: Planning and Deploying Data Quality ServicesTechnical Workshop
3:15 – 4:30pm Room 15B
Thursday July 17
ArcGIS Data Reviewer: Integrating Data Validation Capabilities Into Web Applications
Demo Theater 9:30 – 10:00amWeb & Server
Theater
ArcGIS Data Reviewer: Implementing Data Quality Reporting in Web Clients
Demo Theater 10:30 – 11:00amWeb & Server
Theater
Questions ?
Planning and Deploying Data Quality Services
Esri UC 2014 | Technical Workshop |
Thank you…
• Please fill out the session survey:
Offering ID: 1313
Online – www.esri.com/ucsessionsurveys
Paper – pick up and put in drop box
Planning and Deploying Data Quality Services
Esri UC 2014 | Technical Workshop | Planning and Deploying Data Quality Services