arcgis data reviewer: an introduction · workshop scenario my organization needs to improve data...

ArcGIS Data Reviewer: An IntroductionJay Cary, Kumar Gaurav

Workshop agenda

• Defining data quality

• What is ArcGIS Data Reviewer?

• Automated review

• Semi-automated review

• Error management and reporting

• Summary/resources

ArcGIS Data Reviewer: An Introduction

Defining data quality


Defining qualityA technical perspective

Spatial Accuracy

ISO-19157:2013 Geographic information – Data quality


Lake feature has been shifted


Spatial Accuracy

Thematic Accuracy



Swimming pool captured as wetland


Spatial Accuracy

Thematic Accuracy

Completeness



Neighborhood with missing building footprint


Spatial Accuracy

Thematic Accuracy

Completeness

Logical Consistency



Highway with road surface type gravel


Spatial Accuracy

Thematic Accuracy

Completeness

Logical Consistency

Temporal Quality



Updated chart should show closed runway

Outdated chart showing open runway


Spatial Accuracy

Thematic Accuracy

Completeness

Logical Consistency

Temporal Quality

Usability



Used to map National Parks

Used to route emergency vehicles

Defining qualitySources of data quality requirements


Industry

standards /

specifications



Subject matter

experts

Industry

standards /

specifications



Subject matter

experts

Industry

standards /

specifications

Training and

experience



Subject matter

experts

Industry

standards /

specifications

Quality

assurance plan

Training and

experience

Data quality managementCapabilities of ArcGIS


Geodatabase integrity

• Schema constraints

• Attribute rules

• Versioning

Advanced data types

• Topologies

• Parcel fabric

• Utility network

ArcGIS

Data Reviewer

• Automated review

• Semi-automated review

• Error management

• Quality reporting

What is ArcGIS Data Reviewer?Data quality management in ArcGIS

• ArcGIS Desktop

- ArcMap extension

- ArcGIS Pro extension

• ArcGIS Server

• Data Reviewer API

- JavaScript

- ArcGIS Pro SDK for Microsoft .NET

• Web AppBuilder for ArcGIS


Apps

Desktop

APIs

Enterprise

Managing quality controlQuality control processes


Automated

review

Quality

reporting

Semi-automated

review

Implementing

automated review



Automated

review

Quality

reporting

Semi-automated

review


Types of Quality Control

Fast

Consistent and

repeatable

Objective

100% coverage

Automated

review


After data creation

During data creation

Automating data review

• Implementing quality requirements

- 40+ configurable checks

- Feature integrity

- Collection rules


Cutbacks Duplicate Vertex

Evaluate Part Count

Invalid Geometry

Monotonicity Polyline/Path Closes Self




- Feature integrity

- Collection rules

- Attribution

- Feature and table values


Domain Query Attributes

Regular Expression

Subtypes Table to Table Attribute

Unique ID




- Feature integrity

- Collection rules

- Attribution


- Spatial

- Spatial relationships


Duplicate Feature Event on Event Feature on Feature

Find Dangles Polygon Overlap/Gap is

Sliver

Unnecessary Polygon Boundaries




- Feature integrity

- Collection rules

- Attribution


- Spatial

- Spatial relationships


http://esriurl.com/12379

Implementing data quality rulesReviewer Rule implementation workflow

Authoring Reviewer Rules

- Identify data quality requirements

- What ArcGIS capability supports the requirement?

- What workflow will the capability support?

Industry

standards /

specifications

Subject matter

experts

Training and

experience

Quality

assurance

plans


Validation

Constraint

Implementing data quality rulesReviewer Rules in constraint workflows

• Attribute (constraint) rules

- Created using ArcGIS Pro

- Validates data sources

- Geodatabase features/tables

- 9 check types

- Rules stored in Geodatabase

- Supported in

- File Geodatabase

- Enterprise Geodatabase

- Feature Services


Implementing data quality rulesReviewer Rules in validation workflows

• Map rules

- Created using ArcGIS Pro

- Validates layers


- Shapefiles

- Feature services

- 10 check types

- Rules stored in Map

- Supported in

- ArcGIS Pro


Implementing data quality rulesReviewer Rules in validation workflows

• Reviewer Batch Jobs

- Created using ArcMap

- Validates data sources


- Shapefiles

- 43 check types

- Rules stored in a file

- Supported in

- ArcGIS Desktop

- ArcGIS Pro

- ArcGIS Server


Demonstration

Authoring

data quality rules


Workshop scenario

My organization needs to improve data quality to support future automation of task for finding

and routing to locations.

Success criteria

❑ Future data quality requirements are integrated into existing data management workflows

❑ Existing and new features should comply with new quality requirements

❑ Features should be accurately positioned

❑ All errors are corrected and verified


Demo Scenario

Industry

standards /

specifications

Subject

matter

experts

Training and

experience

Quality

assurance

plans

Validation

Constraint


Questions?


Automated review


Methods for executing automated reviewValidating features during editing using Reviewer Rules

Constraint workflows

• Triggered during creation,

modification or deletion of a

feature

• Error message displayed when

feature violates constraint


Methods for executing automated reviewValidating features using map-based Reviewer Rules

Validation workflows

• ArcGIS Pro

• Validates

- Selection

- Map extent

- Changed features (versioning)


Methods for executing automated reviewValidating features using Reviewer Batch Jobs


• ArcMap




• ArcMap

• Geoprocessing




• ArcMap

• Geoprocessing

• ArcGIS Workflow Manager




• ArcMap

• Geoprocessing


• ArcGIS Server


http://bit.ly/DR_BVM



• ArcMap

• Geoprocessing


• ArcGIS Server

• Validates

- Selection

- Map extent

- Full database

- Changed features (traditional versioning)


http://bit.ly/DR_BVM

Demonstration

Automated review


Workshop scenario



Success criteria

✓ Future data quality requirements are integrated into existing data management workflows

❑ Existing and new features should comply with new quality requirements




Questions?


Semi-automated review



Automated

review

Quality

reporting

Semi-automated

review


Types of quality control

Fast

Consistent and

repeatable

Objective

100% coverage

Automated

review

Semi-automated

review

Guided Workflows

Streamlined processes

Subjective

Sampling


Value of performing visual review

Discover patterns

Compare against trusted sources

Find missing features


Semi-automated reviewLeveraging ArcGIS Desktop and Server

Tools supporting semi-automated review

- Redlining missing features

- Flagging existing features in error

- Random sampling


Demonstration

Visual data review


Workshop scenario



Success criteria


✓ Existing and new features should comply with new quality requirements




Questions?


Error management

and reporting


Reporting data quality

What are the issues on a geographic extent?

How severe are the issues with my data?

How many types of issues do I have in my data?

How many issues have been reviewed and fixed?

How many issues do I have by source?


Reviewer Errors

• Spatial and non-spatial information

• Attributes describe check, problem, and validation information

• Identifier provides link back to feature that failed validation

• Have a defined lifecycle (Reviewed, Corrected, Verified)

Reviewer

Errors


Managing quality controlQC lifecycle management


Review

Find and

record errors

Correct

Perform editsor

Note exceptions

Verify

Acceptableor

Unacceptable

Demonstration

Managing Errors


Workshop scenario



Success criteria


✓ Existing and new features should comply with new quality requirements

✓ Features should be accurately positioned



Questions?


Workshop review

• Defining data quality

• Forms of data quality control

- Automated review

- Semi-automated review

• ArcGIS Data Reviewer

- Automated validation checks

- Semi-automated tools

- Error management


See Us Here

Tuesday

ArcGIS Data Reviewer: An Introduction (Offering #1) Room 30 C 08:30 am - 09:30 am

Wednesday

ArcGIS Data Reviewer: An Introduction (Offering #2) Room 30 E 08:30 am - 09:30 am

ArcGIS Data Reviewer: Implementing Ready-to-Use Rules across the

platform

Demo Theater 4 10:00 am - 10:45 am

ArcGIS Data Reviewer: Deploying Data Quality Web Services Demo Theater 5 11:15 am - 12:00 pm

ArcGIS Data Reviewer: Validating linear-referenced events Demo Theater 4 11:15 am - 12:00 pm

Thursday

ArcGIS Data Reviewer: Advanced Data Validation Room 30 E 08:30 am - 09:30 am


See Us Here


Data Reviewer

Info

Want to learn more?

• Product page

www.esri.com/datareviewer


Want to learn more?

• Data Reviewer templates

- Local government

- Topographic mapping

- Utilities

- Electric

- Gas

- Water

- Water resources

• Based on Esri industry models

• Use as starting point


solutions.arcgis.com

http://solutions.arcgis.com/

Want to learn more?

• Documentation

- Desktop desktop.arcgis.com


Want to learn more?

• Documentation

- Desktop

- Server

enterprise.arcgis.com


Want to learn more?

• Documentation

- Desktop

- Server

• Training



Want to learn more?

• Documentation

- Desktop

- Server

• Training

• GeoNet Community



Please Share Your Feedback in the App

Download the Esri

Events app and find

your event

Select the session

you attended

Scroll down to

“Survey”

Log in to access the

survey

Complete the survey

and select “Submit”


Questions?


arcgis data reviewer: an introduction · workshop scenario my organization needs to improve data...

Documents