distributed collaboration: sharing data across …...•pause & resume sync •better...

Tosia Shall, Esri

Frank Roberts, Innovate! Inc.

Distributed Collaboration: Sharing Data Across Systems

Agenda

• What value does Distributed Collaboration bring?

• Sharing between Enterprises

• Sharing between Enterprise and ArcGIS Online

• Key concepts in Distributed Collaboration

• Demo: sharing data

• Collaboration in each release

• Case Study

What value does Distributed Collaboration bring?

Problem statements:

I need to work with other organizations on a project but I have no access to their data.

I’m supposed to be helping other departments but I can’t see what information they’re working with.

I have data in one environment but I need to be able to access it in another

one.

And many more…

Invalid

Credentials X

How does Distributed Collaboration help?

• Establishes a trusted connection with other ArcGIS Enterprise environments and ArcGIS Online

• Allows you to quickly and easily share data and information across systems

• Helps to provide a common operational picture of data and make it visible and usable across environments

Sharing Between ArcGIS Enterprises

From one to many:

- Distribute data from a central source to many satellite deployments

- Each can use the data copy in their own maps and apps

- Examples: emergency management and response data sharing, offshore wind farm planning to avoid shipping lanes, protected areas and more.

NOAA

Federal Gov

Local Gov

State Gov

Partners

Copy

of data

Copy of data

Copy of data

Copy of data

Sharing Between ArcGIS Enterprises

From many to one:

- Create an ongoing, centralized view of data in one environment

- Examples: receiving data from government, non-profit, and University sources

Marine Cadastre

Federal Gov

Local Gov

State Gov

NGO/NPO/UniversitiesCentralized

view of data

Sharing between ArcGIS Enterprise and ArcGIS Online

Sharing to ArcGIS Online:

- Benefit from the scalability of a SaaS environment

- Manage data and items in ArcGIS Enterprise and keep it in sync with Online

- Examples: sharing to the public, sharing to contractors who have access to ArcGIS Online

ArcGIS Enterprise

ArcGIS Online

external, public sharing

internal,data management and

creation

Sharing between ArcGIS Enterprise and ArcGIS Online

Sharing from ArcGIS Online:

- Copy data collected using field apps, public surveys, etc.

- Send to Enterprise for QA

- Or just reference the data across environments for awareness

- Examples: collecting marine debris data in the field by citizen scientists

ArcGIS Enterprise

ArcGIS Online

field data collection

quality assurance and management

Key Concepts in Distributed Collaboration

Prerequisites for collaboration

• At a minimum, ArcGIS Enterprise base deployment needed

- Server federated with your portal and set as your hosting server

- ArcGIS Data Store (relational) registered with your deployment

• At least version 10.5 required between Enterprise, 10.5.1 with Online

- New functionality is available at each release

• Some additional configuration may be required (ports, SSL certificates, etc.)

• Data requires certain settings (sync) if sharing as a copy

Setting up a collaboration

• Creates the collaboration• Adds the guests

- Determines if guests can send, receive, or both

- Sends invitation file to connect

Host Guest• Accepts the invitation file• Sends the response file to close the connection• If sending data on a schedule, determines the

sync interval (1 – 24 hours)

Both: Link a group to the collaboration

Determine if feature layers are copied or referenced

Trusted Collaboration

NOAAMarine

Cadastre

NOAA to Marine Cadastre

Authoritative Data from NOAA

Group

NOAA Authoritative Data Share

Group

Workspace:NOAA and Marine Cadastre

Architecture of a collaboration

NOAA Recipient

Copy• Creates a full copy of your data• Updates are synced from you to your

recipients once every 1 – 24 hours• Sync setting required

Reference• References your data in place• Data is always ‘live’• Requires a login to access or to share

with everyone• **10.8 change

NOAA Recipient

Options for sharing feature layers

What items can I share using collaboration?

• Hosted feature layers

• Feature layers published by reference

• CSVs, Word documents, Excel files

• Shapefiles

• Tile packages, vector tile packages

• Web maps

• Web scenes

• Map services (by reference)

• Web AppBuilder apps (including Group-based apps)

• Apps created from Esri templates

• Insights items (workbooks, pages, models, themes)

• Hosted feature layer views (copy)

• + more

Demo: Sharing Data

Release Highlights

Release highlights

10.7.x

(Released at 10.5; 10.5.1 with ArcGIS Online)

10.810.5 – 10.6.1• Share feature layers as copies

• Share web apps

• Content delete policy

• Sync status reporting for administrators

• Ability to sync now (on demand)

• Collaborate Insights items

• Copy hosted feature layer views

• Pause & resume sync

• Better identification of collaborated

content (badge, filter)

• Custom admin role for managing

collaborations

• Group Based Apps

• Store credentials for feature services

shared by reference

• Portal to Portal only, both must be at least

10.8

• Entered and saved by Admin on the

sending Portal

• Can only be done through the ArcGIS

Portal Directory sharing API using the Update

Service Proxy Config operation.

• Will not work on ArcGIS

Enterprise systems that use web-tier

authentication, such as IWA or PKI

Note: you will only have the features available to you at your release, regardless of the release other collaborators are using.

Roadmap

Roadmap

• Two-way editing

• Retain metadata & content categories

• Collaborate different item types like

Operations Dashboards and ArcGIS Enterprise

Sites

Strategizing

Agree on policies for data governance

Who will be sharing data?

What data will be shared?

Frequency of updates?

What are everyone’s goals?

What version is everyone using?

Review your collaborations periodically to ensure they are meeting the intent!

Setting your collaboration up for success

Don’t be afraid to whiteboard!Create an architecture that meets your needs

Case Study of Distributed Collaboration

using ArcGIS Online at US EPA Region 9Frank Roberts, Innovate! Inc.

Use Case Background

This includes:

• Data creation

• Digital and paper map products

• GIS Analysis

• Mobile data solutions

• Management Support of ArcGIS Online (EPA GeoPlatform)

EPA Mission: To protect human and environmental health

Innovate! Provides staff to support the R9 Technology and Data Solutions Center (R9 TDS)

EPA Region 9

Problem Statements

• Multiple Organizations with Data

• EPA R9 (R9)

• EPA HQ

• CA Department of Toxic Substances Control (DTSC)

• Water Board

• USGS

• Increase data sharing and data products across multiple organizations in the area of drinking water well data

• Avoid sending large data sets back and forth over the wire or mail

• Increase awareness/documentation of source data metadata

• Bring multiple organizations into one shared location (sandbox) so decisions are being made using the same data and the most recent data

• Staff Turnover

• Different priorities with different organizations

Before Distributed Collaboration:

• Metadata was very spotty

• Many duplicate datasets but some had more features then others and no clear understanding of why

• Challenged by staff turnover

• Organizations were using similar but different applications to solve similar problems

Before Distributed Collaboration: Metadata

The summary of the inventory was as follows:

• Total Vector GIS layers (shapefiles and feature classes): 111

• GIS layers with preliminary software created metadata: 56

• GIS layers with no metadata: 55

• GIS layers with human entered metadata: 28

• GIS Layers with need of human entry: 83

Collaboration Benefits

If we had a place we could share:

Data

Map Services

Hosted Feature Services

Applications

We then could all be using the same data and tools to make decisions.

When data was updated in shared space all users would immediately have access to it.

A Data management plan could also be created to determine the base rules of how to play in the sandbox

Sandbox Rules

Real Sandbox AGOL Shared Group

Equivalent

Don’t pee in the sandbox Don’t put data in the sandbox

that is not clean

Clean up your own mess Keep the shared group

cleaned up and un-share

items that are no longer used

or needed

Put your name on your toys Create metadata for items

you add so we know where it

came from

Follow the rules Follow the data management

plan

Don’t throw sand at anyone Work things out and work

together to solve the world’s

problems

EPA/DTSC Shared Space

Data

Apps

Web Maps

Resources for Distributed Collaboration

Resources

• Blogs

• Documentation

• Case study

Questions