big data and arcgis: an introduction to arcgis geoanalytics server · 2017-03-16 · geoanalytics...
TRANSCRIPT
![Page 1: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/1.jpg)
Big Data and ArcGIS: An Introduction to
ArcGIS GeoAnalytics ServerSarah Ambrose and Ravi Narayanan
![Page 2: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/2.jpg)
• Introduction
• Analysis using GeoAnalytics Server
• GeoAnalytics Server Administration
Overview
![Page 3: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/3.jpg)
ArcGIS has a new way of processing large vector and tabular
data with both spatial (location) and temporal (time) components
that is designed for fast distributed analytics and storage
What is GeoAnalytics Server?
![Page 4: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/4.jpg)
Geoprocessing
Powerful analyticsDistributed analytics
and distributed storage
GeoAnalytics Web GIS Layers
Rich geoinformation model
GeoAnalytics Server | Extends ArcGIS Analysis Capabilities
![Page 5: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/5.jpg)
GeoAnalytics is out of the box and ready to use within your
ArcGIS Enterprise
GeoAnalytics Server | Why?
![Page 6: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/6.jpg)
Run analytics against data that is too big for a single desktop
GeoAnalytics Server | Why?
months weeks days hours minutes
![Page 7: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/7.jpg)
ArcGIS Pro
Portal Map Viewer
GeoAnalytics Server | Familiar User Interfaces
![Page 8: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/8.jpg)
ArcGIS Pro
Portal Map Viewer
GeoAnalytics Server | Familiar User Interfaces
![Page 9: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/9.jpg)
REST API
Python API
GeoAnalytics Server | Developer Interfaces
![Page 10: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/10.jpg)
REST API
Python API
GeoAnalytics Server | Developer Interfaces
![Page 11: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/11.jpg)
GeoAnalytics Server | From Noise to Intelligence
Gain insight into both spatial and temporal patterns
![Page 12: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/12.jpg)
GeoAnalytics Server | From Noise to Intelligence
Run analysis against massive collections in a scalable manner
![Page 13: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/13.jpg)
Perform Analysis using GeoAnalytics Server
• Demo
• Analysis tools Overview
![Page 14: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/14.jpg)
Reconstruct
tracks
![Page 15: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/15.jpg)
Aggregate
Points
![Page 16: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/16.jpg)
Create Space
Time Cube
![Page 17: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/17.jpg)
• Analysis capabilities patterned from ArcGIS Online Spatial Analysis
• Tools useful for big data with an emphasis on:
- Spatiotemporal analysis
- Summarization and aggregation
Instant Interval
Analytical Overview
![Page 18: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/18.jpg)
Aggregate Points
Join Features
Reconstruct Tracks
Summarize Attributes
Summarize Within
Summarize Data
Find Locations
Analyze Patterns
Use Proximity
Manage DataFind Similar Locations
Calculate Density
Create Space Time Cube
Find Hot Spots
Create Buffers
Copy to Data Store
GeoAnalytics Server | Rich Collection of Analysis Tools
![Page 19: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/19.jpg)
• Aggregate into polygons
• Aggregate into time steps
Target Features Join Features Intermediate Result Final Result
Aggregation
![Page 20: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/20.jpg)
• Points into bins (square or hexagon)
Aggregation Into Space and Time
![Page 21: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/21.jpg)
• Points into Bins
Aggregation of lightning strikes
![Page 22: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/22.jpg)
Aggregation of Lightning Strikes – 6 hours intervals
![Page 23: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/23.jpg)
• Three parameters to define a time step:
- Interval (duration of time in a step)
- Repeat (frequency of a step)
- Reference time (alignment)
• Examples:
- Hourly steps Interval: 1 hour
- Every 12th hour Interval: 1 hour Repeat: 12 hours
- Every Monday Interval: 1 day Repeat: 1 week Reference: Some Monday
1 2 0 0 1
Interval
Repeat
Reference Time
THEN NOW
Time Stepping
![Page 24: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/24.jpg)
• Spatiotemporal joins
Target Features Join Features Intermediate Result Final Result
Joining Features in Space and Time
![Page 25: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/25.jpg)
• Reconstruct Tracks
- Summarize time-enabled points into tracks
Aggregate Points into Tracks
![Page 26: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/26.jpg)
GeoAnalytics Server and Your Data
• Use your GIS data
- Works with layers already in your Web GIS
• Use your own big data sources: big data file shares
- Easily attach folders located locally on a disk or file share
- Data stored in HDFS or Hive
• Output:
- Hosted feature service
- By default, results are stored in the spatiotemporal data store
- Local NetCDF file (Create Space Time Cube)
![Page 27: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/27.jpg)
Registering a Big Data File Share
![Page 28: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/28.jpg)
GeoAnalytics Server Setup and Administration
• GeoAnalytics Server Setup
• Deployment Patterns
![Page 29: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/29.jpg)
Components required for GeoAnalytics Server
ArcGIS for Server (includes GeoAnalytics Server)
ArcGIS Data Store
(relational and spatiotemporal)
Portal for ArcGISArcGIS Webadaptor
![Page 30: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/30.jpg)
Step 1: Setup a base ArcGIS Enterprise deployment
Hosting
Server
Portal
for ArcGIS
ArcGIS
Relational
Data Store
ArcGIS
Web Adaptor (Server)1. Install Portal for ArcGIS
2. Install ArcGIS Server (license as Std or Adv GIS Server)
3. Install ArcGIS Data store
5. Register Relational ArcGIS Data store with Server
6. Federate Server with Portal and configure as Hosting Server
ArcGIS
Web Adaptor (Portal)
4. Install and configure web adaptors for Server and Portal
Base ArcGIS Enterprise(Machine 1)
![Page 31: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/31.jpg)
Step 2: Install and configure ArcGIS Spatiotemporal Data Store
ArcGIS
Spatiotemporal
Data Store
Hosting
Server
Portal
for ArcGIS
ArcGIS
Relational
Data Store
ArcGIS
Web Adaptor
(Hosting Server)
ArcGIS
Web Adaptor
(Portal)
Base ArcGIS Enterprise(Machine 1)
ArcGIS Spatiotemporal Data store
(Machine 2)
1. Install ArcGIS Data store
2. Register as ArcGIS Spatiotemporal Data store
![Page 32: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/32.jpg)
Step 3: Install ArcGIS Server, license and configure as
GeoAnalytics Server
GeoAnalytics
ServerArcGIS
Spatiotemporal
Data Store
Hosting
Server
Portal
for ArcGIS
ArcGIS
Relational
Data Store
ArcGIS
Web Adaptor
(Hosting Server)
ArcGIS
Web Adaptor
(Portal)
Base ArcGIS Enterprise(Machine 1)
ArcGIS Spatiotemporal Data store
(Machine 2)
GeoAnalytics Server(Machine 3)
ArcGIS
Web Adaptor
(GeoAnalytics
Server)
1. Install ArcGIS Server (license as GeoAnalytics Server)
2. Install and configure web adaptorfor Server
3. Federate Server with Portal and configure as GeoAnalytics Server
![Page 33: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/33.jpg)
Deployment Patterns - Simple Deployment
GeoAnalytics
ServerArcGIS
Spatiotemporal
Data Store
Hosting
Server
Portal
for ArcGIS
ArcGIS
Relational
Data Store
ArcGIS
Web Adaptor
(Hosting Server)
ArcGIS
Web Adaptor
(Portal)
Machine 2 Machine 3
ArcGIS
Web Adaptor
(GeoAnalytics
Server)
Base ArcGIS Enterprise(Machine 1)
Big Data
File Share
(HDFS/Hive/File Share)
![Page 34: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/34.jpg)
Deployment Patterns - Multi-Machine Deployment
GeoAnalytics
Server
ArcGIS
Spatiotemporal
Data Store
Hosting
Server
Portal
for ArcGIS
ArcGIS
Relational
Data Store
ArcGIS
Web Adaptor
(Hosting Server)
ArcGIS
Web Adaptor
(Portal)
Machines 2 - 4
Machines 5 - 7
ArcGIS
Web Adaptor
(GeoAnalytics
Server)
Base ArcGIS Enterprise(Machine 1)
Big Data
File Share
(HDFS/Hive/File Share)
![Page 35: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/35.jpg)
System Requirements
• Supported on Windows and Linux platforms
- On-premises, Amazon or Azure.
- Easy to configure with Amazon Cloud formation templates or Azure Cloud Builder.
• Base ArcGIS Enterprise deployment
- Minimum 4 cores and 16 GB RAM
• ArcGIS Spatiotemporal Data Store
- Minimum 32 GB RAM for each machine
- Fast disk/sufficient Disk space for data
• ArcGIS GeoAnalytics Server
- Minimum 4 cores and 16 GB RAM for each machine
- Sufficient temp space for compute.
![Page 36: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/36.jpg)
Web GIS Layers
New Web GIS Layers
Use your Web GIS layers through Pro, Portal, Python Notebooks,
or the REST API
ArcGIS Data Store
Feature Services
GeoAnalytics Architecture – Inputs & Output Considerations
ArcGIS
Relational
Data Store
ArcGIS
Spatiotemporal
Big Data Store
Big Data
File Share
Portal
for ArcGIS
GeoAnalytics
Server
![Page 37: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/37.jpg)
Summary
• ArcGIS GeoAnalytics “out of the box” and “ready to use” within ArcGIS Enterprise
• Distributed Analytics – Powerful tools you get “big jobs” done faster
- Makes big data easier to use and easier to share
- Focuses on space and time analysis for massive dataset processing
• Lets you see the big data picture in a clearer and more meaningful way
![Page 38: Big Data and ArcGIS: An Introduction to ArcGIS GeoAnalytics Server · 2017-03-16 · GeoAnalytics Server and Your Data •Use your GIS data-Works with layers already in your Web GIS•Use](https://reader030.vdocument.in/reader030/viewer/2022013021/5f1e9c410cbbe90b4a6d4fe9/html5/thumbnails/38.jpg)