jay fowler solution engineer - esri · output connectors you can create your own connectors box add...

32
ArcGIS GeoEvent Server REALTIME GIS Jay Fowler Solution Engineer

Upload: others

Post on 01-Jun-2020

6 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

ArcGIS GeoEvent Server

REALTIME GISJay Fowler

Solution Engineer

Page 2: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

G O L D

T H A N K Y O U T O

OUR SPONSORS

S I LV E R

Page 3: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

AgendaREALTIME GIS

• Intro – Real-Time GIS Overview

• Act I – Hop on the bus

• Act II – Radar Love

• Act III – Real time data tells great stories

• Bridge – How does GeoEvent fit into ArcGIS Enterprise

• Q&A

• Outro – Conclusions

Page 4: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

movingsomething that moves

• planes

• vehicles

• animals

• satellites

• storms

stationarystands still but attributes change

• water gauges

• weather stations

• traffic sensors

• air quality

• temperature

• wind

discretesomething that “just happens”

• crimes

• lightning

• accidents

Intro| Real-Time GIStypes of observation data

Page 5: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

Each event represents the latest state of the sensor…

Features

ApplicationsNetwork Sensor

Warehouse Item

110°

Storm

Wind

Temperature

Earthquake

Wild Fire

Police Person

Ambulance

Police Car

Intro| Real-Time GIS datacontinuous stream of events flowing from a data feed

Page 6: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

Consume raw data served in real

time to deliver a targeted application

Act I

Hop on the bus

Page 7: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

Recap

• Raw Data

• GeoEvent

• Filters and

Processors

• Web Map

• Web App

Data, to map, to application with functionality

Page 8: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

FeaturesContinuous

Analysis

Inside

Boundary

Vessel

Alert

Applications

Continuous processing on event data as it is received …

Real-Time analyticswhat fishing vessels are inside designated ‘no fishing’ zones?

Page 9: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

Streaming analytics, policies & orchestrationGeoEvent Server: geoevent services

input1 output1buffer

geofences

filter(spatial)

input2 output2

geofences

geotagger

output3motion

calculator

geometry inside Zones/.*

geometry inside Zones/.*

• A GeoEvent Service configures the flow of real-time data,

- the filtering and geoevent processing steps to perform,

- the input(s) where data comes from and the output(s) to which results are sent.

Page 10: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

Ingestion of real-time dataGeoEvent Server: input connectors

GeoEvent Server

Inp

uts

Ou

tpu

ts

GeoEvent Services

you can create

your own

connectors

Poll an ArcGIS Server for Features

Ou

t o

f th

e B

ox

Esri

Gallery

ActiveMQ

Part

ner

Gallery

Amazon IoT

Azure IoT

CompassLDE

enviroCar

exactEarth AIS

FAA (ASDI)

GNIP

Networkfleet

OSIsoft

Valarm

Waze

AIS

Common Alerting Protocol

Cursor-on-Target

Exploitation Support Data

GTFS

Instagram

Kafka

KML

MQTT

NMEA 0183

RabbitMQ

Sierra Wireless (RAP)

Trimble (TAIP)

Twitter

Poll an external website for GeoJSON, JSON, or XMLHTTP

Receive Features, GeoJSON, JSON, or XML on a REST endpointREST

Receive GeoJSON or JSON on a WebSocketWS

Receive RSS

Receive Text from a TCP or UDP Socket

Subscribe to an external WebSocket for GeoJSON or JSONWS

Watch a Folder for new CSV, GeoJSON, or JSON Files.csv

AIS

Page 11: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

you can create

your own

processors

Streaming analytics, policies & orchestrationGeoEvent Server: processors

GeoEvent Server

Inp

uts

Ou

tpu

ts

GeoEvent Services

Buffer Creator

Convex Hull Creator

Difference Creator

Envelope Creator

Field Calculator

Field Enricher

Field Mapper

Geotagger

Incident Detector

Intersector

Projector

Simplifier

Symmetric Difference

Track Gap Detector

Field Reducer Union Creator

Ou

t o

f th

e B

ox

Add XYZ

Esri

Gallery

Bearing

Ellipse

Event Volume Control

Extent Enricher

Field Grouper

GeoNames Lookup

Range Fan

Reverse Geocoder

Service Area Creator

Symbol Lookup

Track Idle Detector

Unit Converter

Visibility

Motion Calculator Query Report

Page 12: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

GeoEvent Server

Inp

uts

Ou

tpu

ts

GeoEvent Services

Dissemination of real-time dataoutput connectors

you can create

your own

connectors

Ou

t o

f th

e B

ox

Add or Update a Feature

Esri

Gallery

Amazon IoT

Azure IoT

ActiveMQ

Cursor-on-Target

Hadoop

Kafka

MongoDB

MQTT

RabbitMQ

Twitter

Publish Text to a UDP Socket

Push GeoJSON or JSON to an external WebsiteHTTP

Push GeoJSON or JSON to an external WebSocketWS

Push Text to an external TCP Socket

Send a Text Message

Send an Email

Send an Instant Messageim

Send Features to a Stream Service

Write to a CSV, GeoJSON, or JSON File .csv

Add a Feature to a spatiotemporal big data store

Update a feature in a spatiotemporal big data store

Page 13: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

Tap into AVL sensors on fleet

vehicles to gain location intelligence

and produce alerts and notifications

Act II

Radar Love

Page 14: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

Vehicle Location and Monitoring

Page 15: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

Vehicle Location and Monitoring

Page 16: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

Vehicle Location and Monitoring

Page 17: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

Vehicle Location and Monitoring

Page 18: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

Service Typeschoosing a service type: stream service, feature service, map service

Stream Layer

Map Layer

Feature Layer

• Stream layers in apps subscribe to stream services to immediately visualize observations.

- Does not require storage, is low latency, cannot be replayed.

• Map & Features layers in apps periodically poll to visualize most current observations.

- Backed by an enterprise geodatabase (EGDB) or a spatiotemporal big data store (BDS).

- History can be retrieved & queried for playback.

desktop

device

web

ArcGIS

Enterprise

Stream Service

Send Features to a Stream Service subscribe (push)

polling (pull)

Map Service

Feature Service

Add a Feature to a BDS

Update a Feature in a BDS

spatiotemporal

big data store

Add or Update a Feature

EGDBGeoEvent

Server

Page 19: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

Recap

• Input and Output

- Persist vs. Broadcast

• Filters and Processors

• Geofence

• Alerting and Notification

Location intelligence in real time

Page 20: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

Gain value by actuating, analyzing,

and dashboarding real time data

Act III

Real time data tells

great stories

Page 21: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

Alerting stakeholders on discovered

patterns of interest in real-time …

Continuous

AnalysisFeatures

Outside

Boundary

Child

Applications

Real-Time actionsnotify & alert a parent when their child leaves school property

Page 22: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

impacting the environment through actions

Actuation

Adjust behavior of things in our environment through actuation…

GeoEvent

Server

Close a Valve

Open a Window

Activate a Device

Page 23: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

purge based on

data retention

spatiotemporal

big data store

node 1

node 2

node 3node 4

node 5

r = 1

using the spatiotemporal big data store

Storing Real-Time event data

Store observations in a spatiotemporal big data store.…

GeoEvent

Server

Page 24: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

Features can be rendered as aggregations or discrete features

Visualization

Visualize high velocity & volume data as an aggregation or as discrete features…

Page 25: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

Web GIS and Operations Dashboards

Monitor real-time operations

Enables at-a-glance decisionmaking

Key performance indicators (KPIs)

Page 26: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

Recap

• Actuation - Impacting the environment through actions

• Analysis, STBDS, Aggregate to hex bins, etc...

• Visualization Aggregations OR Discrete features

• Dashboards

Do more with your real-time data

Page 27: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

Web GIS Is Connecting EveryoneUsing Web Maps and Apps to Share and Collaborate

Supporting Communication

and Real-Time Awareness

Organizations

Communities

People

Creating a System of Engagement

Page 28: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

DesktopWeb Device

live & historic

aggregates & features

map & feature service

• Bring real-time & Internet of

Things (IoT) data into ArcGIS

• Perform continuous analytics on

real-time data

• Store observations

• Visualize high velocity &

volume data:

- as an aggregation

- or as discrete features.

• Notifications and actuation.

stream service

live features

ArcGIS

Enterprise

GeoEvent

Server

analytics

GeoEvent| ArcGIS Enterprisewith real-time & big data capabilities

GeoAnalytics

Server

spatiotemporal

big data store

storage analytics

visualization

Page 29: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

. . . Successfully Integrated into Thousands of Systems

Open Standards

and Formats

Product

Integration

Open Software

AutoCAD

SAP HANAIBM Cognos

Adobe Creative Cloud

AWS

AzureMS Office

Oracle

SQL Server

TeradataNetezza

REST

OPeNDAP

WMSWCS

CSW

GMLWFS

KML

WMTS

WPS

SLD

GeoPackage

SOAP

IFC

LAS

WWW

ISONetCDF

SQL

JSON

WaterML

CityGML

OneGeology

LERC

Shapefiles

Web Scene (I3S)

INSPIRE

Extensible Architecture

Open-Source Contributions (500+)

Embeddable Components

Open APIs & SDKs

Open Data Access

Open-Source Integration

ArcGIS An Open, Interoperable, and Standards-Compliant Platform

SharePoint

Any System

Page 30: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

Q&AQuestions?

Page 31: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka

Outro| ConclusionsReal-Time GIS: GeoEvent Server

• Hop on the bus

- Consume raw data served in real time to deliver a targeted application

• Radar Love

- Tap into AVL sensors on fleet vehicles to gain location intelligence and produce alerts and notifications

- I/O, Persist and Broadcast, Continuous Analysis, Alerting and Notification, Geofence

• Real time data tells great stories

- Gain value by actuating, analyzing, and dashboarding real time data

• Bridge

- ArcGIS Enterprise is a full-featured mapping and analytics platform & supports GeoEvent Server federation

- Data storage with spatiotemporal data store

- Solutions for high volume data visualization with on-the-fly aggregation

Real-Time applications are in high demand

GeoEvent Server enables Real-Time in your organization

Page 32: Jay Fowler Solution Engineer - Esri · output connectors you can create your own connectors Box Add or Update a Feature y Amazon IoT Azure IoT ActiveMQ Cursor-on-Target Hadoop Kafka