spatial processing redefined with sap hanasap hana vora is an in-memory query engine which leverages...

© 2014 SAP SE or an SAP affiliate company. All rights reserved. 1Public

SPATIAL processing redefined with SAP HANA


Disclaimer

This presentation outlines our general product direction and should not be relied on in making a purchase decision. This presentation is not subject to your license agreement or any other agreement with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or to develop or release any functionality mentioned in this presentation. This presentation and SAP's strategy and possible future developments are subject to change and may be changed by SAP at any time for any reason without notice. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this document, except if such damages were caused by SAP intentionally or grossly negligent.


Agenda

• SAP HANA

• SAP HANA Spatial

• SAP & Esri

• Outlook

• Live Demo


SAP HANAA common Database Approach for OLTP and OLAP Using an In-Memory Column Database


SAP HANABasic Technology

In-Memory Computing• Keep Data in memory to speed up data access

• Enterprise server with terabytes of main memory (scale-up)

• Minimize data movement by• column store

• compression

• performing calculations at database level

• Divide and Conquer• Use multi-core architecture of processors, multi-processor servers

• scale-out into a distributed landscape

12x17x

2000x

150x

Register Cache RAM SSD/Flash Hard Disk

1

0,001

1.000.000

1.000

volatile non-volatile

Id Date Name

1 123 12

2 200 32

3 333 18

# Name

12 John

32 Flore

18 Lee

1 2 3 123 200 333 12 32 18

1 123 12 2 200 32 3 333 18

IN-MEMORY, ACID-COMPLIANT, COLUMN BASED

MULTI-CORE/PARALLELIZATION

ADVANCEDCOMPRESSION


Supports any Device

Any appsAny app serverAny apps

Any app serverSAP Business Suite

SAP NetWeaver Application Server for ABAP (SAP Business Warehouse)

SAP Business Suite SAP NetWeaver Application Server for ABAP

(SAP Business Warehouse)

JSONR Open ConnectivityMDXSQL

Other appsLocationsReal timeHADOOPMachineUnstructuredTransaction

SAP HANA platformSQL, SQLScript, and JavaScriptSQL, SQLScript, and JavaScript

Integration ServicesIntegration Services

SpatialSpatial

Business function library

Business function library

SearchSearch Text miningText mining

Predictive analysis library

Predictive analysis library

DatabaseservicesDatabaseservices

Stored procedure and data modelsStored procedure and data models

Planning enginePlanning engine Rules engineRules engine

Application and UI services

Application and UI services

The SAP HANA platform converges database, data processing, and application platform capabilities and provides libraries for predictive, planning, text,

spatial, and business analytics to enable business to operate in real time.

SAP HANA platform More than just a database

Geospatial TechnologiesSAP HANA Spatial


Spatial processing with SAP HANA

SAP HANA Spatial Engine: Native spatial engine as part of column store Colum-wise storage of spatial data Spatial data types (ST_POINT, ST_GEOMETRY) SQL/MM spatial (ISO/IEC 13249) OGC Simple Feature Access - SFA (TF) 1.2.1 Points, line strings, polygons, circular strings … Spatial predicates, functions, formats … Spatial reference systems, units of measure … Access via SQL, SQLScript, CalcEngine, CDS,

Modeler, MDS, … Application development on XS with geo-content and

mapping services ESRI query layers, feature services support Multidimensional type support Clustering …


Spatial Column Scan

MULTICORE + SIMD + RAM


AdvancedGeoSpatialCapabilities


Clustering


ClusteringNative Clustering Algorithms

x

y

Grid

x

y

DBSCAN

x

y

K-Means

• Good for first impression• Easy to use• Extremely fast grid clustering

• Best for non-spherical clusters• Density based• Higher complexity, better insights

• Best for spherical clusters• Centroid based• Higher complexity, better insights


ClusteringSQL Example – Use Cases

Identify for each point the corresponding clusters (similar to window functions)

SELECT ST_ClusterId()

OVER (CLUSTER BY location USING KMEANS CLUSTERS 5 ) AS cluster_id, vm_id, location, revenue

FROM vending_machinesWHERE revenue < 15000 ORDER BY cluster_id, revenue

Clustering with metadata accessor(similar to group by aggregates)

SELECT ST_ClusterId() AS cluster_id, ST_ClusterCentroid() AS centroid, ST_ClusterEnvelope() AS envelope, count(*) AS num_hholds, avg(income) AS avg_clus_income

FROM households WHERE income > 120000 GROUP CLUSTER BY location USING DBSCAN EPS 4 MINPTS 1000 HAVING count(*) >= 300

Cluster_id Centroid Envelope Num_hholds

Avg_clus_income

1 Point(5 4) Polygon((…)) 311 304.123

2 Point(15 78) Polygon((..)) 621 714.234

…

Cluster_id Vm_id Location Revenue

1 1 Point(1 1) 1.311

1 2 Point(1.4 1.2) 1.166

1 3 Point(1.2 1.3) 799

2 4 Point(5.3 5.0) 2.125

2 5 Point(5.7 6) 1.750

3 6 Point(20 20) 1.532

…


GeospatialEcosystem


SAP HANA Spatial Ecosystem

GIS

Maps, Geo-content & Geo-services

Location Intelligence Solutions

GIS Systems Integrator and Consultancy


EsriEsri

ArcGIS Server & Desktop

SAP HANA + Esri – IntegrationTechnical Overview

Esri

Data ExchangeEsri Shapefile

support for ‘offline’ manual data movement

SAP HANA


Query Layers“Push-down”

spatial and non-spatial queries

directly into SAP HANA Spatial

engine

Esri Shapefile(.shp)

SAP HANA

Query Layers

ETLBatch data movement

via SAP Data Services, Safe FME, Esri Interoperability Extensions, or other third-party solutions

SAP HANA

ETL

Feature ServicesWeb services

allowing reading, writing, editing, and

symbology of spatial data from

SAP HANA

SAP HANA

Feature Services

Esri



Geo-ServicesIntegration with Esri

Server & ArcGIS Online for mapping,

content, and advanced geo-service

processing

SAP HANA

XS Engine

Esri

ArcGIS Server & ArcGIS Online

Spatial Data & Processing






Esri

Advanced Geoprocessing

Esri advanced functions running

in SAP HANA

SAP HANA



Esri Embedded Process

Esri

GeodatabaseSAP HANA as a

certified geodatabase

SAP HANA



Esri ArcSDE

SAP HANA + Esri – IntegrationNext Steps & Vision

Geodatabase• Full geodatabase support for SAP HANA

• topologies• Networks• Versioning• …

• Coming Q4 2016 with ArcGIS v10.4.2

Advanced Geoprocessing• Leveraging SAP HANA technology with AFL SDK

• Hotspot analysis• Stream network• Watershed• Geometric network trace• …


Geographic Data Enrichment

Spatial Analytics

Rich Mapping

ArcGIS For ServerSAP HANA

ArcGIS For Desktop

ArcGIS Online

SAP

ERP

CRM SCM

SAP and EsriBringing Location to the Enterprise

HCP / HEC

SAP HANA

SAP & Esri Portfolio• Enterprise Applications• Database Technology• Cloud Services• Mobile SDK• Analytics


The Five Megatrends Driving Our Digitized WorldAnd Their Implications for Distributed Big Data Management

Hyper ConnectivityEverybody has

access

Super Computing

Super computers power everywhere

Cloud Computing

The cloud is where we compute

Smart World

Your fridge knows what you want for

dinner

Cyber-Security

High-powered security is now the

norm


SAP HANA VoraWhat’s Inside and What Does It Do?

DemocratizeData Access

Make PrecisionDecisions

SimplifyBig DataOwnership

SAP HANA Vora is an in-memory query engine which leverages and extends the Apache Spark execution framework to provide enriched interactive analytics on Hadoop. Drill Downs on HDFS

Mashup API EnhancementsCompiled Queries

HANA-Spark ControllerUnified LandscapeOpen Programming

Any Hadoop Clusters


YARN

HDFS

Enable Precision DecisionsWith Contextual Insights In Enterprise Systems

Other Apps

Files Files Files

HANA-Spark Controller for improved performance between distributed systems

Gain business coherence with business data, spatial data and big data

Compiled queries enable applications & data analysis to work more efficiently across nodes

Familiar OLAP experience on Hadoop to derive business insights from big data such as drill-down into HDFS data

Compiled Queries

Spark Controller

Drill Downs

SAP HANA in-memory platform

Vora

Spark

Vora

SparkIn-Memory

StoreApplication Services

Database Services

Integration Services

Processing Services

SAP HANA Platform

Vora

SparkHANA Smart Data

Access Spark Controller


Spatial Compliance

Co-Innovation (Customers, ESRI)

Customer-Driven types, functions

Start with OGC

SQL M/M Standard

Import/Export

HANA spatial roadmapNext goals

SQLOptimizer / Statistics

Full Round Earth support

Raster data

New Innovations & Features

Geometry Compression

Spatial temporary analysis

OGC services (WFS, CSW, …)

QGis, MapServer, GeoServer, …

Topology-, Network-support

Routing, GeoCoder, …

3D function support (e.g. ST_Intersects3D), Visual Enterprise, AutoCAD, …

Streaming

Polygon Simplifier

SPS10

multi-dimensional type support

ST_Transform

ST_AsSVGAggr

Spatial preview in hdbstudio

Spatial support in CDS

Spatial support Modeller

Improved support forSQLScript

Improved supportability

SPS07 SPS10SPS08This is the current state of planning and may be changed by SAP at any time.

SPS09

SPS09

Spatial Join

Full Spatial Support in CDS

First Spatial Support SQLScript

GEOCODE INDEX

HERE Partnership

ST_CircularString

ST_UnionAggr

SPS11 next

SPS11

Clustering

ConvexHullAggr, EnvelopeAggr, IntersectionAggr

Scalar type support in SQLScriptand LLANG

Multi dimensional modeling(Cloud for analytics)

Partitioning with spatial types

Metadata Installer

OGC compliance (SFS 1.2.1)

Spatial support in SDA

Grid Join improvements

SFC improvements

SPS12

Performance Optimization(grid join, scans)

Focus on Esri development

Modeler support with SpatialFilter and calcuated columns

Spatial Type support in AFL

Spatial Viewer

Extended Round Earth Support

Research

PoC

Graph and Spatial(networking and topologies)

Developer howtos, Development Series, Blogs, whitepapers, documentation, hands on labs

SPS12


LIVEdemo

© 2014 SAP SE or an SAP affiliate company. All rights reserved.

Thank youSpatial Processing with SAP HANAhttp://hana.sap.com/abouthana/hana-technology/advanced-analytics.html

SAP HANA Spatial Referencehelp.sap.com/hana/SAP_HANA_Spatial_Reference_en.pdf

SAP HANA Academywww.saphana.com/academy

HANA Spatial Team

spatial processing redefined with sap hanasap hana vora is an in-memory query engine which leverages...

Documents