in-memory analytics · pdf filebi andweb & social data reporting discovery lab actionable...

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

In-Memory Analytics & Big Data Discovery

Impulsvortrag & Diskussion Björn Ständer Business Analytics - Director Business Development Oracle Deutschland B.V. & Co KG, München October 21st , 2014

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


Enterprise Computing Trends

GLOBALIZATION

9 Billion Internet Devices

in 2012

50 Billion by 2020

DATA EXPLOSION

90% Created within Last Two Years

50X Growth by 2020

RISE OF MOBILITY

6 Billion Mobile Subscribers

87%

of World‘s Population

Mobile Data

Growing 78% CAGR

CLOUD MODERNIZE TO SURVIVE

Lots of

20 year-old Legacy

Applications

90% of new software

delivery for public cloud

Public IT cloud

services will grow

to $107B in 2017


“Engage me everywhere.”

“Know me. Wow me.”

“Meet my expectations.”

“Understand and reward me.”

Customers Demand Great Experiences


Actionable Events

Data Reservoir

Data Factory Data Warehouse In-Memory

BI and Reporting

Discovery Lab

Actionable Information

Actionable Insights

Data Streams

Execution

Innovation

Discovery Output

Events & Data

Next Generation Data Management Architecture

6

Enterprise Data

Web & Social Data

Event Engine & Real-Time Decision


Use Data

12%

Executives who feel they understand the impact data

will have on their organizations

Produce Data

Big Problem With Big Data


Big Data Discovery & Oracle In-Memory


Oracle BI – Key Investment Areas

• Business User Experience

• Mobile BI

• Analytics in the Cloud – Platform and Applications

• Big Data Discovery

• Innovations in Engineered Systems and In-Memory Analytics

9


Hadoop Data Reservoir Concept Gaining Momentum

10

Data Warehouse Data Reservoir

Emerging Sources Existing Sources

Source: wikibon.org/wiki/v/Big_Data_Vendor_Revenue_and_Market_Forecast_2013-2017 Source: 451 Research – Total Data Warehousing: 2013-2018

Source: The Forrester WaveTM: Big Data Hadoop Solutions, Q1 2014


Oracle Big Data Analytics Strategy

Data Lake (BDA or Object Storage Service)

Data Factory In-Memory Data Warehouse

Data Discovery Analytic Models

Event Data

Social Data

Enterprise Data

Oracle Visual Analytics (OBIEE)

Oracle Big Data Discovery


Oracle Big Data Discovery In Preview

12


Not Easy to Get Analytic Value from Hadoop Data Reservoir

13

• Existing analytic tools fall short – Too much effort on upfront data preparation

– Manual exploration for understanding new data sets

– Depend on ETL to cleanse data and make it ready

– Assume questions known in advance

• Only point solutions emerging – Separate data wrangling, visualization

– Leads to constant context switching

• Need end-to-end capabilities

• Native Hadoop tools are complex – Pig, Oozie, Sqoop, Hive, Spark

• Specialized skills are scarce – Programming languages

(e.g. Map Reduce, Python, Scala)

– Statistics and machine learning

– Command line interfaces


Big Data Analytics. Requires a Fundamentally New Approach

14

Explore

Transform Discover

Find

An intuitive, interactive and visual user interface

for anyone to quickly find, explore, transform and analyze

data in Hadoop

then share results for enterprise leverage


• Navigate a rich catalog of all data in the Hadoop cluster

• Familiar search and guided navigation for ease of use

• Access data set summaries, annotation and recommendations

• Provision your own data through self-service upload

• Data is automatically enriched with extracted locations, terms, sentiment

• Browse personal big data projects and those shared by the community

15

Easily Find Relevant Data Sets


• Understand shape of the data. Visualize attributes by type

• Machine learning algorithms sort attributes by importance

• View attribute statistics, data quality and outliers

• See statistical correlations between attribute combinations

• Evaluate whether a data set is worthy of further investment

16

Explore the Data and Understand Potential


• Interactive, intuitive user-driven data wrangling

• Library of data transformations to replace values, convert types, collapse, reshape, pivot, group, custom tag, merge and much more

• Data enrichments for inferring location and language. Theme, entity and sentiment enrichments for text

• Preview results, undo, commit and replay transforms

• Run on sample data in memory or full data set in Hadoop

17

Transform and Enrich Data to Make it Ready


• Mash up different data sets for deeper perspectives

• Filter through data with powerful search and intuitive guided navigation

• Drag and drop from a rich library of interactive visualizations to compose discovery dashboards

• Publish blended data sets back to Hadoop

• Share projects, bookmarks and snapshots with team members for collaboration

18

Analyze the Data to Discover New Insights

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 19

Oracle Big Data Discovery. The Visual Face of Hadoop

Explore

Transform Discover

Find


Share Results and Publish for Enterprise Leverage

20

• Share and collaborate with the team

– Share projects, bookmarks and snapshots then collaborate and iterate

• Publish back to Hadoop

– Transforms and enrichments may be applied to original data sets in Hadoop

– Publish blended data sets back to HDFS

• Leverage results in other tools

– Publish data to Hadoop in format optimized for advanced analytic tools (e.g. ORAAH)

– Hadoop compliant BI tools (e.g. OBIFS) can burst out to the masses

– Leverage any native Hadoop tooling (e.g. Pig, Hive, Impala, Python, etc)

– Integrate BDD data sets with DWH to secure, govern and optimize for query performance (e.g. Oracle Big Data SQL)

Oracle Big Data Discovery plays well with the Big Data ecosystem

Explore

Transform Discover

Find

Share & Collaborate

raw data

transformed data

data reservoir

(HDFS)

Publish

data warehouse

business intelligence

advanced analytics

other hadoop tools

Leverage


Oracle Big Data SQL

21


Barriers to Big Data Adoption Complexity

• Skills

– Lack tools and training to exploit Big Data

– IT Operations ability administer and manage Big Data

• Integration – Adding Big Data to existing architecture is complex

– Too much effort required in data preparation

• Security

– No clear route to governance or enforcement


Oracle Big Data Analytics

Prediction & Collaboration

Exploration & Discovery

Data Transformation

Data Profiling & Preparation

Hadoop

Big Data Discovery Visual Face to Hadoop

Big Data SQL Fast, Secure Oracle SQL on All your Data

SQL

Query Offload to Data Nodes

Query Offload to Storage Servers


Oracle Big Data SQL. One Fast Query Across all Data, in Place

24

Tables in Hadoop

Tables in DB

SQL join

PRODUCT DIMENSION

SALES FACT

ENRICHED WEBLOGS

CUSTOMER DIMENSION

ENRICHED WEBLOG

{"logid":"L23588999", "customerid":"[email protected]", "platform":"Desktop", "eventdate":"11/6/12", "httpcode":"200", "brandname":"Coca-Cola”, "ipaddress":"167.134.198.24", "qty":1}

• What does it do?

– Query tables in Hadoop via SQL as if they lived natively in Oracle DB

• Avoid data movement + data latency

• Use the SQL skills you already have

– Apply security and redaction features of Oracle DB to Hadoop

• How do we make it fast?

– Smart Scan for HDFS. Apply filters and project columns before streaming results back to Oracle DB

– Storage Indexing. Instead of executing full scans – only read the data relevant to a given query

– Complex evaluation. Performing expensive parsing of JSON/XML where the data lives


• OBIEE

– Query & Report on Hadoop, NoSQL & Relational

– Certified with 12c & Big Data SQL

• Endeca – Use and extend your existing

investment

Oracle Business Analytics on Big Data


Oracle 12c In-Memory DB

26


Accelerate Mixed Workload OLTP

Real Time

Analytics No Changes to

Applications

Trivial to

Implement

Oracle Database In-Memory Goals

100x

27


Row Format Databases vs. Column Format Databases

Rows Stored Contiguously

Transactions run faster on row format

– Example: Query or Insert a sales order – Fast processing few rows, many columns

Columns Stored

Contiguously

Analytics run faster on column format

– Example : Report on sales totals by region – Fast accessing few columns, many rows

SALES

SALES

28


Oracle Big Data Analytics

Column Format

Memory

Row Format

Memory

Analytics OLTP Sales Sales

Data Mashups

In-Memory ROLAP

In-Memory M-OLAP

Predictive Analytics Sales

In-Memory Database 100X Faster: OLTP & Analytics

Oracle BI: Visual Analytics Interactive Visual Data Analysis


Oracle In-Memory Requires Zero Application Changes

Full Functionality - ZERO restrictions on SQL

Easy to Implement - No migration of data

Fully Compatible - All existing applications run unchanged

Fully Multitenant - Oracle In-Memory is Cloud Ready

Uniquely Achieves All In-Memory Benefits With No Application Changes

30


Oracle Database In-Memory is Trivial to Deploy No changes required to existing applications

Step1: Configure Memory Capacity

– inmemory_area = XXXX GB

Step2: Configure tables or partitions to be in memory

– alter table | partition … inmemory;

Step3: Drop analytic indexes to speed up OLTP


Oracle Database In-Memory Customer Experience Schneider Electric analysis of up to 2 billion General Ledger entries

0

20

40

60

80

100

2B 300K 30K

S

e

c

o

n

d

s

Analytics Query Results

Row Format Column Format

• Analytic queries 7-128x faster

• OLTP transactions 5-9x faster

• 76% storage savings


Oracle Applications In-Memory Examples

Oracle Application Module Improvement Elapsed Time

In-Memory Cost Management 1003x Faster 58 hours to 3.5 mins

In-Memory - Financial Analyzer 1,354x Faster 4.3 hours to 11 seconds

In-Memory Sales Order Analysis 1,762x Faster 22.5 minutes to < 1 sec

Subledger Period Close Exceptions 200x Faster 600 seconds to 3 secs

Call Center Ad-hoc query pattern 1247x Faster 129 seconds to < 1 secs

33


Getting The Most From In-Memory

• Fast cars speed up travel, not meetings

• In-Memory speeds up analytic data access, not:

– Network round trips, logon/logoff

– Parsing, PL/SQL, complex functions

– Data processing (as opposed to access) • Complex joins or aggregations where not much data is filtered before processing

– Load and select once – Staging tables, ETL, temp tables

Understand Where it Helps

34

Know your bottleneck!


Become a Real-Time Enterprise Using Oracle Database In-Memory

• Data-Driven

• Get immediate answers to any question with real-time analytics

• Agile

• Eliminate latency with analytics directly on OLTP data

• Efficient

• Easily and Non-disruptive deployment accelerates all applications

35

AGILE

EFFICIENT

DATA-

DRIVEN


“Now we can run time-sensitive analytical queries directly against our OLTP database. This is something we wouldn't have dreamt of earlier.”

– Arup Nanda, Enterprise Architect Starwood Hotels and Resorts

36


“Full support for RAC scale-out means Oracle Database In-Memory can be used on our largest Data Warehouse, enabling more near real-time analytics.”

– Sudhi Vijayakumar, Senior Oracle DBA Yahoo Inc.

37


“Downtime is extremely costly for our business. Oracle’s In-Memory architecture takes the right approach to balancing real-time speed with continuous availability.”

– Jens-Christian Pokolm Analyst IT-DB Architecture & Engineering Postbank Systems AG

38


Oracle Exalytics In-Memory Machine

Oracle Confidential – Internal/Restricted/Highly Restricted 39

Best Platform for…

Optimum Performance

Best End-to-end Experience

Best Total Cost of Ownership


Oracle Exalytics In-Memory Machine

Oracle Confidential – Internal/Restricted/Highly Restricted 40

Best Platform for…

Oracle’s strategic platform for Analytic applications

Business Intelligence

Enterprise Performance Management

Endeca Information Discovery


Introducing Exalytics X4-4

• Brand new Software

– Oracle Database In-Memory Option: 100% Data In-Memory

– Oracle Essbase In-Memory option: Maximum concurrency; 2X faster on X4-4

• Brand new Hardware

– Intel Xeon E7-V2; 50% faster clock speed; 50% more processing cores; co-engineered with Oracle

– 50% more memory; 100% more flash; 33% more storage; 10X more networking

Continued exploitation of D-RAM & Flash to make Analytics 100X faster


Exalytics evolution

X2-4 X3-4

X4-4

Upgrade Kit

1 TB RAM

4 Intel® Xeon® E7-4870

40 physical cores total

3.6 TBs of raw disk capacity

40 Gbps InfiniBand – 2 ports

10 Gbps Ethernet – 2 ports


2 TB RAM

4 Intel® Xeon® E7-4870


2.4TB PCI Flash





2 TB DRAM

4 Intel® Xeon® E7-8895 v2


2.4TB PCI Flash



10 Gbps Ethernet- 4 ports

Flash Upgrade kit

Memory and Flash

Upgrade kit

T5-8

+ 1TB RAM + 2.4 TB PCI Flash + bigger disks

+ Super-SKU CPU + even bigger disks


• Rapid global growth since GA

• 100+ certified analytic applications and tools, including full suite of OBI and Hyperion applications

• Proven ROI based on superfast analytics and TCO advantages

Oracle Exalytics Solid Customer Momentum

43


Oracle Big Data Stack

z BY INDUSTRY & LINE OF BUSINESS B

IG D

ATA

A

PP

LIC

ATI

ON

S

DISCOVERY

BU

SIN

ESS

AN

ALY

TIC

S

BUSINESS ANALYTICS

DATA RESERVOIR

BIG

DA

TA

MA

NA

GEM

ENT

DATA WAREHOUSE

SOU

RC

ES


Questions & Answers

in-memory analytics · pdf filebi andweb & social data reporting discovery lab actionable...

Documents