empowering the data-driven organization - sas · patterns of using sas with hadoop for analytics...

Copyright © 2014, SAS Institute Inc. All rights reserved.

Empowering the Data-Driven OrganizationJeroen Dijkxhoorn, SASLars Slagboom, ABN AMRO

In 5 years from now…Elephants will rule the world

Acting on predictive Decisions will be standard

Real Time Analytics is to blame for a crash

Mobile User Interfacing will be the Standard

Data will be everywhere and Nobody knows where exactly


Trends Big Data, Storage, Hadoop & In-memory Technology

$- $20.000 $40.000 $60.000 $80.000 $100.000

Vertica

Teradata

Greenplum

Oracle

Microsoft PDW

Hadoop

Today 2009

Cost of Storage, Memory, Computing • In 2000 a GB of Disk $17 today < $0.07

• In 2000 a GB of Ram $1800 today < $10

• In 2009 a TB of RDBMS was $70K today < $ 20K

Cost per Terabyte

Technology Push: storage costs and CPU speed

To enable analytics in this changing environment, you need to:

Bring the Analytics to the Data…

…and run it in a distributed mode


Business pull: two Eras . . .two mindsets

Process-centric

Everything is

forbidden unless it is

permitted

Focus on cost control

Technology constrained

Discovery-centric

Everything is

permitted unless it is

forbidden

Focus on value

Technology empowered

To enable analytics in this changing environment, you need to:

Provide self-service analytic capabilities…

…and automate the decision making process


Data-Driven with Analytics as the main enabler


From Data to Decision

TEXT

MANAGE

DATA

EX

PL

OR

E

DA

TA

DEVELOP

MODELS

DE

PL

OY

&

MO

NIT

OR

Challenges:

• Growth in Demand

• Growth of Data

• Access to Talent

• Controlling Cost

Needs:

• Scale the Process

• Avoid Replication

• Increase Productivity

• Decouple Cost & Growth


SAS Directions to address these needs

Scale the Process

SPEED UP THE DATA TO DECISION LIFECYCLE

1. Event Stream Processing

2. High Performance Analytics

3. Decision Management

1

Avoid Replication

MOVE SAS PROCESSING TO THE DATA

1. In-Database Processing

2. Scoring Accelerators

3. Code Accelerators

2

Increase Productivity

PROVIDE INTERACTIVE, SELF-SERVICE INTERFACES

1. Data Loader for Hadoop

2. Visual Analytics, Visual Statistics & In-Memory Statistics

3. Move to responsive web-apps based on HTML5

3

Decouple Cost & Growth

SUPPORT IT COST EFFICIENCY EFFORTS

1. Span data and processing across a Grid or Cluster

2. Virtual Apps to deploy in Private, Public or Hybrid Cloud

3. On-premise deployment within 3 hours

4


…… …

……

on a single platform

annual savings

production time

19 models

€15 billion

−30%

Platform Strategy, Automotive Engineering


…

……

…

…

……

……

Risk

Sales

Partners

Fraud

Controlling

Marketing

Logistics

Purchasing

IT

Production

50% reduction in costs for BI/Analytics

Double the value of BI/Analytics projects

per year

Platform strategy: Basis of the Analytics Factory


Standardization Consolidation Industrialization

3 steps towards an Analytics Factory


Standardization

• Coming together by agreeing what capabilities to use

Consolidation

• Keeping together by centralizing the platform

Industrialization

• Working together by scaling and speeding up the process

3 steps towards an Analytics Factory

Data en Informatie bij ABN AMRO

Introductie

• ABN AMRO

• Enterprise Data & Information

22

23

Standardization Consolidation Industrialization

Standardization

Kenmerken

• Focus op systeemlandschap

• Iedereen zijn eigen voorkeur

• Data decentraal

Succesfactoren

• Externe druk

• Bedrijfsbreed thema

• Beleid

24

Standardization

Consolidation

Kenmerken

• Focus naar gebruiker

• Waarde van geïntegreerde data wordt onderkent

• Wachttijden in je datawarehouse ontwikkeling

Succesfactoren

• Introductie gebruikersteams

• Vermarkt je datawarehouse en BI omgeving

25

Consolidation

Industrialization

Kenmerken

• Focus op gebruik

• Snellere groei van data dan systemen

• Meer vraag dan aanbod

• Data is een keten

Succesfactoren

• Businessprocessen meenemen in je verandering

• Organiseer bronsystemen

26

Industrialization


Marc Lammers:

“50 keer 2% is ook 100%”


Back to the elephant…


Where is Hadoop being used for?

Hadoop as a Data PlatformHadoop as a core component of next

generation analytical platform

TEXT

MANAGE

DATA

EX

PL

OR

E

DA

TA

DEVELOP

MODELS

DE

PL

OY

&

MO

NIT

OR


Usage 1: Hadoop as Data Platform

Initiator

• This paradigm is mostly driven by IT

Drivers

• Increasing costs of data storage

• Increasing volume of data

• Latency to deliver information

Benefits

• Large-scale distributed storage and

batch processing


Ingest/Load Data

Cleanse & Transform

Data

Load Data To Other Sources

/ Memory

Metadata Documentation

Usage 1: Hadoop as data platform

• SAS/ACCESS

• SAS Data Management

• SAS Event Stream Processing

• SAS Federation Server

• SAS Data Loader for Hadoop

SAS Data Quality Accelerator for

Hadoop

SAS Code Accelerator for Hadoop

• SAS/ACCESS



• SAS Metadata Server


Usage 2: Hadoop as core of next generation analytical platform

TEXT

MANAGE

DATA

EX

PL

OR

E

DA

TA

DEVELOP

MODELS

DE

PL

OY

&

MO

NIT

OR

Initiator

• This paradigm is mostly driven by business

Drivers

• Increasing question to a variety of different

and additional information

• The need for a flexible data platform to

store, process, and analyze data at any

scale

Benefits

• The business can start thinking big again

when it comes to data


Usage 2: Hadoop as core of next generation analytical platform

TEXT

MANAGE

DATA

EX

PL

OR

E

DA

TA

DEVELOP

MODELS

DE

PL

OY

&

MO

NIT

OR

• SAS/ACCESS


• SAS Event Stream Processing


• SAS Data Loader for Hadoop

SAS Data Quality Accelerator for

Hadoop

SAS Code Accelerator for Hadoop • SAS Visual Analytics

• SAS In-memory

Statistics for Hadoop

• SAS HPA Products

• SAS Visual Statistics

• SAS In-memory Statistics

for Hadoop

• SAS Decision Manager

• SAS Scoring Accelerator for

Hadoop


Patterns of using SAS with Hadoop for Analytics & reporting

SAS with Hadoop

Hive

Extract from Hadoop pushing

some SAS pre-processing to

Hadoop

Embedded Process - Push

SAS data processing to

Hadoop with Map Reduce

SAS in Hadoop

Score A Code AImpala

In-Memory Analytics - Use

Hadoop for Storage persistence

and commodity computing.

SAS on Hadoop

HPA LASR


Continuity of Business

Bring SAS processing to the Data

Leverage Hadoop for new Technology offerings

Breadth and depth of modern analytic methods in Hadoop

SAS for Hadoop directions

DIRECTIONAL THEMES


13.30 Parallel Sessions

• Big Data and Visual Analytics – Rabobank

• Business Analytics – SAS

• Data Management – Ziekenhuis Gelderse Vallei

• Visual Analytics – Mercachem

13.30 Guided Tours

• Visual Analytics

15.45 Parallel Sessions

• Big Data and Visual Analytics – Belastingdienst

• Business Analytics – iBridge/ Randstad

• Data management – DSM

• Visual Analytics – H@nd

Information on breakouts Analytical platform

14.30 What’s Hot Sessions

• Big Data Analytics met Hadoop

• Data Management 3.0: What about Hadoop?

• What’s hot in Data Governance

• Modernisatie: meer mogelijkheden, minder risico’s

• Geavanceerd modelleren met SAS

• What’s new in SAS Visual Analytics 7.1

• Best Practices in Visualisatie en Dashboard design

14.30 Roundtables (max 20 pers.)

• The Analytical Bank

• Data monetization

empowering the data-driven organization - sas · patterns of using sas with hadoop for analytics...

Documents