news in sas data management · high level capabilities sas data management for hadoop at a glance...

Copyr i g h t © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d.

Company Confidential - For Internal Use Only

Copyright © 2014, SAS Institute Inc. All rights reserved.

NEWS IN SAS DATA MANAGEMENT

TERJE VATLE, NORDIC CENTER OF EXCELLENCE

ENTERPRISE ANALYTICS

2Copyright © 2010, SAS Institute Inc. All rights reserved.

(1) Kostnadseffektiv skalering med SAS på Hadoop

(2) Se data utover datavarehuset med SAS Federation Server

(3) Gjør det enklere å kombinere og overvåke ETL-jobber med SAS Data Management Advanced

FANS 10/9: Nyheter i SAS Data Management

(4) Ny måte å spore sammenhenger i metadata med ny SAS Lineage

3Copyright © 2010, SAS Institute Inc. All rights reserved.

(1) Kostnadseffektiv skalering med SAS på Hadoop

4Copyright © 2010, SAS Institute Inc. All rights reserved.

Hvorfor Hadoop? (Line of business)

5Copyright © 2010, SAS Institute Inc. All rights reserved.

Hvorfor Hadoop? (IT)

6Copyright © 2010, SAS Institute Inc. All rights reserved.

Hvorfor Hadoop?

7Copyright © 2010, SAS Institute Inc. All rights reserved.

Hva er Hadoop?

“Hadoop is one way of using a set of cheap computers to store an enormous amount of

data and then to process that data in parallel.“

Forenklet: Gjør at mange, billige maskiner ser ut som én stor, med redundans og skalerbarhet

8Copyright © 2010, SAS Institute Inc. All rights reserved.

Hvilke muligheter gir Hadoop?

Skalere analyse på

måter som tidligere ikke

var mulig

Ved å hoppe bukk over

begrensningene til dagens databaser

gir det nye muligheter for

selvbetjening innen business

intelligence

Nye muligheter innen data

management til å laste, vaske,

transformere og forvalte større

datamengder

Støtter behovene for data, BI og

analytics på en mer kostnadseffektiv

måte

Connected cars: a

big data headache or

opportunity for

insurers?Telematics take-up may be only by five percent

of the market right now – but insurance

companies must prepare for when it reaches

critical mass

EKSEMPEL FRA FORSIKRINGSBRANSJEN

WHY SHOULD YOU CARE?

SOME OF THE ORGANISATIONS THAT PUBLICLY STATE USE OF HADOOP

Copyr i g h t © 2013, SAS Ins t i tu t e Inc . A l l r ights reser ve d.

Hadoop i SAS er en av flere muligheter

HADOOP

Store it All

IN-DATABASE

Manage in Place

IN-MEMORY

Interactive Analysis

GRID

Distribute Workload

12Copyright © 2010, SAS Institute Inc. All rights reserved.

Hva gjør SAS med Hadoop?

IDENTIFY /

FORMULATE

PROBLEM

DATA

PREPARATION

DATA

EXPLORATION

TRANSFORM

& SELECT

BUILD

MODEL

VALIDATE

MODEL

DEPLOY

MODEL

EVALUATE /

MONITOR

RESULTS

Hadoop as a Data Platform(standalone or as part of a broader ecosystem)

Hadoop as a core component of the next

generation of BI and Analytics

.. to support innovative business usage.. to support an IT Transformation

Copyr i g h t © 2013, SAS Ins t i tu t e Inc . A l l r ights reser ve d.

HIGH LEVEL CAPABILITIES SAS DATA MANAGEMENT FOR HADOOP AT A GLANCE

Self-service data query and

transformation

in Hadoop

SAS/Access to Hadoop,

SAS/Access to Impala,

Base SAS

SAS Event Stream Processing

Engine

SAS Federation Server

Web Based Data

Management

interface for Hadoop

SAS Data Quality Accelerator

for Hadoop

SAS Code Accelerator for

Hadoop

New in August 2014 New in August 2014

SAS DI Studio

Hva gjør SAS med Hadoop?

Copyr i g h t © 2013, SAS Ins t i tu t e Inc . A l l r ights reser ve d.

Web Based DM

interface for

Hadoop

Self-service data

manipulation in

Hadoop + Loading

into LASR

SAS/Access to Hadoop, SAS/Access to Impala,

BASE SAS, SAS Federation Server

SA

S E

vent S

tream

Pro

cessin

g

Engin

e

Access to HDFS,

Hadoop scripting (Pig,

Map Reduce…) and

HIVE/Cloudera Impala

through SAS coding and

GUI + Reuse of DQ and

ETL/ELT processing

SAS DI Studio

EDW

On-Hadoop data processing

Data virtualization &

masking across Hadoop

and other data stores

All other DM Clients

Hadoop Accelerated Clients BAU SAS DM clients

Third party clients +

SAS BI + SAS

Analytics + SAS

Solutions

Other clients

Bring streaming data from various sources

into Hadoop and/or the RDBMS or generate

events before data hits downstream store

Hva gjør SAS med Hadoop?

Copyr i g h t © 2013, SAS Ins t i tu t e Inc . A l l r ights reser ve d.

Hvordan kan SAS bruke Hadoop?

16Copyright © 2010, SAS Institute Inc. All rights reserved.

(2) Se data utover datavarehuset med SAS Federation Server

Copyr i g h t © 2013, SAS Ins t i tu t e Inc . A l l r ights reser ve d.

Bruk av SAS Federation Server

Copyr i g h t © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d.

SAS® FEDERATION

SERVER

CENTRALIZED

SECURITY

Permissions granted at a higher level

will be inherited at the lower levels in

accordance with the SAS Federation

Server Privilege Inheritance

Copyr i g h t © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d.

SAS® FEDERATION

SERVER

Data masking

• Data masking is implemented using a set of rules and arguments

• applied to a column of a table or view managed by Federation Server

• The rule types supported are

• ENCRYPT/DECRYPT

• HASH

The HASH rule

hashes a single value

into a fixed-length

hash digest or HMAC

string and is not

reversible

20Copyright © 2010, SAS Institute Inc. All rights reserved.

(3) Gjør det enklere å kombinere og overvåke ETL-jobber med SAS Data Management Advanced

Copyr i g h t © 2013, SAS Ins t i tu t e Inc . A l l r ights reser ve d.

Enklere å kombinere SAS-jobber (DI, DQ og 3. part)

Copyr i g h t © 2013, SAS Ins t i tu t e Inc . A l l r ights reser ve d.

Enklere å overvåke SAS-jobber (DI, DQ og 3. part)

23Copyright © 2010, SAS Institute Inc. All rights reserved.

(4) Ny måte å spore sammenhenger i metadata med ny SAS Lineage

Copyr i g h t © 2013, SAS Ins t i tu t e Inc . A l l r ights reser ve d.

Spore sammenhenger med SAS Lineage

25Copyright © 2010, SAS Institute Inc. All rights reserved.

(1) Kostnadseffektiv skalering med SAS på Hadoop

(2) Se data utover datavarehuset med SAS Federation Server

(3) Gjør det enklere å kombinere og overvåke ETL-jobber med SAS Data Management Advanced

FANS 10/9: Nyheter i SAS Data Management

(4) Ny måte å spore sammenhenger i metadata med ny SAS Lineage

Copyr i g h t © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d. sas.com

QUESTIONS?