news in sas data management · high level capabilities sas data management for hadoop at a glance...
TRANSCRIPT
Copyr i g h t © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d.
Company Confidential - For Internal Use Only
Copyright © 2014, SAS Institute Inc. All rights reserved.
NEWS IN SAS DATA MANAGEMENT
TERJE VATLE, NORDIC CENTER OF EXCELLENCE
ENTERPRISE ANALYTICS
2Copyright © 2010, SAS Institute Inc. All rights reserved.
(1) Kostnadseffektiv skalering med SAS på Hadoop
(2) Se data utover datavarehuset med SAS Federation Server
(3) Gjør det enklere å kombinere og overvåke ETL-jobber med SAS Data Management Advanced
FANS 10/9: Nyheter i SAS Data Management
(4) Ny måte å spore sammenhenger i metadata med ny SAS Lineage
3Copyright © 2010, SAS Institute Inc. All rights reserved.
(1) Kostnadseffektiv skalering med SAS på Hadoop
4Copyright © 2010, SAS Institute Inc. All rights reserved.
Hvorfor Hadoop? (Line of business)
5Copyright © 2010, SAS Institute Inc. All rights reserved.
Hvorfor Hadoop? (IT)
6Copyright © 2010, SAS Institute Inc. All rights reserved.
Hvorfor Hadoop?
7Copyright © 2010, SAS Institute Inc. All rights reserved.
Hva er Hadoop?
“Hadoop is one way of using a set of cheap computers to store an enormous amount of
data and then to process that data in parallel.“
Forenklet: Gjør at mange, billige maskiner ser ut som én stor, med redundans og skalerbarhet
8Copyright © 2010, SAS Institute Inc. All rights reserved.
Hvilke muligheter gir Hadoop?
Skalere analyse på
måter som tidligere ikke
var mulig
Ved å hoppe bukk over
begrensningene til dagens databaser
gir det nye muligheter for
selvbetjening innen business
intelligence
Nye muligheter innen data
management til å laste, vaske,
transformere og forvalte større
datamengder
Støtter behovene for data, BI og
analytics på en mer kostnadseffektiv
måte
Connected cars: a
big data headache or
opportunity for
insurers?Telematics take-up may be only by five percent
of the market right now – but insurance
companies must prepare for when it reaches
critical mass
EKSEMPEL FRA FORSIKRINGSBRANSJEN
WHY SHOULD YOU CARE?
SOME OF THE ORGANISATIONS THAT PUBLICLY STATE USE OF HADOOP
Copyr i g h t © 2013, SAS Ins t i tu t e Inc . A l l r ights reser ve d.
Hadoop i SAS er en av flere muligheter
HADOOP
Store it All
IN-DATABASE
Manage in Place
IN-MEMORY
Interactive Analysis
GRID
Distribute Workload
12Copyright © 2010, SAS Institute Inc. All rights reserved.
Hva gjør SAS med Hadoop?
IDENTIFY /
FORMULATE
PROBLEM
DATA
PREPARATION
DATA
EXPLORATION
TRANSFORM
& SELECT
BUILD
MODEL
VALIDATE
MODEL
DEPLOY
MODEL
EVALUATE /
MONITOR
RESULTS
Hadoop as a Data Platform(standalone or as part of a broader ecosystem)
Hadoop as a core component of the next
generation of BI and Analytics
.. to support innovative business usage.. to support an IT Transformation
Copyr i g h t © 2013, SAS Ins t i tu t e Inc . A l l r ights reser ve d.
HIGH LEVEL CAPABILITIES SAS DATA MANAGEMENT FOR HADOOP AT A GLANCE
Self-service data query and
transformation
in Hadoop
SAS/Access to Hadoop,
SAS/Access to Impala,
Base SAS
SAS Event Stream Processing
Engine
SAS Federation Server
Web Based Data
Management
interface for Hadoop
SAS Data Quality Accelerator
for Hadoop
SAS Code Accelerator for
Hadoop
New in August 2014 New in August 2014
SAS DI Studio
Hva gjør SAS med Hadoop?
Copyr i g h t © 2013, SAS Ins t i tu t e Inc . A l l r ights reser ve d.
Web Based DM
interface for
Hadoop
Self-service data
manipulation in
Hadoop + Loading
into LASR
SAS/Access to Hadoop, SAS/Access to Impala,
BASE SAS, SAS Federation Server
SA
S E
vent S
tream
Pro
cessin
g
Engin
e
Access to HDFS,
Hadoop scripting (Pig,
Map Reduce…) and
HIVE/Cloudera Impala
through SAS coding and
GUI + Reuse of DQ and
ETL/ELT processing
SAS DI Studio
EDW
On-Hadoop data processing
Data virtualization &
masking across Hadoop
and other data stores
All other DM Clients
Hadoop Accelerated Clients BAU SAS DM clients
Third party clients +
SAS BI + SAS
Analytics + SAS
Solutions
Other clients
Bring streaming data from various sources
into Hadoop and/or the RDBMS or generate
events before data hits downstream store
Hva gjør SAS med Hadoop?
Copyr i g h t © 2013, SAS Ins t i tu t e Inc . A l l r ights reser ve d.
Hvordan kan SAS bruke Hadoop?
16Copyright © 2010, SAS Institute Inc. All rights reserved.
(2) Se data utover datavarehuset med SAS Federation Server
Copyr i g h t © 2013, SAS Ins t i tu t e Inc . A l l r ights reser ve d.
Bruk av SAS Federation Server
Copyr i g h t © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d.
SAS® FEDERATION
SERVER
CENTRALIZED
SECURITY
Permissions granted at a higher level
will be inherited at the lower levels in
accordance with the SAS Federation
Server Privilege Inheritance
Copyr i g h t © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d.
SAS® FEDERATION
SERVER
Data masking
• Data masking is implemented using a set of rules and arguments
• applied to a column of a table or view managed by Federation Server
• The rule types supported are
• ENCRYPT/DECRYPT
• HASH
The HASH rule
hashes a single value
into a fixed-length
hash digest or HMAC
string and is not
reversible
20Copyright © 2010, SAS Institute Inc. All rights reserved.
(3) Gjør det enklere å kombinere og overvåke ETL-jobber med SAS Data Management Advanced
Copyr i g h t © 2013, SAS Ins t i tu t e Inc . A l l r ights reser ve d.
Enklere å kombinere SAS-jobber (DI, DQ og 3. part)
Copyr i g h t © 2013, SAS Ins t i tu t e Inc . A l l r ights reser ve d.
Enklere å overvåke SAS-jobber (DI, DQ og 3. part)
23Copyright © 2010, SAS Institute Inc. All rights reserved.
(4) Ny måte å spore sammenhenger i metadata med ny SAS Lineage
Copyr i g h t © 2013, SAS Ins t i tu t e Inc . A l l r ights reser ve d.
Spore sammenhenger med SAS Lineage
25Copyright © 2010, SAS Institute Inc. All rights reserved.
(1) Kostnadseffektiv skalering med SAS på Hadoop
(2) Se data utover datavarehuset med SAS Federation Server
(3) Gjør det enklere å kombinere og overvåke ETL-jobber med SAS Data Management Advanced
FANS 10/9: Nyheter i SAS Data Management
(4) Ny måte å spore sammenhenger i metadata med ny SAS Lineage
Copyr i g h t © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d. sas.com
QUESTIONS?