powering the future of data
TRANSCRIPT
![Page 1: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/1.jpg)
1 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Powering the Future of Data Pasi Vuorela Nordic Sales Manager
![Page 2: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/2.jpg)
2 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
EMBRACE AN OPEN APPROACH
MASTER THE VALUE OF DATA
EVERY BUSINESS IS A DATA BUSINESS
![Page 3: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/3.jpg)
3 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
The Growth of the Flow of Data
Much of the new data exists in-‐flight between systems and devices as part of the Internet of Anything NEW
TRADITIONAL
Ability to consu
me data
The Opportunity Unlock transformaKonal business value from a full fidelity of data and analyKcs for all data.
GeolocaKon
Server logs
Files & emails
ERP, CRM, SCM
TradiFonal Data Sources
Internet of Anything
Sensors and machines
Clickstream
Web & social
![Page 4: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/4.jpg)
4 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Data Unleashed: More Volume & More Types
I N C R E A S I N G D A T A V A R I E T Y A N D C O M P L E X I T Y
USER GENERATED CONTENT
MOBILE WEB
SMS/MMS
SENTIMENT
EXTERNAL DEMOGRAPHICS
HD VIDEO
SPEECH TO TEXT
PRODUCT/ SERVICE LOGS
SOCIAL NETWORK
BUSINESS DATA FEEDS
USER CLICK STREAM
WEB LOGS
OFFER HISTORY DYNAMIC PRICING
A/B TESTING
AFFILIATE NETWORKS
SEARCH MARKETING
BEHAVIORAL TARGETING
DYNAMIC FUNNELS PAYMENT RECORD
SUPPORT CONTACTS
CUSTOMER TOUCHES PURCHASE
DETAIL
PURCHASE RECORD
SEGMENTATION OFFER DETAILS
PETABYTES
TERABYTES
GIGABYTES
EXABYTES
E R P
B I G D ATA
W E B
C R M
I O T D ATA SENSORS INFOTAINMENT SYSTEMS WEARABLE DEVICES
CYBER SECURITY LOGS
CONNECTED VEHICLES
MACHINE DATA
ZETTABYTES
![Page 5: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/5.jpg)
5 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Blind Spots Block Your Ability to Use All the Data
GROUP 3
GROUP 2 GROUP 4
GROUP 1 INTERNET
OF ANYTHING
Fragmented data-‐at-‐rest increases the cost of insight
Data-‐in-‐moKon streams through your blind spots
![Page 6: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/6.jpg)
6 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
AcFonable Intelligence from Connected Data PlaSorms
à Capturing perishable insights from data in moKon
à Ensuring rich, historical insights on data at rest
à Necessary for modern data applicaKons
DATA AT REST DATA IN MOTION
ACTIONABLE INTELLIGENCE
Modern Data ApplicaFons
Hortonworks DataFlow
Hortonworks Data PlaSorm
![Page 7: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/7.jpg)
7 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Connected Data PlaSorms Enable Architectural TransformaFons
Data in MoFon (Cloud)
Data in MoFon
(on-‐premises)
Data at Rest
(on-‐premises)
Edge Data
Data in MoFon
Edge AnalyFcs
Data at Rest (Cloud)
Edge Data
Data at Rest
(on-‐premises)
Closed Loop AnalyFcs
Machine Learning
Deep Historical Analysis
![Page 8: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/8.jpg)
8 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Hortonworks® customers leverage our Connected Data Pla[orms to transform their industries – renovaKng their IT architectures and innovaKng with their Data in MoKon or Data at Rest to power acKonable intelligence through Modern Data ApplicaKons.
Social Mapping
Payment Tracking
Factory Yields
Defect DetecKon
Call Analysis Machine Data
Product Design M & A
Due Diligence
Next Product Recs
Cyber Security
Risk Modeling
Ad Placement
ProacKve Repair
Disaster MiKgaKon
Investment Planning
Inventory PredicKons
Customer Support
SenKment Analysis
Supply Chain
Ad Placement
Basket Analysis Segments
Cross-‐ Sell
Customer RetenKon
Vendor Scorecards
OpKmize Inventories
OPEX ReducKon
Mainframe Offloads
Historical Records
Data as a Service
Public Data Capture
Fraud PrevenKon
Device Data Ingest
Rapid ReporKng
Digital ProtecKon
8 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
![Page 9: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/9.jpg)
9 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Renovation Examples
We’ve helped hundreds of customers optimize their data architectures: • Major US retailer – were spending $50k/TB on
EDW, 37% of processing was ETL • Major global bank – avoided $46 mil EDW
expansion • British Airways – moved 75% of data out of
EDW into HDP • Centrica British Gas – avoided 5 mil GBP EDW
expansion and enriched environment with smart meter data
• TrueCar – $0.23/GB with HDP vs. $19/GB with traditional EDW
• Neustar – moved from keeping 1% of data for 65 days to keeping 100% for 2 years+
![Page 10: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/10.jpg)
10 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Merck’s Journey
The Golden Batch
ScienFfic Search
Sensor Data Storage
Vaccine Yield OpFmizaFon
Innovate
Renovate The Journey to the Golden Batch
à Combined 10 years data on one vaccine: 1 billion records
à 5.5 million batch comparisons
à 1st year yield boost of 40K more doses à $10M profit impact
à McKinsey: 50% yield increase
Epidemiology
![Page 11: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/11.jpg)
11 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Symantec’s Journey
Digital Security
Metadata Capture
Threat PredicKons
Aiacker DetecKon
Unified Security
Security Log Analysis
Threat Archive
Device Data Ingest
Threat DetecKon
Greenplum Offload
Innovate
Renovate
Data Science Speeds Time to ProtecFon
à Threat detecKon latency reduced from 4 hours to 2 seconds
à Time to protecKon improved 5000x
à Machine learning over tens of petabytes of historical data predicts threats to customers
à Cloud team uses Ambari and Cloudbreak for dynamic clusters to meet peak workloads
![Page 12: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/12.jpg)
12 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Case Study Mercy’s Journey
Beier Health Billing Vital Sign
Monitoring
Single PaFent Record
Lab Notes Archive
Privacy Database
Medical Decision Support
Device Data Ingest
PrevenFve Care
Epic Enrichment
OPEX Efficiency
Epic EMR ReplicaFon
Innovate
Renovate
Be^er Health Through Data
à Searches of free-‐text lab notes, speed researcher insight from “never” to “seconds”
à Ingest of ICU vital signs increased by 900X, lemng clinicians respond more quickly
à Mercy is building real-‐Kme tools to support surgical decisions and prevenKve care
![Page 13: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/13.jpg)
13 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Case Study Progressive’s Journey
Rewarding Safer Drivers and Improving Traffic Safety
à Snapshot plug-‐in devices capture driving detail
à Progressive stores more than 10 billion miles driven
à Through a web app, customers can review their own driving detail and improve their safety
à Snapshot and usage-‐based insurance drove $2.6 billion in 2014 Progressive premiums
Innovate
Renovate
Safe Roads
Claims Notes Mining
Individual Driving Histories
Usage-‐Based Insurance (UBI)
Web Log Analysis
Online Ad Placement
Sensor Data Ingest
![Page 14: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/14.jpg)
14 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
The Hortonworks SoluFon Powering the Future of Data
![Page 15: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/15.jpg)
15 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
DATA AT REST DATA IN MOTION
ACTIONABLE INTELLIGENCE
Modern Data ApplicaFons
PERISHABLE INSIGHTS
HISTORICAL INSIGHTS
INTERNET OF
ANYTHING
Hortonworks DataFlow
Hortonworks Data PlaSorm
Hortonworks Delivers Connected Data PlaSorms
![Page 16: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/16.jpg)
16 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Secure
Real-‐Fme
AdapFve
Integrated
Hortonworks DataFlow for Data in MoFon Powered by Apache NiFi
![Page 17: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/17.jpg)
17 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
A SimplisFc View of Enterprise Data Flows
The Data Flow Thing
Process and Analyze Data Acquire Data
Store Data
![Page 18: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/18.jpg)
18 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
A RealisFc View of Enterprise Data Flow
![Page 19: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/19.jpg)
19 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Real-‐Time, Visual Control of Data Flows
Add and Adjust Data Sources to maximize the opportunity that you capture from perishable insights
Visually Trace the Data Path to manage the what, who, where and how around data in moKon
Dynamically Adjust the Pipeline to match the dataflow with your bandwidth
HORTONWORK S DA TA F LOW Add and adjust
data sources
Visually trace the data path
Dynamically adjust the pipeline
![Page 20: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/20.jpg)
20 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Hortonworks Data PlaSorm for Data at Rest Powered by Open Enterprise Hadoop
Open
Interoperable
Ready
Central
![Page 21: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/21.jpg)
21 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
100% Open Source Connected Data PlaSorms
MA X IMUM C OMMUN I T Y I N N O V A T I O N
T H E I N N O V A T I O N A D V A N T A G E
P RO P R I E T A R Y H A DOO P
T IM E
INNOVATIO
N
O P E N C OMMUN I T Y
Eliminates Risk of vendor lock-‐in by delivering 100% Apache open source technology
Maximizes Community InnovaFon with hundreds of developers across hundreds of companies
Integrates Seamlessly through commiied co-‐engineering partnerships with other leading technologies
![Page 22: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/22.jpg)
22 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
100% Open Approach = Fastest Path to InnovaFon
HORTONWORKS DATA PLATFORM
Ha
doop
&
YA
RN
Flume
Oozie
Pig
Hive
Tez
Sqo
op
Cloud
break
Amba
ri
Slid
er
Kag
a
Kno
x
Solr
Zoo
keep
er
Spa
rk
Falcon
Ran
ger
HBa
se
Atla
s
Accum
ulo
Storm
Pho
enix
4.10.2
DATA MGMT DATA ACCESS GOVERNANCE & INTEGRATION OPERATIONS SECURITY
HDP 2.2 Dec 2014
HDP 2.1 April 2014
HDP 2.2 Dec 2014
HDP 2.1 April 2014
HDP 2.0 Oct 2013 0.12.0 0.12.0
0.12.1 0.13.0 0.4.0
1.4.4 1.4.4 3.3.2 3.4.5
0.4.0 0.5.0
0.14.0 0.14.0 3.4.6 0.5.0 0.4.0 0.9.3 0.5.2
4.0.0 4.7.2
1.2.1 0.60.0 0.98.4 4.2.0 1.6.1 0.6.0 1.5.2 1.4.5 4.1.0 1.7.0
1.4.0 1.5.1 4.0.0
1.3.1
1.5.1 1.4.4 3.4.5
1.3.1
2.2.0
2.4.0
2.6.0
2.7.1 1.4.6 1.0.0 0.6.0 0.5.0 2.1.0 0.8.2 3.4.6 1.5.2 5.2.1 0.80.0 1.1.1 0.5.0 1.7.0 4.4.0 0.10.0 0.6.1 0.7.0 1.2.1 0.15.0 HDP 2.3 July 2015 4.2.0
Ongoing InnovaFon in Apache
0.96.1
0.98.0 0.9.1
0.8.1
![Page 23: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/23.jpg)
23 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
HDP delivers a completely open data plaSorm
Hortonworks Data PlaSorm 2.3
Hortonworks Data PlaSorm provides Hadoop for the Enterprise: a centralized architecture of core enterprise services, for any applicaKon and any data.
Completely Open
• HDP incorporates every element required of an enterprise data platform: data storage, data access, governance, security, operations
• All components are developed in open source and then rigorously tested, certified, and delivered as an integrated open source platform that’s easy to consume and use by the enterprise and ecosystem.
YARN: Data Operating System (Cluster Resource Management)
1 ° ° ° ° ° ° °
° ° ° ° ° ° ° °
Apa
che
Pig
° °
° °
° ° °
° ° °
HDFS (Hadoop Distributed File System)
GOVERNANCE BATCH, INTERACTIVE & REAL-TIME DATA ACCESS
Apache Falcon
Apa
che
Hiv
e C
asca
ding
A
pach
e H
Bas
e A
pach
e A
ccum
ulo
Apa
che
Sol
r A
pach
e S
park
Apa
che
Sto
rm
Apache Sqoop
Apache Flume
Apache Kafka
SECURITY
Apache Ranger
Apache Knox
Apache Falcon
OPERATIONS
Apache Ambari
Apache Zookeeper
Apache Oozie
Apache Atlas Apache Cloudbreak
Apache Atlas
![Page 24: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/24.jpg)
24 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Hortonworks Reference Architecture
![Page 25: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/25.jpg)
25 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
1600+ Partners
3000+ members
15,000+ Weekly visitors
ParFcipaFng with a Growing and Thriving Ecosystem
![Page 26: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/26.jpg)
26 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Why Hortonworks? Powering the Future of Data
![Page 27: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/27.jpg)
27 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Hortonworks Influences the Apache Community
APACHE HADOOP COMMITT ERS
We Employ the Commi^ers one third of all commiiers to the Apache® Hadoop™ project, and a majority in other important projects
Our Commi^ers Innovate and expand Open Enterprise Hadoop
We Influence the Hadoop Roadmap by communicaKng important requirements to the community through our leaders
![Page 28: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/28.jpg)
28 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
STORA
GE STO
RAGE
Hortonworks Provides Full Lifecycle Support
ARCHITECT &
DEVELOP
DEPLOY
OPERATE
Project 1
Project 5
Project 4
Project 3
Project 2
Project 6
EXPAND
Hortonworks ExperFse from the original architects of Apache Hadoop and Apache NiFi
Annual SubscripFons align your success with ours
Apache Commi^ers advocate for the requirements of our customers and provide them roadmap visibility to help guide their journey
Expert ConsulFng and Training help you and your team get the most from your Open Data Pla[orms
![Page 29: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/29.jpg)
29 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Hortonworks Training & Certification
![Page 30: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/30.jpg)
30 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Hortonworks Delivers ProacFve Support
Hortonworks SmartSense™ with machine learning and predicKve analyKcs on your cluster Integrated Customer Portal with knowledge base and on-‐demand training
Knowledge Base
Integrated Customer Portal
On-‐Demand Training
Customer Environment Any cloud • Hybrid Environment • MulK-‐tenant
Hortonworks SmartSense
![Page 31: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/31.jpg)
31 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
About Hortonworks Customer Momentum à ~800 customers (as of November 4, 2015)
à 152 customers added in Q3 2015
à Publicly traded on NASDAQ: HDP
The Leader in Connected Data PlaSorms à Hortonworks DataFlow for data in moKon
à Hortonworks Data Pla[orm for data at rest
à Powering new modern data applicaKons
Partner for Customer Success à Leader in open-‐source community, focused
on innovaKon to meet enterprise needs
à Unrivaled support subscripKons
Founded in 2011
Original 24 Architects, Developers, Operators of Hadoop from Yahoo!
800+ EMP LO Y E E S
1500+ E CO S Y S T EM P A R T N E R S
![Page 32: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/32.jpg)
32 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Thank You
![Page 33: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/33.jpg)
33 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Securing Your Data with Tag-‐Based Access Policies
Manage Access Policies and Audit Logs
Track Metadata and Lineage
![Page 34: Powering the Future of Data](https://reader034.vdocument.in/reader034/viewer/2022042610/58aa9be91a28ab85678b6321/html5/thumbnails/34.jpg)
34 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Data-‐Defined Cyber Security – Apache Metron (incubaFng)
Enriched 360
Correlated
Searchable
Discoverable
3rd Party Feeds
StaFc Rules
ML Models
IOC Sharing
Parsers
Enrichers
Threat Intel
UI Widgets
SIEM
PCAP Replay
Evidence Store
HunFng PlaSorm
Check Out the Technical Preview!
Tracing the Flow of a Security Telemetry Event though Metron
Pluggable Framework
Security ApplicaFon
Security Data Lake
Threat Intelligence