you’ve got two weeks: why you shouldn’t discuss cool ideas near your...

22
You’ve got two weeks: Why you shouldn’t discuss cool ideas near your manager Dan Chaffey Itinerant Hacker and Solutions Engineer, Hortonworks

Upload: others

Post on 21-May-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data

You’ve got two weeks:Why you shouldn’t discuss cool ideas near your managerDan Chaffey

Itinerant Hacker and Solutions Engineer, Hortonworks

Page 2: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data

2 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

About HortonworksCustomer Momentum

~1000 customers

>150 customers added per quarter

Publicly traded on NASDAQ: HDP

The Leader in Connected Data Platforms

Hortonworks DataFlow for data in motion

Hortonworks Data Platform for data at rest

Powering new modern data applications

Partner for Customer Success

Leader in open-source community, focused on innovation to meet enterprise needs

Unrivaled support subscriptions

Founded in 2011

Original 24 Architects, Developers, Operators of Hadoop from Yahoo!

1000+E M P L O Y E E S

1500+E C O S Y S T E M PA R T N E R S

Page 3: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data

3 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Actionable Intelligence fromConnected Data Platforms

Capturing perishable insights from data in motion

Ensuring rich, historical insightson data at rest

Necessary for moderndata applications

DATA AT RESTDATA IN MOTION

ACTIONABLEINTELLIGENCE

Modern Data Applications

Hortonworks DataFlow

Hortonworks Data Platform

Page 4: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data

4 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Modern Data AppsCustom or Off the Shelf

Real-Time Cyber Securityprotects systems with superior threat detection

Smart Manufacturingdramatically improves yields by managing more variables in greater detail

Connected, Autonomous Vehiclesdrive themselves and improve road safety

Automatic Recommendation Enginesmatch products to preferences in milliseconds

Credit Fraud Preventionpredict in real-time based on each customer’s transaction record

DATA ATREST

DATA IN MOTION

ACTIONABLEINTELLIGENCE

Modern Data Applications

Hortonworks DataFlow

Hortonworks Data Platform

Page 5: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data

Credit Fraud Prevention on a Connected Data PlatformBuilt by: Kirk Haslbeck & Vadim Vaks

Page 6: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data

6 © Hortonworks Inc. 2011 – 2016. All Rights ReservedPage 6

Improved Experience

/Reduced Cost

Immediate Customer Feedback

Years of Customer

Transaction Data

Fraud Detection

Complete Customer

Profile

Real time ingest of

transactions

Proactively identify potential fraudulent transactions to protect the customer and improve customer experience• Proactively monitor every credit

card transaction using machine learning to catch potential fraud

• Customer Service Analyst reviews flagged transactions in real time via a next generation application running on the connected platform

• Control real time flow of data in and out of the connected platform to the various source and destination points

Innovate

Renovate

Purchase Behavior Insight

Journey to Fraud Detection

Page 7: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data

7 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

DATA ATREST

DATA IN MOTION

Actionable Intelligence fueled by Adaptive

Machine Learning

Customer Service Analyst

Customer

DATASOURCES

Fraud Detection Application Architecture

Page 8: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data

8 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Fraud Detection Demo Architecture

Distributed Storage: HDFS

Many Workloads: YARN

Real-time Serving

Machine Learning

UI and HTTP PubSub

Real-Time Data Movement

Data Science

Resource Allocation Interactive Query

Co

nfigu

ration

Man

agemen

t(A

mb

ari)Au

tho

rization

(Ran

ger)

Real Time Processing Inbound Messaging

D A T A I N

M O T I O N

D A T A I N

M O T I O N

D A T A I N

M O T I O N

Go

vernan

ce(A

tlas)

Page 9: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data

PiWiNiFi:Bi-directional IoTsensor networksBuilt by: Simon Elliston Ball & Dan Chaffey

Page 10: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data

10 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

System Architecture

HDF

HDF

HDF

BASTION

CLOUDVENUE INTERNAL

Page 11: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data

11 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Demo Features

BASTION

CLOUD

VENUE

INTERNAL

Booth DRDMZ Prod

Solr / Banana Zeppelin/Spark

Raspberry Pi

Codebase

Page 12: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data

12 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Data FlowData in Motion at the core and at the edge.

http://nifi.apache.org/http://hortonworks.com/hdf/http://community.hortonworks.com/

Page 13: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data

13 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

ResilientClustering, Availability, Load Balanced Site-to-Site

https://www.flickr.com/photos/rosefirerising/3818133872

Page 14: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data

14 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

DistributedSite-to-Site takes processing and prioritization right to the edge. Coming soon: MiNiFi!

Page 15: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data

15 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Thank You

Page 16: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data

16 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

References

http://hortonworks.com/blog/credit-card-fraud-prevention-on-a-connected-data-platform/

https://community.hortonworks.com/repos/27236/credit-fraud-prevention-demo.html– https://github.com/vakshorton/CreditCardTransactionMonitor

– https://github.com/vakshorton/CreditCardTransactionMonitorMobileApp

https://github.com/simonellistonball/PiWiNiFi

Page 17: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data

17 © Hortonworks Inc. 2011 – 2016. All Rights ReservedPage 17

Credit Fraud Analyst Inbox

Page 18: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data

18 © Hortonworks Inc. 2011 – 2016. All Rights ReservedPage 18

Hortonworks Data Flow

Page 19: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data

19 © Hortonworks Inc. 2011 – 2016. All Rights ReservedPage 19

Hortonworks Data Flow

Page 20: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data

20 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Page 21: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data

21 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Page 22: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data

22 © Hortonworks Inc. 2011 – 2016. All Rights Reserved