tquila single customer view - how big data can fuel your marketing strategy

14
Single Customer View (SCV) Presented by Vincent Spehner 20th Mars 2014 V0.1 A Hadoop based solution

Upload: vincent-spehner

Post on 15-Jan-2015

381 views

Category:

Data & Analytics


0 download

DESCRIPTION

 

TRANSCRIPT

Page 1: Tquila Single Customer View - How Big Data can fuel your Marketing strategy

Single Customer View (SCV)

Presented by

Vincent Spehner

20th Mars 2014

V0.1

A Hadoop based solution

Page 2: Tquila Single Customer View - How Big Data can fuel your Marketing strategy

©Tquila | Client Name | Date

Problem Statement

Our solution

Technical Architecture

Heroku and Treasure Data

Q&A and Next Steps

Agenda

Page 3: Tquila Single Customer View - How Big Data can fuel your Marketing strategy

©Tquila | Client Name | Date

Problem StatementWhy you need a SCV ?

Customer

Opportunities

Google Analytics

ERPWebsite data

Marketing Campaign

Social Media

• Aggregated view of customer data• Improve marketing ROI• Increase cross sell, Upsell

It’s all about those millions, if not billions, of records:• Spread on multiple places• Not connected• Sleeping and unvalued

Page 4: Tquila Single Customer View - How Big Data can fuel your Marketing strategy

©Tquila | Client Name | Date

Problem Statement

Building a Big Data infrastructure is complicated, expensive, and requires skills that are very hard to hire.

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

O/SHardw

are

Networking

Storage Layer

Frameworks

Processing Engines

Application Interfaces

Streaming Collection

Batch Collection

Business Rules

Data Integration

Consumption / Analytics

Big Data is complex

Page 5: Tquila Single Customer View - How Big Data can fuel your Marketing strategy

©Tquila | Client Name | Date

Problem Statement Lots of companies today produce Big

DataBut few have the resources to value them through a Single Customer View approach

60-70% of a company’s Big Data time & budget consumed by:

• Infrastructure setup & Maintenance• Building Data Collection & Storage

Flows• Hiring/Training Hadoop Expertise

It typically takes at least 6 months to get a Hadoop environment into production

Big Data takes time

Page 6: Tquila Single Customer View - How Big Data can fuel your Marketing strategy

©Tquila | Client Name | Date

Our solution

A Cloud-based solution accelerates and reduces your cost of implementing a Single Customer View.

Use this new customer insights straight into Salesforce Marketing tools (Exact Target, Pardot) and share new report

Cloud based SCV

• Salesforce based solution

• Hadoop on demand

All Your Data

Aggregate / Store / Analyze

Page 7: Tquila Single Customer View - How Big Data can fuel your Marketing strategy

Data preprocessing

Worker Worker Worker Worker Worker

Pig Transformation

Hive queries

Segmented

customers export (.json, .csv,

API)ExactTarget

Transactional DB

Page 8: Tquila Single Customer View - How Big Data can fuel your Marketing strategy

©Tquila | Client Name | Date

Technical ArchitectureData Acquisition

# Connect to APIs......

# Post events to Treasure DataTD.event.post(’login’,{:uid=>123,:pg=>13})

...

...TD.event.post('access’,{:uid=>123})

Treasure Data LibraryJava, Ruby, PHP, Perl, Python, Scala, Node.js

Heroku Workers

Reliable & Secure

(SSL)

Compressed

Upload

Automatic Micro-batching

Local buffering Fall-back

Network Tolerance

Treasure Agent

{ format:

JSON }

Page 9: Tquila Single Customer View - How Big Data can fuel your Marketing strategy

Technical ArchitectureData Storage

Stored “schema-less” as JSON Schema can be applied/updated

AFTER storage

Compressed & columnar format For higher query performance

Optimized for time-based filtering Quickly scale-up processing power

WITHOUT reloading/redistributing the data

time v

1384160400 {“ip”:”135.52.211.23”, “code”:”0”}

1384162200 {“ip”:”45.25.38.156”, “code”:”-1”, “action”:”upload}

1384164000 {“ip”:”97.12.76.55”, “code”:”99”,”status”:”ok”}

time ip : string code : int action : string

1384160400 135.52.211.23 0 NULL

1384162200 45.25.38.156 -1 upload

1384164000 97.12.76.55 99 NULL

Default (schema-less)

Schema applied

SELECT v[‘ip’] as ip, v[‘code’] as code …

~30% Faster

SELECT ip, code …

Page 10: Tquila Single Customer View - How Big Data can fuel your Marketing strategy

Technical ArchitectureData Analysis

Scripted Processing (Pig):- DataFu (LinkedIn)

- Piggybank (Apache)

Heavy Lifting SQL (Hive):- Hive’s Built-in UDFs- TD Added Functions:

- Time Functions- First, Last, Rank

- Sessionize JDBC Connectivity:- Custom Java Apps- Standards-based- BI Tool Integration

Push SCV Results:

- Exact Target import- Salesforce Objects- Custom reports

Interactive SQLTreasure Query Accelerator

Scheduled Jobs- SQL, Pig Scripts

- Data PushesREST API

Page 11: Tquila Single Customer View - How Big Data can fuel your Marketing strategy

©Tquila | Client Name | Date

What is Heroku?

Any languageApart from the 6 native languages can run any

language (PHP, GO, R,Erlang ..)

Extend with Addons

From databases to in-memory stores or

image/video encoding

Git based deployUse GIT as a way to send

code on Heroku and trigger deployment

Create StackBased on your code, Heroku

creates the appropriate Stack

Manage dependencies

Heroku recognize the libraries dependencies and

install them

Scale and auto scale

Heroku Apps can be allocated more resources

on the fly, by API or automatically

Multi-regionSelect US or EU to run app

closer to your users

High availabilityApps are monitored,

restarted or redeployed to an appropriate cluster in

case of failure

Monitor

Easy, free

export to

cloud or local system

s

Addons such as NewRelic or Papertrail

allows application

logging, monitoring

and

Flat monthly rate || Managed service || Fast setup || Secure || Live Support

Java,Ruby,

Python,Node.js,Clojure,Scala

Social Networks,

E-commerce,Portals,

Company websites

And many more…

BUILD DEPLOY

RUN APP

Page 12: Tquila Single Customer View - How Big Data can fuel your Marketing strategy

©Tquila | Client Name | Date

What is Treasure Data?

Treasure AgentLightweight data stream processor for real-time

upload

Bulk ImportFrom databases,

business applications and other systems via fast, parallel upload

Data StorageScalable, adaptable, extensible, columnar

storage system

Treasure Query Accelerator

Performance processing for ad hoc analysis of big data

Management Console

Monitor jobs, resources, access control, and support

tickets

Treasure Viewer Drag and drop analysis of

your data in real-time

Direct QueryUse familiar SQL syntax to

join, aggregate, and analyze data

BI Tool ConnectivityVisualize with your

existing tools, such as Tableau, Metric Insights,

Excel® and others

EXPORT

Easy, free

export to

cloud or local system

s

Free, simple export to

cloud or local systems:

RDBMSExcel®AmazonGoogle

Flat monthly rate || Managed service || Fast setup || Secure || Live Support

Web logsApp logsMobileSensor

RDBMSExcel®

CSVERPCRM

And many more…

ACQUIRE

STORE

ANALYZE

DATA

Page 13: Tquila Single Customer View - How Big Data can fuel your Marketing strategy

©Tquila | Client Name | Date

QUESTIONS ?

Page 14: Tquila Single Customer View - How Big Data can fuel your Marketing strategy

©Tquila | Client Name | Date

thankyou Vincent Spehner

07 46 29 120 45

[email protected]

@vzmind