tquila single customer view - how big data can fuel your marketing strategy
DESCRIPTION
TRANSCRIPT
Single Customer View (SCV)
Presented by
Vincent Spehner
20th Mars 2014
V0.1
A Hadoop based solution
©Tquila | Client Name | Date
Problem Statement
Our solution
Technical Architecture
Heroku and Treasure Data
Q&A and Next Steps
Agenda
©Tquila | Client Name | Date
Problem StatementWhy you need a SCV ?
Customer
Opportunities
Google Analytics
ERPWebsite data
Marketing Campaign
Social Media
• Aggregated view of customer data• Improve marketing ROI• Increase cross sell, Upsell
It’s all about those millions, if not billions, of records:• Spread on multiple places• Not connected• Sleeping and unvalued
©Tquila | Client Name | Date
Problem Statement
Building a Big Data infrastructure is complicated, expensive, and requires skills that are very hard to hire.
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
O/SHardw
are
Networking
Storage Layer
Frameworks
Processing Engines
Application Interfaces
Streaming Collection
Batch Collection
Business Rules
Data Integration
Consumption / Analytics
Big Data is complex
©Tquila | Client Name | Date
Problem Statement Lots of companies today produce Big
DataBut few have the resources to value them through a Single Customer View approach
60-70% of a company’s Big Data time & budget consumed by:
• Infrastructure setup & Maintenance• Building Data Collection & Storage
Flows• Hiring/Training Hadoop Expertise
It typically takes at least 6 months to get a Hadoop environment into production
Big Data takes time
©Tquila | Client Name | Date
Our solution
A Cloud-based solution accelerates and reduces your cost of implementing a Single Customer View.
Use this new customer insights straight into Salesforce Marketing tools (Exact Target, Pardot) and share new report
Cloud based SCV
• Salesforce based solution
• Hadoop on demand
All Your Data
Aggregate / Store / Analyze
Data preprocessing
Worker Worker Worker Worker Worker
Pig Transformation
Hive queries
Segmented
customers export (.json, .csv,
API)ExactTarget
Transactional DB
©Tquila | Client Name | Date
Technical ArchitectureData Acquisition
# Connect to APIs......
# Post events to Treasure DataTD.event.post(’login’,{:uid=>123,:pg=>13})
...
...TD.event.post('access’,{:uid=>123})
Treasure Data LibraryJava, Ruby, PHP, Perl, Python, Scala, Node.js
Heroku Workers
Reliable & Secure
(SSL)
Compressed
Upload
Automatic Micro-batching
Local buffering Fall-back
Network Tolerance
Treasure Agent
{ format:
JSON }
Technical ArchitectureData Storage
Stored “schema-less” as JSON Schema can be applied/updated
AFTER storage
Compressed & columnar format For higher query performance
Optimized for time-based filtering Quickly scale-up processing power
WITHOUT reloading/redistributing the data
time v
1384160400 {“ip”:”135.52.211.23”, “code”:”0”}
1384162200 {“ip”:”45.25.38.156”, “code”:”-1”, “action”:”upload}
1384164000 {“ip”:”97.12.76.55”, “code”:”99”,”status”:”ok”}
time ip : string code : int action : string
1384160400 135.52.211.23 0 NULL
1384162200 45.25.38.156 -1 upload
1384164000 97.12.76.55 99 NULL
Default (schema-less)
Schema applied
SELECT v[‘ip’] as ip, v[‘code’] as code …
~30% Faster
SELECT ip, code …
Technical ArchitectureData Analysis
Scripted Processing (Pig):- DataFu (LinkedIn)
- Piggybank (Apache)
Heavy Lifting SQL (Hive):- Hive’s Built-in UDFs- TD Added Functions:
- Time Functions- First, Last, Rank
- Sessionize JDBC Connectivity:- Custom Java Apps- Standards-based- BI Tool Integration
Push SCV Results:
- Exact Target import- Salesforce Objects- Custom reports
Interactive SQLTreasure Query Accelerator
Scheduled Jobs- SQL, Pig Scripts
- Data PushesREST API
©Tquila | Client Name | Date
What is Heroku?
Any languageApart from the 6 native languages can run any
language (PHP, GO, R,Erlang ..)
Extend with Addons
From databases to in-memory stores or
image/video encoding
Git based deployUse GIT as a way to send
code on Heroku and trigger deployment
Create StackBased on your code, Heroku
creates the appropriate Stack
Manage dependencies
Heroku recognize the libraries dependencies and
install them
Scale and auto scale
Heroku Apps can be allocated more resources
on the fly, by API or automatically
Multi-regionSelect US or EU to run app
closer to your users
High availabilityApps are monitored,
restarted or redeployed to an appropriate cluster in
case of failure
Monitor
Easy, free
export to
cloud or local system
s
Addons such as NewRelic or Papertrail
allows application
logging, monitoring
and
Flat monthly rate || Managed service || Fast setup || Secure || Live Support
Java,Ruby,
Python,Node.js,Clojure,Scala
Social Networks,
E-commerce,Portals,
Company websites
And many more…
BUILD DEPLOY
RUN APP
©Tquila | Client Name | Date
What is Treasure Data?
Treasure AgentLightweight data stream processor for real-time
upload
Bulk ImportFrom databases,
business applications and other systems via fast, parallel upload
Data StorageScalable, adaptable, extensible, columnar
storage system
Treasure Query Accelerator
Performance processing for ad hoc analysis of big data
Management Console
Monitor jobs, resources, access control, and support
tickets
Treasure Viewer Drag and drop analysis of
your data in real-time
Direct QueryUse familiar SQL syntax to
join, aggregate, and analyze data
BI Tool ConnectivityVisualize with your
existing tools, such as Tableau, Metric Insights,
Excel® and others
EXPORT
Easy, free
export to
cloud or local system
s
Free, simple export to
cloud or local systems:
RDBMSExcel®AmazonGoogle
Flat monthly rate || Managed service || Fast setup || Secure || Live Support
Web logsApp logsMobileSensor
RDBMSExcel®
CSVERPCRM
And many more…
ACQUIRE
STORE
ANALYZE
DATA
©Tquila | Client Name | Date
QUESTIONS ?
©Tquila | Client Name | Date
thankyou Vincent Spehner
07 46 29 120 45
@vzmind