page
VOLTDB AND FLYTXT PRESENT: BUILDING A SINGLE TECHNOLOGY PLATFORM FOR REAL-TIME AND ITERATIVE ANALYTICS ON FAST + BIG DATA
page© 2015 VoltDB PROPRIETARY
VOLTDB OVERVIEW
Mike Stonebraker
FASTWorld Record Cloud Benchmark:
YCSB (Yahoo Cloud Serving Benchmark) - 2.4 million tps (transactions per second)
Other Stonebraker Companies
Customers
3
Technology• In-Memory (but data is durable to disk)• Scale-Out shared-nothing architecture• Reliability and fault tolerance
• SQL + Java with ACID• Hadoop and data warehouse integration• Open source and commercially licensed (24X7)
Founded by winner of the 2014 ACM Turing Award
page© 2015 VoltDB PROPRIETARY
Collect Explore
AnalyzeAct
4
Big Data analytic results:
1. Discoveries: seasonal predictions, scientific results, long-term capacity planning
2. Op.miza.ons: market segmentation, fraud heuristics, optimal customer journey
page© 2015 VoltDB PROPRIETARY
FAST DATA – BIG DATA
5
Fast Data Pipeline
Ingest Export
Big Data
Real-Time Analytics & Decisions
Fast Data: the velocity side of Big Data
Milliseconds
page© 2015 VoltDB PROPRIETARY
DATA ARCHITECTURE FOR FAST + BIG DATA
Enterprise Apps
ETL
CRM ERP Etc.
Data Lake (HDFS, etc.)
BIG DATA SQL on Hadoop
Map Reduce
Exploratory Analytics
BI Reporting
Fast Operational Database
FAST DATA
Export Ingest / Interactive
Real-time Analytics
Fast Serve Analytics
Decisioning
6
page© 2015 VoltDB PROPRIETARY
7
“89% of marketers surveyed plan to
compete primarily on the basis of customer experience by 2016.”
Source: Gartner 2014 survey, Companies > $50M in revenue
page© 2015 VoltDB PROPRIETARY
FAST DATA SOURCES AND DRIVERS
Mobile
IoT
Social
Sensors
Logs
Data is doubling every two years
• 26 billion connected devices by 2020 (Gartner 2014)
• 37% of most data will be processed at the edge in milliseconds (Cisco IoT Study 12/11/14)
Mobile
IoT
8
page© 2015 VoltDB PROPRIETARY
THE FAST DATA PIPELINE
9
Calculations Serving of Results
Real Time, Per Event, Interactive
page© 2015 VoltDB PROPRIETARY
STREAMING: REAL TIME ANALYTICS
• Operational analytics and monitoring
• RT analytics enabling user-facing applications
• KPI for internal BI/Dashboards
10
page© 2015 VoltDB PROPRIETARY
STREAMING OPERATORS NEED STATE
Require State
• Filter
• Join
• Aggregate
• Group By
Stateless
• Partition
11
page© 2015 VoltDB PROPRIETARY
REAL-TIME ANALYTICS
12
Database
Metadata (Dimension table)
Session state (Fact table) • Operational analytics and
monitoring
• RT analytics enabling user-facing applications
• KPI for internal BI/Dashboards
• In-memory MPP SQL over ODBC/JDBC
• Cheap + correct materialized views for streaming aggregations
SQL, Views
Ingest
page© 2015 VoltDB PROPRIETARY
INTEGRATING WITH EXPORT TARGETS
13
• Local file system export• JDBC export• Kafka export• RabbitMQ export• HDFS export• HTTP export• Extensible API
page© 2015 VoltDB PROPRIETARY
DATA PIPELINES WITH EXPORT
15
Database
Metadata (Dimension table)
Session state (Fact table)
• Filtering (ex: only RFID / iBeacon readings that show change from previous location).
• Sessionization
• Common version re-writing
• Data enrichment
• MPP streaming Export
• Row data, Thrift messages, CSV
• OLAP, HDFS and message queues
Export
page© 2015 VoltDB PROPRIETARY
FLYTXT OVERVIEW
17
} Vision: Create >10% measurable economic value for Communica8on Services Providers through Big Data Analy8cs
} Flytxt’s internal and external mone8za8on solu8ons increase revenue, reduce churn and improve customer experience
} Dutch company with corporate office in Dubai, global delivery centres in India and regional presence in Mexico City, Johannesburg, Singapore, Dhaka and Nairobi.
Partners Operators Customers and Partners
Brands
Sample text
Awards & Achievements Vision, Mission & Impact
page© 2015 VoltDB PROPRIETARY
FLYTXT’S INTEGRATED ANALYTICS SOLUTION ARCHITECTURE: BIG DATA, ITERATIVE AND REAL-TIME ANALYTICS
Files and BI outputs
Event Filter Trigger Detector
Real 2me Trigger Engine (RTE)
Processed Events
Con2nuous Insight Engine (CIE)
Data Fusion Engine
Batch Analy2cs
Triggered Rules
Scheduled Rules
Rendering Engine
Persistence Store
KPIs, Insights, Recommenda2ons, Ac2ons & Rules
Hadoop 2
Hadoop 2, Hbase Jboss, Apache, SMPP sim, Tomcat, Ext JS, Hornet Q
Hadoop 2, Spark, Pig, Hive, Mahout, COIN-‐OR, Weka, MLlib
VoltDB, Drools
Response/ Input
Capture Network / BSS / OSS
GUI
N/W Integra2on
Network / BSS / OSS
Chan
nels
Ope
rator
Subscribers
Triggers
Lookup Subscriber Insights
Scan Subscriber Insights
Itera2ve Analy2cs
2
Streaming Data
page© 2015 VoltDB PROPRIETARY
BIG DATA ANALYTICS USE CASE: BEST FIT PRODUCT RECOMMENDATION
19
Profile, Usage
Data Fusion Engine
Batch Analy2cs
Persistence Store
Hadoop 2
Con2nuous Insight Engine (CIE)
Best Fit Product Recommenda2on
Usage Product U8lity
Business Constraints Model
Ranking
Context 1
Context 2
Context 3
Recommended Offers P1 P2 P3
P3 P2 P1
P5 P4 P7
Ranking based on Similari8es
Objec8ve: Recommend best fit product to subscribers based on usage and business objec8ves
Recommended Offers Recommended Offers
Hadoop 2, Hbase
page© 2015 VoltDB PROPRIETARY
0
5
10
15
20
Offer-‐1 Offer-‐2 Offer-‐3 Offer-‐4 Offer-‐5
Conversion
Rate in Ju
l-‐2014
in %)
0
5
10
15
20
25
Conversion
Rate in
Jul-‐2
014 (in
%)
Circles Rule-‐based campaign Best-‐fit recommenda8on (fair)
CASE STUDY: CONTEXTUAL PRODUCT RECOMMENDATION FOR TIER 1 OPERATOR
Recommenda2on Personas: CLV (HVC, MVC, LVC), Vola8le, Early Adopter, Frequent Handset Changer, Heavy Data user, Social Media Fan, Bollywood Fan, Music Fan, Sports Fan, poten8al iPad buyer, Interna8onal Caller Etc……. Objec2ves: Cross sell, Upsell, S8mulate recharge/usage/Service adop8on Etc… Offers: Data Plan, 3G plan, VAS usage, Interna8onal Calling packs, Bundle offers, Recharge s8mula8on, Seeding, ebill subscrip8on etc…… Channels: IVR, In store, Retailer, WAP portal, Customer care portal
page© 2015 VoltDB PROPRIETARY
ITERATIVE ANALYTICS USE CASE: MICRO-SEGMENTATION
21
Files and BI outputs
Data Fusion Engine
Itera2ve Analy2cs
Persistence Store
Spark, MLlib
Con2nuous Insight Engine (CIE)
Micro-‐segmented Offers
Gaussian Mixture Model
Segment Offers
Segment Offers Segment Offers
Soi Clustering
S1 P3
S2 P8
S4 P9
Clustering to enable micro segmentation for
personalized offers
Hard Clustering
Soft Clustering
Hadoop 2, Hbase
page© 2015 VoltDB PROPRIETARY 22
USE CASE: REAL-TIME ANALYTICS SUPPLEMENTED BY BIG DATA, ITERATIVE ANALYTICS
Files and BI outputs
Event Filter Trigger Detector
Processed Events
Con2nuous Insight Engine (CIE)
Data Fusion Engine
Analy2cs Engine
Triggered Rules
Scheduled Rules
KPIs, Insights, Recommenda2ons, Ac2ons & Rules
Hadoop 2
Hadoop 2, Hbase
Hadoop 2, Spark, Pig, Hive, Mahout, COIN-‐OR, Weka, MLlib
VoltDB, Drools
Itera2ve Analy2cs
Persistence Store
Big Data+ Itera2ve analy2cs
Real-‐2me analy2cs
Real-‐8mes Ac8ons
Balance Threshold
Recharge
Usage
Network
Behaviou
r
Preferen
ce
Recommen
da8o
n
Contextual Needs
Streaming Data
page© 2015 VoltDB PROPRIETARY
CASE STUDY: REAL-TIME TRIGGER BASED MICRO-SEGMENTED OFFERS
23
Drop in overall ARPU
Drop in data usage AR
PU Slabs
VAS
SMS
Voice usage
DATA
Outgoing
Incoming Long term inac2vity
Short term inac2vity
Product Affinity
Local
Long distance
Drop in O/G MoU
Drop in balance
Leg-‐wise Usage
Segments
Usage Behaviour
Client Objec2ve • Improve customer
engagement for ARPU enhancement
Solu2on
• Marke8ng Program based on usage behaviour driven micro-‐segmenta8on and tripwire monitoring
Data Analyzed
• Customer usage history, ARPU charts, spend palerns and preferences
Impact • 2% increase in month-‐on-‐
month revenue • 28% higher revenues &
MOU
page© 2015 VoltDB PROPRIETARY
QUESTIONS?
• Use the chat window to type in your questions or hashtag #VoltDBFlytxt
• Know more about Flytxt• Visit www.Flytxt.com
• Try VoltDB yourself:Ø Free trial of the Enterprise Edition:
• www.voltdb.com/download
Ø Try VoltDB in the CloudØ Amazon’s Cloud Formation
Ø Open source version is available on github.com
24