analyzing complex behavior graphs in hadoop at scale
DESCRIPTION
Employing traditional approaches to analyzing customer behavior graphs and event sequences requires data simplifications, generalizations, and segmentations that severely degrade prediction accuracy and can lead to the loss of valuable information. But there’s an alternative approach that retains the fidelity of customer profile data, the full sequence of events data, and enables powerful business intelligence and predictive analytics. In this webcast, join Apigee’s Joy Thomas, Chief Scientist, and Sanjeev Srivastav, VP Data Strategy, as they explore the superiority of new methods of behavior graph analysis over the simplifications required for traditional data storage and classical predictive algorithms. Join to learn: • How shortcomings of traditional approaches make it difficult to build and analyze complex behavior graphs • Why GRASP—graph and sequence processing technology on Hadoop presents a different and more effective approach to behavior graph analysis • How descriptive analytics using GRASP uncover hidden patterns in historical customer journey data • How Predictive analytics that use behavior graphs and Bayesian algorithms, along with machine learning, ensure model performance over time Download Video: http://youtu.be/CwxvlgW9aZY Download Podcast: https://soundcloud.com/apigee/analyzing-complex-behavior-graphs-in-hadoop-at-scaleTRANSCRIPT
Analyzing Complex Behavior Graphs in Hadoop at Scale
Today’s speakers
Sanjeev Srivastav@ssrivastav
Joy Thomas@JoyAThomas1
© 2014 Apigee – For Public display
Apigee social channels
3
YouTubehttp://youtube.com/apigee
Slideshare
http://slideshare.com/apigee
© 2014 Apigee – For Public display
Agenda
• Customer behavior graphs
• GRASP: an event model for customer interactions
• GRASP for– the customer journey– predictive modeling
Customer behavior graphs
© 2014 Apigee – For Public display
Why do we need behavior graphs?
© 2014 Apigee – For Public display
Customer view: a journey
7
© 2014 Apigee – For Public display 8
Understand each customer’s journey
customer journeysiloed view
© 2014 Apigee – For Public display 9
Identify common interactions and influences
common interactions & influencescustomer journey
© 2014 Apigee – For Public display
Customer behavior
10
Behavior graph• sequence of events:
– actions experienced and taken
Social graph• links between people & activities
– at a particular point in time
Behavior graph
Social graph
© 2014 Apigee – For Public display
Why do we need new technology for behavior graphs?
© 2014 Apigee – For Public display 12
Challenges with current technologies
• SQL is not efficient for sequences of events and quick counting of customer journeys
• Custom algorithms are expensive
• Various graph systems are oriented toward social graphs and computations of neighborhoods, “friend of a friend,” etc.
Event model for customer interactions: GRASP
© 2014 Apigee – For Public display© 2014 Apigee Confidential – All Rights Reserved 14
• Graph and sequence processing
- time-sequenced graph analytics on Hadoop
GRASP
© 2014 Apigee – For Public display
Model for user behavior
Users act on nodes in a temporal sequence of events
1
3 52
2 50
0
USER PROFILEUserID: U56Gender: MGeo: San FranciscoInterests: bikes, fashion
USER PROFILEUserID: U57Gender: FInterests: news, financeAge: 35-40
NODE PROFILEType: ContentPageID: P100Category: product reviewSubCat: mountain bike
NODE PROFILEType: CreativeID: Creative95Category: VideoAdAdvertiser: BikePros
EVENTType: PageViewUserID: U56PageID: P100TimeSpent: 180 sec. Scrolls: 3
EVENTType: AdViewUserID: U56AdID: Creative95PlayTime: 30 sec.Rewinds: 1
© 2014 Apigee – For Public display
Aggregated behavior graph
0
1
2
3
5
0
1
3 52
2 5Impressions: 1TimeSpent: 20Clicks: 1
0
0Impressions: 4TimeSpent: 10Clicks: 0
Impressions: 5TimeSpent: 30Clicks: 1
Combine
© 2014 Apigee – For Public display
Event streams
17
Event stream
Combined event streamPurchase
health insurance
Offer for Health
Insurance
GRASP merges event streams and normalizes time relative to responses
Store visits
Emails
Phone calls
Purchases
F, 35, Married
M,25, Married
© 2014 Apigee – For Public display
Data representation • Data model: events & dimensions• Data structure: aggregated
behavior • Graphs for events, not tables• Data access: API
GRASP: graphs vs. tables
18
Event facts are represented as graphs, not tables
SQL is not effective for sequence queries
1
2
3
40
Userdimension
Nodedimension
Events
Data storage & computation• Distributed data structure on
Hadoop• Computation using map reduce
Dual use of GRASP: customer journey & predictive models
© 2014 Apigee – For Public display
How has this been used for solving customer engagement problems?
© 2014 Apigee – For Public display© 2013 Apigee Confidential – All Rights Reserved 21
Dual use of GRASP for analytics
GRASP
GQM
Data
Profiles
Text
Events
Feature extraction
Events
Data scientist
Adaptive applications
Developer
Custom app
Business user
Segments manager, GQM
DeveloperDeveloper
Predictive
Descriptive
A
Y
X
C D
B
Predictive & descriptive analyticsGraph and sequence processing (GRASP)
© 2014 Apigee – For Public display
Visualize customer journeys across all interactions
• Examples: telco, healthcare, retail
© 2014 Apigee – For Public display© 2013 Apigee Confidential – All Rights Reserved
• Machine learning using path sequences
Build behavior prediction models
© 2014 Apigee – For Public display
Relevant problems to solve
24
• Multi-channel customer event data for understanding customer journeys and predicting behavior
• Modeling evolving customer behavior using updates to behavior graphs
Questions?
Sanjeev Srivastav@ssrivastav
Joy Thomas@JoyAThomas1
© 2014 Apigee – For Public display
Thank you