predictive conversion modeling - amazon s3 · outputs from predictive conversion modeling •...
TRANSCRIPT
Predictive Conversion ModelingLifting Web Analytics to the next levelSuperweek, Hungary | February 1, 2017
@mertanen & @ronluhtanen
Regional
“Centres of
Excellence”
alongside agency
embedded
experts
1200employees
globallyNA Offices
MENA Offices
EMEA Offices
APAC Offices
It’s our people that make the difference
Our approach to data-driven marketing –a global focus
Using machine learning for explaining and predicting user behavior from web analytics data.
3
Universal Analytics made the platform more open.
Tag Management was a game changer!
5
Insights based on aggregated data.
6
Advanced methods limited to differentregression models.
7
Traditional Web Analytics is dying...
• Time consuming
• Not cost effective
• Human brains are not able to work with
large amount of complex data
• Outputs depends too much on the analyst
• Insights are too simple
• Predicting in a very rough level
What are the problems with traditional Web Analytics?
9
Web Analytics Data Science
Easy & fast implementation for the Modeling.
• Tag website features and elements like
never before, more is more in this case!
• Collect session ID
• Save browser ID
• Think about User ID
• Adform cookie ID or similar
Setup for the Modeling12
This means we see every interaction that each user has during each visit.
The granularity of the data greatly increases
the possible model selection as well as the accuracy of the
models.
Extreme Gradient Boosting
• The bulk of the modeling is done by
Extreme Gradient Boosting
• The method is a decision tree based
algorithm
• Gradient Boosting can handle
regresssion as well as multiclass
classification
• We have great flexibility with selecting
the KPIs that we want to model and
predict, without having to change the
core modeling algorithm
About the modeling15
odor=noneCover: 1628,25Gain: 4000,53
spore-print-color = greenCover: 703,75Gain: 198,174
< -9,53674e-07
< -9,53674e-07
< -9,53674e-07
stalk-root=clubCover: 924,5Gain: 1158,21
LeafCover: 13,25Gain: 1,85965
LeafCover: 690,5
Gain: -1,94071
LeafCover: 112,5
Gain: -1,70044
LeafCover: 812
Gain: 1,7128
LeafCover: 309,453Gain: -0,96853
< -9,53674e-07
< -9,53674e-07
LeafCover: 458,937Gain: 0,784718
LeafCover: 20,4624Gain: -6,23624
odor=noneCover: 768,39Gain: 569,725
stalk-root=rootedCover: 788,852Gain: 832,545
• Incredibly accurate, hard to overfit and
very fast
• Ability to extract complicated non-linear
relationships from very varied data
• The Algorithm uses only the relevant data
from all the data that is available to it
• Huge improvement over some other
regression models that break if they are
fed with irrelevant data
About the modeling16
https://github.com/dmlc/xgboost
Outputs from the Predictive Conversion Modeling
17
Outputs from Predictive Conversion Modeling
• Generally the output of the analysis is a predictive model that gives a predictions
for the measurement we are modeling against.
• The predictions can be used by themselves or further analysis can be done on
the model to further explain the dependencies in the user interactions.
• The model will be available for digital marketers and analysts.
• Following are 4 example uses for the modeling.
18
Data-to-output in Predictive Conversion Model application
Input Output
Enhanced Web Analytics data
Profiling by clustering
customers based on on-site
behavior
Retargeting based on
predicted responses
Twinning to expand reach
to the most prospective
customer profiles
Conversion optimization
19
Machine learning
based predictive
modelling
The predictions can be used in more effective retargeting. Instead of bombarding all
the past site visitors with advertisements we can target the advertisements based
on the specific interactions as well as the likelihood of having converted. For
instance we can create a rule that targets people who have over 20% probability of
purchase and have visited the promotion page of a specific product.
Output Application 1: Enhanced Retargeting
IF THENProbability of purchase>20%
Visited product page
Target advertisement to
specific people
Recipe
Trigger Action
20
The modeling process can also be used in acquiring valuable information on the
behavioral differences of the users. Uncovering certain dependencies in their interactions
allows the marketers to design (and later automate) their marketing messages
differently and more effectively to each of their visitor groups (segments).
Output Application 2: Clustering and Profiling
Person A
Person B
WEB BEHAVIOR
On-site behavior Off-site behavior
Likes gamblingsites
Buys clothesonline
Has visited bookingpage twice
Has visited promotionpage three times
Visits homepageregularly
Has read product descriptionpage for three minutes
Readsgardening blogs
Watches regularlymovie trailers online
21
The machine learning models can help in conversion optimization. We are not
restricted with just A/B testing, but instead we can create rules that change the site in
order to maximize the likelihood of purchase or conversion of each and every user.
By leveraging the trained model we can direct the user towards the
interactions that are most effective in increasing the likelihood of conversion.
Output Application 3: Conversion Optimization
WEBSITE CONTENT RULES
Activated rule Not actived rule
22
Once we have identified the most beneficial behavioral patterns, we can use the cookie
data of the most prospective visitors in order to build larger target groups out of
similar web users. The groups can then be used in programmatic buying of advertisements.
Output Application 4: Twinning
BUYING RULESfor different targetgroups
23
How to target marketing so that it maximizes users likelihood to convert?
25
• Finland’s largest shipyard – builds and
operates cruise ships
• Operates in a very competitive online
environment
• High maturity with online optimization and
data-driven marketing
• Large portion of sales through online
Case Tallink Silja26
9 mil. Passengers *
Annually
945 mil. Turnover *
• Very accurate predictions for non-
converting visitors
• Possibility to adjust prediction
treshold for different actions
The model27
ROC CurveAccuracy 98%• Sensitivity 99%
• Specificity 75%
• Previously possible only to create
custom segments
• Now clustering using
unsupervised machine learning
over 240 dimensions
• Four distinct behavioral groups• Heavy users
• Intermediate users
• Reactivated
• Just visiting
Clustering using on-site behavioral data28
Mean Conversion % - Indexed
1
9,5
8,48,8
Exploring differences time spent on site29
Mean Duration from past Session* - IndexedMean Session Duration - Indexed
10,5
10
27
*Calculated as a cumulative sum with 50% daily decay
1
21
1,62,4
Not limited to averages30
3
1
4,3
Session Duration – Just Visting
ConvertedNo convertion
1,5
0,28
ConvertedNo convertion
Session Duration – Heavy User
240
280
213
315
250
216
Proportional differences depending on source31
Proportion of visitors from DisplayProportion of visitors from Direct
*Calculated as a cumulative sum with 50% daily decay
32Feature Importance
• Partial dependencies• Change inputs
• Observe outputs
• Automate
• Can be applied to advertisement
messages, channels, or on-site
elements
• Possible to use smart optimization
algorithms to identify actions that
maximize conversion probability
Gaining insight from a complex model33
https://github.com/fmfn/BayesianOptimization
Discount campaign’s effect on mean propability for conversion
34
Heavy Users Intermediate Users Reactivated Just visiting
Future development with the Predictive Conversion Modeling
Machine learning
leveraged analytics
and real time
predictive
modelling
Input Output
Enchanced Web Analytics data
Basic Web Analytics data
Client’s Customer Data
Semantic data
ID’s from ad serving platforms
Organic clustering based on off-
and on-site data
Immediate onsite adaptations
based on off-site data
AI driven marketing:
test and modify content based
on predicted behaviours
Retarget to increase
conversion percentage
36
Future Development Streams
Automated optimization of online
advertising spending
• Spend less time on manual analysis
• No more headache with complex data and pressure for outputs
• Think more about the business questions
• The model will do the counting and give answers with a high confidelity level
• You will interpret results for the business and edit the model for more in-depth analysis
• You are able to enable analysts with tools previously available only to data scientists
• Shift the focus from simple metrics to the actual business objects
• Set up automatically optimizing feedback loops in order to continiously increase conversion rates
How this is changing our work?37
Executive Summary
• No more time consuming, labor heavy and
expensive manual analysis
• Enable analysts with machine learning
• Fast to implement and quick to show results
• Ask another question
• Continiously improve marketing efficiency
and ROI
• Get real competitive edge with analytics
Executive Summary39
Petri MertanenDirector, Digital Analytics
+358 400 792 616
Ron LuhtanenAnalyst, Data Science
+358 50 431 8166
Q&A
Petri MertanenDirector, Digital Analytics
+358 400 792 616
Annalect Finland is a part of Omnicom Media Group.
Ron LuhtanenAnalyst, Data Science
+358 50 431 8166
Annalect Finlandwww.annalect.fi
@annalect_fi