early churn prediction and personalised interventions in top eleven game
TRANSCRIPT
Early Churn Prediction and Personalised Interventions in
TOP 11
Miloš Milošević, Nordeus
About me
Data engineer at Nordeus.
Main focus: Predictive machine learning pipelines; Their deployment and maintenance.
About Nordeus
● Award winning gaming company
● Offices in London, Dublin, San Francisco, Skopje and Belgrade
● Creators of Top Eleven
About TopEleven
● Our flagship product
● 110+ million registered users
● Global
● 5 years old
Agenda
IDENTIFY- Predict early churners using machine
learning
TARGET- Construct a personalized message for every
churner
INTERVENE- Send messages via a scalable notification
system
Early churn
● What is it?
● Why is its reduction important?
● Can we predict it and how?
When to engage the prediction algorithm?
We tried 3 things
Features
Models● Best model -
Gradient Boosting Trees
● Model we use - Logistic Regression
● Model to play with - Recurrent Neural Network
And what do we do then?
● Optimize backend
● Target churners
● Stay prediction
Targeting
Messages need to be meaningful, otherwise...
What kind of notifications work well on what type of user?
How to find the perfect match?
Use the user’s first day activity to create meaningful personalized notifications.
Targeting
User segmentation
Examples
Measuring results
Split users into 3 groups:
● Control
● Baseline
● Test
Our results
● On average, baseline group achieved 30% better retention
● Test group achieved 40% better retention than baseline
Notification engine
Specs:● Modular design● Scalable● On time delivery● Near real time message creation
● Logging system● A/B test support
The power of Spark
Conclusion
● Everyone can do early churn prediction
● Target with care
● When in doubt, Apache Spark
