wisenet performing highly accurate predictions through ... › ~alanwags › dlai2016 ›...
TRANSCRIPT
![Page 1: WiseNet Performing Highly Accurate Predictions Through ... › ~alanwags › DLAI2016 › (Zaratiegui+) IJCAI-16 … · Summary Novel method to encode customer behavior into images](https://reader036.vdocument.in/reader036/viewer/2022081406/5f106bde7e708231d449070f/html5/thumbnails/1.jpg)
WiseNetPerforming Highly Accurate Predictions
Through Convolutional Networks for Actual Telecommunication Challenges
Jaime Zaratiegui, Ana Montoro & Federico CastanedoData Science Team
IJCAI 2016 Workshop on
Deep Learning for Artificial Intelligence (DLAI)
●..
![Page 2: WiseNet Performing Highly Accurate Predictions Through ... › ~alanwags › DLAI2016 › (Zaratiegui+) IJCAI-16 … · Summary Novel method to encode customer behavior into images](https://reader036.vdocument.in/reader036/viewer/2022081406/5f106bde7e708231d449070f/html5/thumbnails/2.jpg)
Outline
1. Introduction2. Data Sets3. Data Sources & Representation4. Data Normalization5. Network Architecture6. Experimental Results7. Generalization & Feature Model Results8. t-SNE dimensionality reduction9. Summary
![Page 3: WiseNet Performing Highly Accurate Predictions Through ... › ~alanwags › DLAI2016 › (Zaratiegui+) IJCAI-16 … · Summary Novel method to encode customer behavior into images](https://reader036.vdocument.in/reader036/viewer/2022081406/5f106bde7e708231d449070f/html5/thumbnails/3.jpg)
IntroductionAt Wise Athena we develop predictive models for vertical Industries: Telecom & Consumer Packaged Goods (CPGs).
We present the application of ConvNets trained with GPUs on the non-trivial problem of predicting customer churn in prepaid telecom.In this business scenario, we define churn as an inactvity periodfor a balance replenishment event (i.e. 28 days).
This is a long standing problem that has been traditionally approachedby training ML classifiers with hand-crafted features (human expensive).
![Page 4: WiseNet Performing Highly Accurate Predictions Through ... › ~alanwags › DLAI2016 › (Zaratiegui+) IJCAI-16 … · Summary Novel method to encode customer behavior into images](https://reader036.vdocument.in/reader036/viewer/2022081406/5f106bde7e708231d449070f/html5/thumbnails/4.jpg)
Data SetsOur model learns from structured data commonly found in the telecom industry. From real data (1.2M users) with 22% of base churn ratio we generated the following sets:
● 130k users (supersampled with different time offsets)● 102k train● 18k validation● 37k test
In order to validate the generalization performance of WiseNet, we have prepared a second test data drawn from a different country.
![Page 5: WiseNet Performing Highly Accurate Predictions Through ... › ~alanwags › DLAI2016 › (Zaratiegui+) IJCAI-16 … · Summary Novel method to encode customer behavior into images](https://reader036.vdocument.in/reader036/viewer/2022081406/5f106bde7e708231d449070f/html5/thumbnails/5.jpg)
Data Sources & Representation
● Mobile Originating Calls MOC● Mobile Terminating Calls MTC● Topups TU
The x-dimension of each pixel corresponds to a 2-hour period
Start of the week mark
![Page 6: WiseNet Performing Highly Accurate Predictions Through ... › ~alanwags › DLAI2016 › (Zaratiegui+) IJCAI-16 … · Summary Novel method to encode customer behavior into images](https://reader036.vdocument.in/reader036/viewer/2022081406/5f106bde7e708231d449070f/html5/thumbnails/6.jpg)
Data NormalizationMOC MTC
Power law:Intensity = (Fill fraction)^(1/7)
As most variance is in the low fill fraction. Intensity saturates at a certain cutoff.
Topups
![Page 7: WiseNet Performing Highly Accurate Predictions Through ... › ~alanwags › DLAI2016 › (Zaratiegui+) IJCAI-16 … · Summary Novel method to encode customer behavior into images](https://reader036.vdocument.in/reader036/viewer/2022081406/5f106bde7e708231d449070f/html5/thumbnails/7.jpg)
![Page 8: WiseNet Performing Highly Accurate Predictions Through ... › ~alanwags › DLAI2016 › (Zaratiegui+) IJCAI-16 … · Summary Novel method to encode customer behavior into images](https://reader036.vdocument.in/reader036/viewer/2022081406/5f106bde7e708231d449070f/html5/thumbnails/8.jpg)
Network Architecture
Two convolutional layers and three dense. Trained with GPUs.
![Page 9: WiseNet Performing Highly Accurate Predictions Through ... › ~alanwags › DLAI2016 › (Zaratiegui+) IJCAI-16 … · Summary Novel method to encode customer behavior into images](https://reader036.vdocument.in/reader036/viewer/2022081406/5f106bde7e708231d449070f/html5/thumbnails/9.jpg)
Experimental ResultsWiseNet outperforms all other ML algorithms studied.
AUC Log-loss TP5 Brier
WiseNet 0.8787 0.4274 0.8929 0.1383
xgboost 0.8561 0.4722 0.8908 0.1594
GBM 0.8512 0.4995 0.8750 0.1662
GLM 0.8228 0.6782 0.7592 0.2433
randomForest 0.8169 1.2482 0.8636 0.2018
![Page 10: WiseNet Performing Highly Accurate Predictions Through ... › ~alanwags › DLAI2016 › (Zaratiegui+) IJCAI-16 … · Summary Novel method to encode customer behavior into images](https://reader036.vdocument.in/reader036/viewer/2022081406/5f106bde7e708231d449070f/html5/thumbnails/10.jpg)
Experimental Results/comparison
WiseNet xgboost
![Page 11: WiseNet Performing Highly Accurate Predictions Through ... › ~alanwags › DLAI2016 › (Zaratiegui+) IJCAI-16 … · Summary Novel method to encode customer behavior into images](https://reader036.vdocument.in/reader036/viewer/2022081406/5f106bde7e708231d449070f/html5/thumbnails/11.jpg)
Generalization & Feature Model Results
WiseNet AUC Log-loss TP5 Brier
Market-1 0.8787 0.4274 0.8929 0.1383
Market-2 0.8788 0.4449 0.9163 0.1428
AUC Log-loss TP5 Brier
WiseNet 0.8787 0.4274 0.8929 0.1383
Feature Model 0.8552 0.4602 0.7184 0.1528
Generalization
Feature Model comparison
![Page 12: WiseNet Performing Highly Accurate Predictions Through ... › ~alanwags › DLAI2016 › (Zaratiegui+) IJCAI-16 … · Summary Novel method to encode customer behavior into images](https://reader036.vdocument.in/reader036/viewer/2022081406/5f106bde7e708231d449070f/html5/thumbnails/12.jpg)
t-SNE Dimensionality ReductionUsing the states of the 1024-neuron dense layer we have performed at-SNE over a random selection of 26k users.
![Page 13: WiseNet Performing Highly Accurate Predictions Through ... › ~alanwags › DLAI2016 › (Zaratiegui+) IJCAI-16 … · Summary Novel method to encode customer behavior into images](https://reader036.vdocument.in/reader036/viewer/2022081406/5f106bde7e708231d449070f/html5/thumbnails/13.jpg)
Summary● Novel method to encode customer behavior into images that allows
using ConvNets on structured data.
● An experimental evalution with differet ML models, showing the capability of WiseNet to learn features.
● A comparison with a production model developed using hand-crafted features, showing the advantage of WiseNet.
● State the generalization and transfer learning property by applying the same model on a different market (without retraining).