aws re:invent 2016: predicting customer churn with amazon machine learning (mac307)
TRANSCRIPT
![Page 1: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/1.jpg)
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Denis V. Batalov, PhD
AWS Solutions Architect, EMEA
November 30, 2016
Predicting Customer Churnwith Amazon Machine Learning
@dbatalov
MAC307
![Page 2: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/2.jpg)
Customer churn
![Page 3: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/3.jpg)
Machine learning
Science
• Computer Science
• Statistics
• Neuroscience
• Operations Research
Artificial Intelligence
• Rule extraction from data
• Inspired by human learning
• Adaptive algorithms
Engineering
• Training: Data Models
• Prediction: Models Forecast
• Decision: Forecast Actions
![Page 4: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/4.jpg)
ML: Robotics
![Page 5: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/5.jpg)
ML: Robotics
![Page 6: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/6.jpg)
ML: Image recognition
![Page 7: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/7.jpg)
Supervised learning
![Page 8: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/8.jpg)
Supervised learning
Input Outcome
![Page 9: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/9.jpg)
Supervised learning
Input Outcome
Input
Input
Input
Outcome
Outcome
Outcome
![Page 10: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/10.jpg)
Supervised Learning
Input Outcome
Input
Input
Input
Outcome
Outcome
Outcome
Supervised
Learning
known historical data
Amazon ML
![Page 11: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/11.jpg)
Supervised learning
Input Outcome
Input
Input
Input
Outcome
Outcome
Outcome
Supervised
Learning
Unseen Input Same Outcome
known historical data
Amazon ML
![Page 12: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/12.jpg)
Amazon Machine Learning service
![Page 13: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/13.jpg)
Amazon Machine Learning service
![Page 14: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/14.jpg)
Amazon Machine Learning service
![Page 15: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/15.jpg)
Amazon Machine Learning service
![Page 16: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/16.jpg)
Telco churn dataset
• US telco customers, their cell phone plans and usage
• 21 attributes, 3333 rows:
• Customer: State, Area_Code, Phone
• Plan: Intl_Plan, VMail_Plan
• Behavior: VMail_Messages, Day_Mins, Day_Calls,
Day_Charge, Eve_Mins, Eve_Calls, Eve_Charge,
Night_Mins, Night_Calls, Night_Charge, Intl_Mins,
Intl_Calls, Intl_Charge
• Other: Account_Length, CustServ_Calls, Churn
![Page 17: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/17.jpg)
Telco churn dataset
• US telco customers, their cell phone plans and usage
• 21 attributes, 3333 rows:
• Customer: State, Area_Code, Phone
• Plan: Intl_Plan, VMail_Plan
• Behavior: VMail_Messages, Day_Mins, Day_Calls,
Day_Charge, Eve_Mins, Eve_Calls, Eve_Charge,
Night_Mins, Night_Calls, Night_Charge, Intl_Mins,
Intl_Calls, Intl_Charge
• Other: Account_Length, CustServ_Calls, Churn
![Page 18: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/18.jpg)
Telco churn dataset
KS, 128, 415, 382-4657, 0, 1, 25, 265.100000, 110, 45.070000, 197.400000, 99, 16.780000, 244.700000, 91, 11.010000, 10.000000, 3, 2.700000, 1, 0
OH, 107, 415, 371-7191, 0, 1, 26, 161.600000, 123, 27.470000, 195.500000, 103, 16.620000, 254.400000, 103, 11.450000, 13.700000, 3, 3.700000, 1, 0
NJ, 137, 415, 358-1921, 0, 0, 0, 243.400000, 114, 41.380000, 121.200000, 110, 10.300000, 162.600000, 104, 7.320000, 12.200000, 5, 3.290000, 0, 0
OH, 84, 408, 375-9999, 1, 0, 0, 299.400000, 71, 50.900000, 61.900000, 88, 5.260000, 196.900000, 89, 8.860000, 6.600000, 7, 1.780000, 2, 0
OK, 75, 415, 330-6626, 1, 0, 0, 166.700000, 113, 28.340000, 148.300000, 122, 12.610000, 186.900000, 121, 8.410000, 10.100000, 3, 2.730000, 3, 0
AL, 118, 510, 391-8027, 1, 0, 0, 223.400000, 98, 37.980000, 220.600000, 101, 18.750000, 203.900000, 118, 9.180000, 6.300000, 6, 1.700000, 0, 0
![Page 19: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/19.jpg)
Console: Creating datasource for Amazon ML
![Page 20: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/20.jpg)
Console: Creating datasource for Amazon ML
![Page 21: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/21.jpg)
Console: Building the Amazon ML model
![Page 22: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/22.jpg)
Recipe
{ "groups": {
"NUMERIC_VARS_NORM": "group('Intl_Charge','Night_Calls','Day_Calls','Eve_Calls','Eve_Mins','Intl_Mins','VMail_Message','Intl_Calls','Day_Mins','Night_Mins','Day_Charge','Night_Charge','Eve_Charge','Account_Length')” },
"assignments": {},
"outputs": [
"ALL_BINARY",
"State",
"Area_Code",
"normalize(NUMERIC_VARS_NORM)",
"CustServ_Calls"
]
}
![Page 23: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/23.jpg)
Recipe: normalize() function
Account_Length Normalized Value
128 0.808771865
107 -0.047574816
137 1.175777586
84 -0.985478323
75 -1.352484044
118 0.400987732
![Page 24: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/24.jpg)
Building the Amazon ML model
![Page 25: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/25.jpg)
Cost of errors
• Cost of customer churn and acquisition (false negative):
• Foregone cash flow
• Advertising costs
• POS and sign-up admin costs
• Customer retention cost (false + true positive)
• Discounts
• Phone upgrades
• Etc.
![Page 26: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/26.jpg)
Financial outcome of applying a model
Prior Churn Churn Cost Cost without ML
14.49% $500.00 $72.46
![Page 27: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/27.jpg)
Financial outcome of applying a model
Prior Churn Churn Cost Cost without ML
14.49% $500.00 $72.46
False Negative True + False Pos Retention Cost Cost with ML
4.80% 12.10% + 14.30% $100.00 $50.40
![Page 28: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/28.jpg)
Financial outcome of applying a model
Prior Churn Churn Cost Cost without ML
14.49% $500.00 $72.46
False Negative True + False Pos Retention Cost Cost with ML
4.80% 12.10% + 14.30% $100.00 $50.40
• Threshold 0.3 0.17
• $22.06 of savings per customer
• With 100,000 customers over $2MM in savings with ML
![Page 29: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/29.jpg)
What’s next?
• https://aws.amazon.com/getting-started/projects/build-
machine-learning-model/
• https://aws.amazon.com/machine-learning/developer-
resources/
• https://github.com/dbatalov/cost_based_ml
![Page 30: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/30.jpg)
Thank you!
Denis V. Batalov, PhD
AWS Solutions Architect, EMEA
@dbatalov
![Page 31: AWS re:Invent 2016: Predicting Customer Churn with Amazon Machine Learning (MAC307)](https://reader031.vdocument.in/reader031/viewer/2022030305/587543701a28abb8208b5769/html5/thumbnails/31.jpg)
Remember to complete
your evaluations!