intro to machine learning for web folks @ blendwebmix
TRANSCRIPT
Intro to Machine Learningfor web folks
“Machine Learning, je m’y mets dès demain”
@louisdorard #blendwebmix 2015
–Mike Gualtieri, Principal Analyst at Forrester
“Predictive apps are the next big thing
in app development.”
Lars Trieloff
@trieloff
(see source)
–Waqar Hasan, VISA
“Predictive is the ‘killer app’ for big data.”
Amazon for David Jones (@d_jones, see source)
Amazon for David Jones (@d_jones, see source)
1. Machine Learning
2. Data
TECH ??
–Charles Parker, PhD, Allston Trading
“ML isn’t about ML”
BIZ
DESIGN
RECH
code
code
code
BLEND !!
@louisdorard
“Where makers of Predictive APIs and apps meet”
Machine Learning Use cases
Limitations Modern tools
Case study ML Canvas
DemystifyingMachine Learning
“Which type of email is this?
— Spam/Ham”
“Which type of email is this?
— Spam/Ham”
⇒ Classification
I
O
“Which type of email is this?
— Spam/Ham”
??
“How much is this house worth?
— X $”
-> Regression
Bedrooms Bathrooms Surface (foot²) Year built Type Price ($)
3 1 860 1950 house 565,0003 1 1012 1951 house2 1.5 968 1976 townhouse 447,0004 1315 1950 house 648,0003 2 1599 1964 house3 2 987 1951 townhouse 790,0001 1 530 2007 condo 122,0004 2 1574 1964 house 835,0004 2001 house 855,0003 2.5 1472 2005 house4 3.5 1714 2005 townhouse2 2 1113 1999 condo1 769 1999 condo 315,000
Bedrooms Bathrooms Surface (foot²) Year built Type Price ($)
3 1 860 1950 house 565,0003 1 1012 1951 house2 1.5 968 1976 townhouse 447,0004 1315 1950 house 648,0003 2 1599 1964 house3 2 987 1951 townhouse 790,0001 1 530 2007 condo 122,0004 2 1574 1964 house 835,0004 2001 house 855,0003 2.5 1472 2005 house4 3.5 1714 2005 townhouse2 2 1113 1999 condo1 769 1999 condo 315,000
ML is a set of AI techniques where “intelligence” is built by referring to
examples
Use cases
• Real-estate
• Spam
• Priority inbox
• Crowd prediction
property price
email spam indicator
email importance indicator
location & context #people
Zillow
Gmail
Gmail
Tranquilien
I. Get more customers • Reduce churn
• Score leads
• Optimize campaigns
customer churn indicator
customer revenue
customer & campaign interest indicator
II. Serve customers better • Cross-sell
• Increase engagement
• Optimize pricing
customer & product purchase indicator
user & item interest indicator
product & price #sales
III. Serve customers more efficiently • Predict demand
• Automate tasks
• Use predictive enterprise apps
context demand
credit application repayment indicator
Predictive enterprise apps • Priority filtering
• Message routing
• Auto-configuration
message priority indicator
request employee
user & actions settings
RULES
–Katherine Barr, Partner at VC-firm MDV
"Pairing human workers with machine learning and automation
will transform knowledge work and unleash new levels of human
productivity and creativity."
Limitations
Need examples of inputs AND outputs
What if not enough data points?
What if similar inputs have dissimilar outputs?
Bedrooms Bathrooms Price ($)
3 2 500,0003 2 800,0001 1 300,0001 1 800,000
Bedrooms Bathrooms Surface (foot²) Year built Price ($)
3 2 800 1950 500,0003 2 1000 1950 800,0001 1 500 1950 300,0001 1 500 2014 800,000
• Need examples of inputs AND outputs
• Need enough examples
• Need enough “features”
–@louisdorard
“A model can only be as good as the data it was given to train on”
–McKinsey & Co. (2011)
“A significant constraint on realizing value from big data will
be a shortage of talent, particularly of people with deep
expertise in statistics and machine learning.”
MLaaS & Predictive APIs: ML for all
HTML / CSS / JavaScript
HTML / CSS / JavaScript
The two phases of machine learning:
• TRAIN a model
• PREDICT with a model
The two methods of predictive APIs:
• TRAIN a model
• PREDICT with a model
The two methods of predictive APIs:
• model = create_model(dataset)
• predicted_output = create_prediction(model, new_input)
The two methods of predictive APIs:
• model = create_model(‘training.csv’)
• predicted_output = create_prediction(model, new_input)
From Large to Small & Medium Enterprises
• recommendations in e-commerce
• => 71% increase in revenue
• churn detection
• => 11% increase in retention
Microsoft Azure ML
PredictionIO
Case study:churn analysis
• Who: SaaS company selling monthly subscription
• Question asked: “Is this customer going to leave within 1 month?”
• Input: customer
• Output: no-churn or churn
• Data collection: history up until 1 month ago
• Baseline: if no usage for more than 15 days then churn
Learning: OK
but
• How to represent customers?
• What to do after predicting churn?
Customer representation:
• basic info (age, income, etc.)
• usage of service (# times used app, avg time spent, features used, etc.)
• interactions with customer support (how many, topics of questions, satisfaction ratings)
Taking action to prevent churn:
• contact customers (in which order?)
• switch to different plan
• give special offer
• no action?
Measuring accuracy:
• #TP (we predict customer churns and he does)
• #FP (we predict customer churns but he doesn’t)
• #FN (we predict customer doesn’t churn but he does)
• Compare to heuristic/baseline
Return On Investment:
• Taking action for each TP (and FP) has a cost
• For each TP we “gain”: (success rate of action) * (revenue /cust. /month)
• Imagine…
• perfect predictions
• revenue /cust. /month = 10€
• success rate of action = 20%
• cost of action = 2€
• Which ROI?
Machine Learning Canvas
Machine Learning Canvas
PREDICTIONS OBJECTIVES DATA
Context
Who will use the predictive system / who will beaffected by it? Provide some background.
Value Proposition
What are we trying to do? E.g. spend less time onX, increase Y...
Data Sources
Where do/can we get data from? (internaldatabase, 3rd party API, etc.)
Problem
Question to predict answers to (in plain English)
Input (i.e. question "parameter")
Possible outputs (i.e. "answers")
Type of problem (e.g. classification, regression,recommendation...)
Baseline
What is an alternative way of making predictions(e.g. manual rules based on feature values)?
Performance evaluation
Domain-specific / bottom-line metrics formonitoring performance in production
Prediction accuracy metrics (e.g. MSE ifregression; % accuracy, #FP for classification)
Offline performance evaluation method (e.g.cross-validation or simple training/test split)
Dataset
How do we collect data (inputs and outputs)?How many data points?
Features
Used to represent inputs and extracted fromdata sources above. Group by types andmention key features if too many to list all.
Using predictions
When do we make predictions and how many?
What is the time constraint for making those predictions?
How do we use predictions and confidence values?
Learning predictive models
When do we create/update models? With which data / how much?
What is the time constraint for creating a model?
Criteria for deploying model (e.g. minimum performance value — absolute,relative to baseline or to previous model)
IDE
AS
PE
CS
DE
PLO
YM
EN
T
BACKGROUND
ENGINE SPECS
INTEGRATION
PREDICTIONS OBJECTIVES DATA
BACKGROUND
ENGINE SPECS
INTEGRATION
PREDICTIONS OBJECTIVES DATA
BACKGROUND End-user Value prop Sources
ENGINE SPECS ML problem Perf eval Preparation
INTEGRATION Using pred Learning modelINTEGRATION Using pred Learning model
Why fill in ML canvas?
• Target the right problem for your company
• Choose right algorithm, infrastructure, or ML solution
• Guide project management
• Improve team communication
Recap
• ML to create value from data
• 2 phases: TRAIN and PREDICT
• MLaaS & Predictive APIs make it more accessible
• Good data is essential
• What do we do with predictions?
• Accuracy is not the objective! A/B test?
• Start with the ML Canvas
• Later: deploy, maintain, improve…
