adventures in data science
Post on 15-Apr-2017
195 Views
Preview:
TRANSCRIPT
Brian MoranWherein our intrepid explorer provides an account of many curious encounters with data...
28 September 2016
Adapted from http://www.anlytcs.com/2014/01/data-science-venn-diagram-v20.html
Data Science
Mathematics and Statistics
Research Methods
UNICORN
Machine Learning
Subject Matter Expertise
Programming Skills
Computer Science
What Does it Involve?
Business Understanding
Data Understanding
Data PreparationModelling
Evaluation
Deployment
What Does it Do?
The CRISP data mining process
Check email
Naive Bayes
Internet search
PageRank
Watch Netflix
Boltzman machine
Buy lunch
Artificial neural net
Use Sat-Nav
Dijkstra’s algorithm
Apply for a loan
Decision trees
Shop on Amazon
Matrix factorization
Get a letter
k-means clustering
Diagrams (but not text) from: http://machinelearningmastery.com/a-tour-of-machine-learning-algorithms/
Data Science Applications are Everywhere
Welcome back Suzy
Welcome back Jon
Recommend money advice
Alert
Close
82%Likelihood:
Recommend alternative
Alert
Close
90%Chance of broken appointment:
Better alternative:
2:00pm - 5:00pm
Recommend incentives
Advice
Close
Tenancy value:
Negative Low High
One click repair adactus
1. Repairrequest Delay Staff
checkStaffinput
Repairordered
2. Repairrequest
Repairordered
3. Repairrequest
MachineLearning
Staffcheck
Staffinput
Repairordered
RepairorderedProblem? N
Y
Options for Back-office Repairs Process
Proof of concept 1: Natural language processing with Naive Bayes…
TRADEREPAIR REQUEST
Check web form
Naive Bayes
“Our ba th room is leaking through the ceiling onto the stairs and the ceiling is wet through along with the walls where the taps are mounted. And puddle on the stairs”
PLUMBER
Check web form
Naive Bayes
Check web form
Naive Bayes
bathroomleaking
ceilingstairsceiling
wet
walls taps
stairs
PLUMBER
-£300,000
-£225,000
-£150,000
-£75,000
£0
£75,000
£150,000
£225,000
£300,000
10% channel shift 20% channel shift 30% channel shift 40% channel shift 50% channel shift 60% channel shift 70% channel shiftNet c
ost /
ben
efit o
f rep
airs
sel
f-ser
vice
90% accuracy
10% accuracy
50% accuracy
‘Friction Free’ Self Service: Avoiding Pyrrhic Victories
Proof of concept 2: Clustering with K-means…
CUSTOMISE SERVICES & INFORMATION
Assign to cluster
K-means clustering
Blocked means of escape
Alert
Close Blocked for one day
56 King StreetLeighWN7 4LJ
Unusual transaction
Alert
Close
Audit file ready
Advice
Close
Values beginning with
Highlighted for attention
1166
Proof of concept 3: Identifying unnatural numbers with Benford’s Law…
Find outliers
Benford’s Law
1 2 3 4 5 6 7 8 9
6’0”
5’0”
4’0”
3’0”
5’6”
4’6”
3’6”
6’0”
5’0”
4’0”
3’0”
5’6”
4’6”
3’6”I’m going to beat Benford’s Law
I’m going to beat Benford’s Law
I’m going to beat Benford’s Law I’m going to beat
Benford’s Law
6’0”
5’0”
4’0”
3’0”
5’6”
4’6”
3’6”
6’0”
5’0”
4’0”
3’0”
5’6”
4’6”
3’6”
BUSTED!
BUSTED!
BUSTED!
CRIMINAL GENIUS!
Payment cycle broken
Alert
Close
Proof of concept 4: Forecasting arrears with Holt-Winters…
Forecast time series data
Holt-Winters Triple Exponential Smoothing
Staffing forecast ready
Advice
Close
Recommend replacement
Advice
Close
97%Likelihood:
Boiler beyond economic repair
Optimal time to sell property
Alert
Close
Housing Association Costs
Proof of concept 5: Tenancy survival with Kaplan-Meier…
Survival Analysis
Kaplan-Meier Estimator
S(t)=1−F(t) = Prob(T≥t)
Histogram of Tenancy Length (Current Tenants)
A Simple Model of Tenancy Lengths
Tenancy Length (years)
Tenancy Length (years)
A Simple Model of Tenancy Lengths
Tenancy Length (years)
Unlikely to need component
replacements
Likely to needcomponent replacements
Even Simple Models can Lead to Interesting Thoughts
RECEPTIONCONTACT CENTRE
Getting Technology isn't the Same as Really Getting it
Digital front of office Digital back of office +
Really Getting Channel Shift Choice
Brian Moran
brian.moran@ adactushousing.co.uk
top related