techconnex think beyond bi: machine learning
TRANSCRIPT
enabling modern enterprise
1
Think Beyond Business Intelligence for your organization – Machine Learning
Albert Hui, MBA, M.A.Sc., CSMData Economist
05/03/2023 2enabling modern enterprise
About Me
2
• Co Founder of Data Economist, a data consulting firm based in Toronto.
• 18 years in data management consulting, business Intelligence, data warehousing and big data.
• Master in Engineering in the area of Artificial Intelligence • Big Data Architect, Data Scientist• Conference Speaker at IOUG, TOUG Collaborate since 2011• Technical editor on Oracle 12c Book. • M.A.Sc., MBA, University of Toronto• Toronto based • Twitter: @dataeconomist• Father of two twin boys
05/03/2023 3enabling modern enterprise
Agenda3
Objective of this Session
What is machine learning?
BI vs ML
Models and Tools
Use Cases
What’s for the Organization?
Concluding Thoughts
05/03/2023 4enabling modern enterprise
Objective of this Talk4
Introduction of Machine Learning
5
History of Data &Machine Learning
05/03/2023 6enabling modern enterprise
The Meaning of.....6
Data: Granular and raw and viewed as the
lowest level of abstraction from which information and knowledge are derived.
Information: Extracting data in order to effectively
derive value and meaning and establishing a relevant context, often selecting from many possible contexts.
05/03/2023 7enabling modern enterprise
The Meaning of.....7
Intelligence: Individuals differ from one another in their ability to understand complex ideas, to adapt effectively to the environment, to learn from experience.
Wisdom:A deep understanding of people, things, events or situations, resulting in the ability to choose or act to consistently produce the optimum results with a minimum of time and energy.
05/03/2023 8enabling modern enterprise
8
Gandalf
Legolas
05/03/2023 9enabling modern enterprise
Build Knowledge thru Experience
9
10
What is Machine Learning?
05/03/2023 11enabling modern enterprise
Definition11
Computer systems that automatically learn or improve with experience (data).
05/03/2023 12enabling modern enterprise
What is machine learning?12
Machine Learning is not new
05/03/2023 13enabling modern enterprise
Common Machine Learning Applications13
05/03/2023 14enabling modern enterprise
Machine learning: two major types14
Supervised Supervised learning is tasked with learning a function from
labelled training data in order to predict the value of any valid input. Common examples of supervised learning include classifying e-mail messages as spam, labelling Web pages according to their genre, and recognizing handwriting. Many algorithms are used to create supervised learners, the most common being neural networks, Support Vector Machines (SVMs), and Naive Bayes classifiers.
Unsupervised Unsupervised learning, is tasked with making sense of data
without any examples of what is correct or incorrect. It is most commonly used for clustering similar input into logical groups. It also can be used to reduce the number of dimensions in a data set in order to focus on only the most useful attributes, or to detect trends. Common approaches to unsupervised learning include k-Means, hierarchical clustering, and self-organizing maps
05/03/2023 15enabling modern enterprise
Data mining vs machine learning15
Data Mining Focus on extracting patterns, unknown properties on the data. Marketing Surveillance Fraud Detection science discovery Discover items usually purchased together
Machine learning Focus on extracting prediction models, based on known
properties learned from the training data E-Mail spam classification News-topic discovery Building recommendations
05/03/2023 16enabling modern enterprise
16
“I skate to where the puckIs going to be,not where it has been”
-- Wayne Gretzky
17
Let’s try it ourselves
05/03/2023 18enabling modern enterprise
Quick Quiz18
In US, a 45year male, Around 150-180K income, Post Graduate Education, if he wants to buy a car. Which brand?
05/03/2023 19enabling modern enterprise
Quick Quiz19
In US, a 45year male, 3 children, 180K income, Graduate School Education, if he wants to buy a car. And he lives in Texas, then which brand?
05/03/2023 20enabling modern enterprise
Quick Quiz20
In US, a 45year male, 3 children, Graduate School Education, making $60K/year. If he wants to buy a car. Then which brand?
21
BI vs. ML
05/03/2023 22enabling modern enterprise
Remember this?22
05/03/2023 23enabling modern enterprise
23
Make a decision
05/03/2023 24enabling modern enterprise
BI Reports ….24
05/03/2023 25enabling modern enterprise
Key factors of Successful Enterprises25
05/03/2023 26enabling modern enterprise
26
05/03/2023 27enabling modern enterprise
Learning Enterprise27
05/03/2023 28enabling modern enterprise
28
05/03/2023 29enabling modern enterprise
Decide – Learn - Experience29
Make better Builds
05/03/2023 30enabling modern enterprise
How do we keep up changes?30
When the winds of change blow,some people build walls and others build windmills
-- Chinese proverb
05/03/2023 31enabling modern enterprise
31
32
Models & Tools
05/03/2023 33enabling modern enterprise
Main Models33
Classification Regression
Clustering
05/03/2023 34enabling modern enterprise
Machine Learning Components34
05/03/2023 35enabling modern enterprise
Machine Learning : Data35
05/03/2023 36enabling modern enterprise
Modelling Concept (Dimensionality Reduction)
SeasonsWeather
Sal
es
Project three dimensional space into two dimensional space
Principal Component Analysis
05/03/2023 37enabling modern enterprise
Modelling Concept (generalization) 37
x
y Is it a better model?
31
52
54
4
332
210 xxbxbxxbxbxbby
)min(0
ii
n
i
b
Objective Function
Regularization
05/03/2023 38enabling modern enterprise
Build the model38
Learn a model from a manually trained dataset Predict the class of an unseen object based on features
05/03/2023 39enabling modern enterprise
Build the model: iterative process39
There is no single answer/model to your questions. It is often based on trial and error.
05/03/2023 40enabling modern enterprise
Some Models Available40
Multiple Variable Regression Decision Tree/Random Forrest Logistic Regression Neural Network Fuzzy Logic Support Vector Machine Bayesian Network K-means KNN – Knearest Neigbors
05/03/2023 41enabling modern enterprise
Tools 41
42
Use Case #1
Ad Click-thru rate Prediction using Machine Learning
05/03/2023 43enabling modern enterprise
43
05/03/2023 44enabling modern enterprise
Clickstream – Case Demo44
An Asia based Hotspot Wi-Fi provider Revenue Model: Advertising
Advertisers place ads before users can connect to Wifi
Data Survey data: Users are required to fill a survey before
logging in. Click logs including Ad click-through
Data Size: 12GB+ compressed a day. 15M signed on and 6% click-thru a day.
Problem definition: click-through rate is too low
Demo
05/03/2023 45enabling modern enterprise
Preference Matching : Clustering 45
Matching
Millions of People Thousands of Ads
05/03/2023 46enabling modern enterprise
Logistic Regression46
05/03/2023 47enabling modern enterprise
Ad: n-dimensional vectors Each Ad is represented as a vector of
customers who have clicked the ads. Probability of Ad clicked based on
how close with individual Ad vectors.Vector for customers who clicked and viewed burger king
Vector for customers who clicked and viewed wholefood
Vector for an signed on customer
05/03/2023 48enabling modern enterprise
Improvement48
Click-thru rate increased from 6% to 13% first year and 19% next
year.
49
Use Case #2
A Major Canadian retailer
05/03/2023 50enabling modern enterprise
Data Science – First Question50
What are the customers saying?
05/03/2023 51enabling modern enterprise
Listen to Customers51
install.packages("twitteR")
rdmTweets <- userTimeline("rdatamining", n=100)
install.packages("ROAuth")install.packages("RCurl")install.packages("tm")install.packages("SnowballC")install.packages("wordcloud")twitCred <- OAuthFactory$new(consumerKey="rp4fHagxYdsdfoSMR3FiOg", consumerSecret="webKiy93XZZDLQcBJzxHw64regfgZvbivJIpLctYcPKY", requestURL="https://api.twitter.com/oauth/request_token", authURL="https://api.twitter.com/oauth/authorize", accessURL="https://api.twitter.com/oauth/access_token")
wordcloud(d$word, d$freq, min.freq=3)
05/03/2023 52enabling modern enterprise
NPL : Sentiment Analysis 52
weather
waiting
Olympics
05/03/2023 53enabling modern enterprise53
Words PopularityOlympics 0.9Winter 0.9Customer Service 0.85Computer 0.85Annoy 0.85Auto Oil 0.7horrible 0.7line 0.7wait 0.7Found 0.65traffic 0.6walmart 0.6good 0.6back 0.57Target 0.56Parking 0.56sick 0.55
Association AssociationGame thanks
Olympics weatherwait productwait slowAsile wait
Engine pricewait stand
standcustomer Service
time findline stand
target weatherphone find
Olympics familyworst goodfind parking
target findservice slow
dictCorpus <- myCorpus# stem words in a text document with the snowball stemmers, # which requires packages Snowball, RWeka, rJava, RWekajarsmyCorpus <- tm_map(myCorpus, stemDocument)# inspect the first three ``documents"inspect(myCorpus[1:3])
myCorpus <- tm_map(myCorpus, stemCompletion, dictionary=dictCorpus)
inspect(myCorpus[1:3])
myDtm <- TermDocumentMatrix(myCorpus, control = list(minWordLength = 1))
#inspect(myDtm[266:270,31:40])print(myDtm)
findFreqTerms(myDtm, lowfreq=2)
findAssocs(myDtm, 'Olympics', 0.10)findAssocs(myDtm, 'service', 0.10)
NPL : Sentiment Analysis
05/03/2023 54enabling modern enterprise
Improvements54
Improve the customer service line
Increase Customer Service Staff on Sat.
Reduce Wait Time
05/03/2023 55enabling modern enterprise
Data Science – What Impact Sales55
Product
Sales
Weather
StoreDemograp
hic
Product Review
Inventory
On-line Clickstrea
m
Competitors
Store Size &
Attributes
Product Price
Environmental Data
Environic Data
Site, ForumReview, twitter
Store Inventory
Site Clickstream
Marketing Analysis
DealerManagement
Promotion
05/03/2023 56enabling modern enterprise
Data Science – What Impact Sales56
Multivariate Linear Regression
57
Use Case # 3
Cash flow projectionfor a major bank
05/03/2023 58enabling modern enterprise
Deposit Saving : Asset Liability Modeling 58
Asset Allocation Optimization
1. Predicting cash balance for customer segments.
2. Optimizing lending and asset allocation.
3. Minimize liquidity risk.4. Enhance pricing strategy.5. Manage better customer
relationship.
Every 10K deposit
10d 20d 1m …... 6m 1y 2 y
Survival modelfor Cash flow projectionbased on customerprofile
Retirees 65+
Single Saver
40 with kids
Single spender
05/03/2023 59enabling modern enterprise
Use Case #3: Benefits59
Estimated 100+ Millions per year revenue opportunity
60
What’s for your organization?
05/03/2023 61enabling modern enterprise
Enhance your BI Reports with prediction capability
61
Provide predictive capability in your BI Reports.
05/03/2023 62enabling modern enterprise
Decision Architecture62
models
businessapplications
BI Reports
Feedback
05/03/2023 63enabling modern enterprise
Decision as a Service 63
Make betterBuilds
05/03/2023 64enabling modern enterprise
Concluding Thoughts:64
In this connected age, what is the most
disruptive?
05/03/2023 65enabling modern enterprise
Customers are actually more disruptive than technology
05/03/2023 66enabling modern enterprise
Decisions Made by IoT
To create the experience, decisions need to be made where the events happen.
05/03/2023 67enabling modern enterprise
Thank you!67
Albert Hui, MBA, MASc., P.Eng, CSM
Data Economist Inc.,Email: [email protected] me at Twitter: @dataeconomist