week 1 - an introduction to machine learning & soft computing -yosi kristian-
TRANSCRIPT
Week 1 - An Introduction to Machine Learning & Soft Computing
-Yosi Kristian-
Soft Computing STTS – Yosi Kristian 2
Soft Computing STTS – Yosi Kristian 3
Definition• Soft Computing is a term applied to a field within computer
science which is characterized by the use of inexact solutions to computationally hard tasks, for which there is no known algorithm that can compute an exact solution.
• Soft computing differs from conventional (hard) computing in that, unlike hard computing, it is tolerant of imprecision, uncertainty, partial truth, and approximation.
Soft Computing STTS – Yosi Kristian 4
Still the Definition..• In effect, the role model for soft computing is the human
mind. • The guiding principle of soft computing is: Exploit the
tolerance for imprecision, uncertainty, partial truth, and approximation to achieve tractability, robustness and low solution cost.
Soft Computing STTS – Yosi Kristian 5
Components of soft computing
• Neural networks (NN)• Support Vector Machines (SVM)• Fuzzy logics (FL)• Evolutionary computation (EC), including:
o Evolutionary algorithms• Genetic algorithms• Differential evolution
o Meta heuristic and Swarm Intelligence• Ant colony optimization• Bees algorithms• Bat algorithm• Cuckoo search• Harmony search• Firefly algorithm• Artificial immune systems• Particle swarm optimization
What ???Are we going to learn them
allin this
subject?
Soft Computing STTS – Yosi Kristian 6
Soft Computing in AI• Soft computing may be viewed as a foundation component for
the emerging field of conceptual intelligence.o Machine Learningo Fuzzy Systemso Evolutionary Computationo Probabilistic Reasoning
• Soft Computing is the CORE component of many Machine Learning System
Soft Computing STTS – Yosi Kristian 7
Machine Learning• Arthur Samuel (1959). Machine Learning: Field of study that
gives computers the ability to learn without being explicitly programmed.
• Tom Mitchell (1998) Well-posed Learning Problem: A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.
Soft Computing STTS – Yosi Kristian 8
Machine learning usage• Usage of Machine Learning is to develop applications
that can’t be programed by hand.• E.g., Autonomous helicopter, handwriting recognition,
most of Natural Language Processing (NLP), Computer Vision etc.
• Or a machine learning system could be trained on email messages to learn to distinguish between spam and non-spam messages. After learning, it can then be used to classify new email messages into spam and non-spam folders.
Soft Computing STTS – Yosi Kristian 9
Machine Learning Categorized By Data and Learning Process
Soft Computing STTS – Yosi Kristian 10
Soft Computing In Machine Learning• Soft Computing is the soul of many machine
learning system.• Classification and Clustering is a very common
soft computing problems.
Soft Computing STTS – Yosi Kristian 11
Intro to Supervised Learning
Soft Computing STTS – Yosi Kristian 12
Example
0 500 1000 1500 2000 25000
100
200
300
400Housing price prediction.
Price ($) in 1000’s
Size in feet2
Regression: Predict continuous valued output (price)
Supervised Learning“right answers” given
Soft Computing STTS – Yosi Kristian 13
Example
Breast cancer (malignant, benign)
ClassificationDiscrete valued output (0 or 1)
Malignant?
1(Y)
0(N)
Tumor Size
Tumor Size
Soft Computing STTS – Yosi Kristian 14
Another Example
Tumor Size
Age
- Clump Thickness- Uniformity of Cell
Size- Uniformity of Cell
Shape…
Soft Computing STTS – Yosi Kristian 15
Exercise• You’re running a company, and you want to develop learning
algorithms to address each of two problems.• Problem 1: You have a large inventory of identical items. You
want to predict how many of these items will sell over the next 3 months.
• Problem 2: You’d like software to examine individual customer accounts, and for each account decide if it has been hacked/compromised.
• Should you treat these as classification or as regression problems?
Soft Computing STTS – Yosi Kristian 16
Answer• Treat both as classification problems. • Treat problem 1 as a classification problem, problem 2 as a
regression problem. • Treat problem 1 as a regression problem, problem 2 as a
classification problem. • Treat both as regression problems.
Soft Computing STTS – Yosi Kristian 17
Classification Example• Another Example is for image Classification / Categorization
Training LabelsTraining
Images
Classifier
Training
Training
Image Features
Trained Classifier
Soft Computing STTS – Yosi Kristian 18
Cont…
Image Features
Testing
Test Image
Trained Classifier
Outdoor
Prediction
Soft Computing STTS – Yosi Kristian 19
Learning a classifier• Given some set of features with corresponding labels, learn a
function to predict the labels from the features• Training labels dictate that two examples are the same or
different, in some sense• Features and distance measures define similarity• Classifiers try to learn weights or parameters for features and
distance measures so that feature similarity predicts label similarity
Soft Computing STTS – Yosi Kristian 20
Intro to Unsupervised Learning
Soft Computing STTS – Yosi Kristian 21
Supervised Learning
x1
x2
Soft Computing STTS – Yosi Kristian 22
Unsupervised Learning
x1
x2
Soft Computing STTS – Yosi Kristian 23
Clustering Example
Soft Computing STTS – Yosi Kristian 24
Contd…
Soft Computing STTS – Yosi Kristian 25
ExerciseOf the following examples, which would you address using an
unsupervised learning algorithm? (Check all that apply.) o Given email labeled as spam/not spam, learn a spam filter.o Given a set of news articles found on the web, group them into set of
articles about the same story. o Given a database of customer data, automatically discover market
segments and group customers into different market segments. o Given a dataset of patients diagnosed as either having diabetes or not,
learn to classify new patients as having diabetes or not.
Soft Computing STTS – Yosi Kristian 26
Warming Up….• Do 10 x Push Ups.
Soft Computing STTS – Yosi Kristian 27
Linear Regression with one variable
500 1000 1500 2000 2500 30000
100000
200000
300000
400000
500000
500 1000 1500 2000 2500 30000
100000
200000
300000
400000
500000Housing Prices(Portland, OR)
Price(in
1000s of dollars)
Size (feet2)
Supervised Learning
Given the “right answer” for each example in the data.
Regression Problem
Predict real-valued output
Soft Computing STTS – Yosi Kristian 28
Linear Regression with one variable
Notation:
m = Number of training examples n = Number of feature x’s = “input” variable / features y’s = “output” variable / “target” variable
Training set ofhousing prices(Portland, OR)
Soft Computing STTS – Yosi Kristian 29
The Concept
Training Set
Learning Algorithm
hSize of
house
Estimated price
How do we represent h ?
Linear regression with one variable.Univariate linear regression.
Soft Computing STTS – Yosi Kristian 30
Cost Function
How to choose ‘s ?
Training Set
Hypothesis:
‘s: Parameters
Soft Computing STTS – Yosi Kristian 31
Contd..
0 1 2 30
1
2
3
0 1 2 30
1
2
3
0 1 2 30
1
2
3
Soft Computing STTS – Yosi Kristian 32
Cost Function..
y
x
Idea: Choose so that is close to for our training examples
Soft Computing STTS – Yosi Kristian 33
Simplification: For the sake of understanding
Hypothesis:
Parameters:
Cost Function:
Goal:
Simplified
Soft Computing STTS – Yosi Kristian 34
Trial 1
0 1 2 30
1
2
3
y
x
(for fixed , this is a function of x) (function of the parameter )
-0.5 0 0.5 1 1.5 2 2.50
1
2
3
Soft Computing STTS – Yosi Kristian 35
Trial 2
0 1 2 30
1
2
3
y
x
(for fixed , this is a function of x) (function of the parameter )
-0.5 0 0.5 1 1.5 2 2.50
1
2
3
Soft Computing STTS – Yosi Kristian 36
Trial 3
-0.5 0 0.5 1 1.5 2 2.50
1
2
3
y
x
(for fixed , this is a function of x) (function of the parameter )
0 1 2 30
1
2
3
Soft Computing STTS – Yosi Kristian 37
Done with simplification, back to real world.
Hypothesis:
Parameters:
Cost Function:
Goal:
Soft Computing STTS – Yosi Kristian 38
The Contour Figures ….
How to find minimum of J in that?
Soft Computing STTS – Yosi Kristian 39
Gradient Descent..• Next Week…