lecture 9.3 introduction to machine learning• support vector machines (svm) • … 8 class...
TRANSCRIPT
![Page 1: Lecture 9.3 Introduction to Machine Learning• Support Vector Machines (SVM) • … 8 Class conditional probability density functions 9 Bayesian decision theory Overview 10 x Density](https://reader033.vdocument.in/reader033/viewer/2022051904/5ff5b7fcfa3f9a34bc08e864/html5/thumbnails/1.jpg)
Lecture 9.3 Introduction to Machine Learning
Idar Dyrdal
![Page 2: Lecture 9.3 Introduction to Machine Learning• Support Vector Machines (SVM) • … 8 Class conditional probability density functions 9 Bayesian decision theory Overview 10 x Density](https://reader033.vdocument.in/reader033/viewer/2022051904/5ff5b7fcfa3f9a34bc08e864/html5/thumbnails/2.jpg)
Machine learning (Pattern recognition)
• Recognition of individuals (instance recognition) • Discrimination between classes (pattern recognition, classification)
Training Labels
Training Images
Classifier Training
Supervised learning
Image Features
Trained Classifier
2
![Page 3: Lecture 9.3 Introduction to Machine Learning• Support Vector Machines (SVM) • … 8 Class conditional probability density functions 9 Bayesian decision theory Overview 10 x Density](https://reader033.vdocument.in/reader033/viewer/2022051904/5ff5b7fcfa3f9a34bc08e864/html5/thumbnails/3.jpg)
Pattern recognition in practice
Working applications of Image Pattern recognition: • Reading license plates, postal codes, bar codes • Character recognition • Automatic diagnosis of medical samples • Fingerprint recognition • Face detection and recognition • …
3
![Page 4: Lecture 9.3 Introduction to Machine Learning• Support Vector Machines (SVM) • … 8 Class conditional probability density functions 9 Bayesian decision theory Overview 10 x Density](https://reader033.vdocument.in/reader033/viewer/2022051904/5ff5b7fcfa3f9a34bc08e864/html5/thumbnails/4.jpg)
Classification system
4
Feature extractor Classifier
Pine
Birch
Camera
![Page 5: Lecture 9.3 Introduction to Machine Learning• Support Vector Machines (SVM) • … 8 Class conditional probability density functions 9 Bayesian decision theory Overview 10 x Density](https://reader033.vdocument.in/reader033/viewer/2022051904/5ff5b7fcfa3f9a34bc08e864/html5/thumbnails/5.jpg)
Image features for object recognition
5
![Page 6: Lecture 9.3 Introduction to Machine Learning• Support Vector Machines (SVM) • … 8 Class conditional probability density functions 9 Bayesian decision theory Overview 10 x Density](https://reader033.vdocument.in/reader033/viewer/2022051904/5ff5b7fcfa3f9a34bc08e864/html5/thumbnails/6.jpg)
Feature vector and feature space
6
![Page 7: Lecture 9.3 Introduction to Machine Learning• Support Vector Machines (SVM) • … 8 Class conditional probability density functions 9 Bayesian decision theory Overview 10 x Density](https://reader033.vdocument.in/reader033/viewer/2022051904/5ff5b7fcfa3f9a34bc08e864/html5/thumbnails/7.jpg)
Training of classifiers
7
Learn a function to predict the class from the given features
Region 1
Region 2
Decision boundary
![Page 8: Lecture 9.3 Introduction to Machine Learning• Support Vector Machines (SVM) • … 8 Class conditional probability density functions 9 Bayesian decision theory Overview 10 x Density](https://reader033.vdocument.in/reader033/viewer/2022051904/5ff5b7fcfa3f9a34bc08e864/html5/thumbnails/8.jpg)
Classifiers and training methods
• Bayes classifier • Nearest-neighbors and K-nearest-neighbors • Parzen windows • Linear and higher order discriminant functions • Neural nets • Support Vector Machines (SVM) • …
8
![Page 9: Lecture 9.3 Introduction to Machine Learning• Support Vector Machines (SVM) • … 8 Class conditional probability density functions 9 Bayesian decision theory Overview 10 x Density](https://reader033.vdocument.in/reader033/viewer/2022051904/5ff5b7fcfa3f9a34bc08e864/html5/thumbnails/9.jpg)
Class conditional probability density functions
9
![Page 10: Lecture 9.3 Introduction to Machine Learning• Support Vector Machines (SVM) • … 8 Class conditional probability density functions 9 Bayesian decision theory Overview 10 x Density](https://reader033.vdocument.in/reader033/viewer/2022051904/5ff5b7fcfa3f9a34bc08e864/html5/thumbnails/10.jpg)
Bayesian decision theory
Overview
10
x
![Page 11: Lecture 9.3 Introduction to Machine Learning• Support Vector Machines (SVM) • … 8 Class conditional probability density functions 9 Bayesian decision theory Overview 10 x Density](https://reader033.vdocument.in/reader033/viewer/2022051904/5ff5b7fcfa3f9a34bc08e864/html5/thumbnails/11.jpg)
Density estimation
Parametric methods: • Assume a given shape of the density
function • Use the training set to estimate the
unknown parameters.
Non-parametric (distribution free) methods: • Point estimation of the density using
the training set directly • Parzen windows • Nearest neighbor estimation (leads
directly to the nearest-neighbor and k-nearest-neighbor classifiers).
11
Example – Gaussian distribution:
![Page 12: Lecture 9.3 Introduction to Machine Learning• Support Vector Machines (SVM) • … 8 Class conditional probability density functions 9 Bayesian decision theory Overview 10 x Density](https://reader033.vdocument.in/reader033/viewer/2022051904/5ff5b7fcfa3f9a34bc08e864/html5/thumbnails/12.jpg)
Parameter estimation
12
![Page 13: Lecture 9.3 Introduction to Machine Learning• Support Vector Machines (SVM) • … 8 Class conditional probability density functions 9 Bayesian decision theory Overview 10 x Density](https://reader033.vdocument.in/reader033/viewer/2022051904/5ff5b7fcfa3f9a34bc08e864/html5/thumbnails/13.jpg)
Discriminant functions
13
Estimate of the density in a given point:
From Bayes rule:
Example of a discriminant function: Decision rule: Choose the class with maximum discriminant function value.
![Page 14: Lecture 9.3 Introduction to Machine Learning• Support Vector Machines (SVM) • … 8 Class conditional probability density functions 9 Bayesian decision theory Overview 10 x Density](https://reader033.vdocument.in/reader033/viewer/2022051904/5ff5b7fcfa3f9a34bc08e864/html5/thumbnails/14.jpg)
Example - linear classifier
14
Uncorrelated features and common covariance matrices Linear decision boundaries
![Page 15: Lecture 9.3 Introduction to Machine Learning• Support Vector Machines (SVM) • … 8 Class conditional probability density functions 9 Bayesian decision theory Overview 10 x Density](https://reader033.vdocument.in/reader033/viewer/2022051904/5ff5b7fcfa3f9a34bc08e864/html5/thumbnails/15.jpg)
Dimensionality reduction - linear transformations
• PCA, ICA, LLE, Isomap • PCA (Principal Components Analysis)
is one of the most important techniques for dimensionality reduction
• It takes advantage of correlations between the features to produce the best possible lower dimensional representation of the data with respect to reconstruction error
• The eigenvectors of the lumped covariance matrix defines the new features in the transformed feature space.
15
![Page 16: Lecture 9.3 Introduction to Machine Learning• Support Vector Machines (SVM) • … 8 Class conditional probability density functions 9 Bayesian decision theory Overview 10 x Density](https://reader033.vdocument.in/reader033/viewer/2022051904/5ff5b7fcfa3f9a34bc08e864/html5/thumbnails/16.jpg)
Summary
Recognition: • Pattern classification • Training of classifiers (supervised learning) • Parametric and non-parametric methods • Discriminant functions • Dimensionality reduction
Read also: Szeliski 14.1
16