support vector machine
Post on 26-Jan-2015
497 Views
Preview:
DESCRIPTION
TRANSCRIPT
SEN – 935 DATA MINING
Anandha L Ranganathan
Support Vector Machine
17. Support Vector Machine 1
Pre 1980s - Almost all learning methods learned linear decision surfaces. - Linear learning methods have nice theoretical properties
1980s - Almost all learning methods learned linear decision surfaces. - Linear learning methods have nice theoretical properties
1990’s - Efficient learning algorithms for non-linear functions based on
computational learning theory developed - Nice theoretical properties.
History
17. Support Vector Machine 2
Two independent developments within last decade
– Computational learning theory– New efficient separability of non-linear functions that use “kernel
functions”
The resultant learning algorithm is optimization algorithm rather than a greedy search.
What is greedy search ?
History
17. Support Vector Machine 3
Find largest sum by traversing through path.
Greedy search
17. Support Vector Machine 4
Find largest sum by traversing through path.
Greedy search
17. Support Vector Machine 5
Find largest sum by traversing through path.
Greedy search
17. Support Vector Machine 6
A system receives data as input. Output a function that can be predict some
features of future data.
Learning Theory
f x yest
f(x,w,b) = sign(w. x - b)
17. Support Vector Machine 7
Not affected by local minima. Do not suffer from the curse of dimensionality. Have modular design that allows one to
separately implement and design other component.
Various properties of the SVM solution help avoid over fitting, even in very high dimensional feature spaces
Features of SVM's
17. Support Vector Machine 8
Support vectors are data points that lie closes to the decision surface.
But they are difficult to classify. They have direct bearing of optimum location
on the surface.
Support Vectors
17. Support Vector Machine 9
d1 and d2 are 2 vectors. And sum of their distance is d1+d2=q.
d1=2x+5y and d2=3x+2y q=d1+d2=5x+7y
Vector Space – Primer
17. Support Vector Machine 10
Computing the inner products between the vectors in the featured space.
Kernal – Primer
17. Support Vector Machine 11
Linear Classifiers
denotes +1
denotes -1
How would you classify this data?
17. Support Vector Machine 12
denotes +1
denotes -1
How would you classify this data?
Linear Classifiers
17. Support Vector Machine 13
denotes +1
denotes -1
How would you classify this data?
Linear Classifiers
17. Support Vector Machine 14
denotes +1
denotes -1
How would you classify this data?
Linear Classifiers
17. Support Vector Machine 15
denotes +1
denotes -1
How would you classify this data?
Linear Classifiers
17. Support Vector Machine 16
denotes +1
denotes -1
Any of these would be fine..
..but which is best?
Linear Classifiers
17. Support Vector Machine 17
Classifier Margin
denotes +1
denotes -1 Define the margin of a linear classifier as the width that the boundary could be increased by before hitting a datapoint.
17. Support Vector Machine 18
Maximum Margin
denotes +1
denotes -1 The maximum margin linear classifier is the linear classifier with the maximum margin.
This is the simplest kind of SVM (Called an LSVM)
Linear SVM17. Support Vector Machine 19
Formulating SVM
17. Support Vector Machine 20
Formulating SVM
17. Support Vector Machine 21
Formulating SVM
17. Support Vector Machine 22
Idea: map to higher dimensional feature space
Kernal - polynomial
17. Support Vector Machine 23
License Plate Recognition
17. Support Vector Machine 24
Pre-process the image of number plate. Segment the image into several parts of
which each contains only a single character.
License Plate Recognition
17. Support Vector Machine 25
Extract the feature vector of each normalized candidate
Recognizes the single character (a digit or a letter) by the set of SVMs trained in advance.
License Plate Recognition
17. Support Vector Machine 26
License Plate Recognition
17. Support Vector Machine 27
License Plate Recognition
17. Support Vector Machine 28
License Plate Recognition
17. Support Vector Machine 29
License Plate Recognition
17. Support Vector Machine 30
License Plate Recognition
If there are no more unclassified samples, then STOP. Otherwise, then repeat the process of recognition of character.
Add these test samples into their corresponding database for further training.
Recognize number plate by bringing all characters used together
17. Support Vector Machine 31
SVM is widely used as classify spam detection in the market.
It supports for Linear and Non-Linear spectrum.
Conclusion
17. Support Vector Machine 32
http://www.cs.ucf.edu/courses/cap6412/fall2009/papers/Berwick2003.pdf
http://physiology.med.cornell.edu/people/banfelder/qbio/resources_2011/2011_Leslie.pdf
http://physiology.med.cornell.edu/people/banfelder/qbio/resources_2011/2011_Leslie.pdf
http://www.cs.columbia.edu/~kathy/cs4701/documents/jason_svm_tutorial.pdf
http://www.slideshare.net/wltongxing/svm-12978262
17. Support Vector Machine 33
top related