support vector machine

SEN – 935 DATA MINING

Anandha L Ranganathan

Support Vector Machine

17. Support Vector Machine 1

Pre 1980s - Almost all learning methods learned linear decision surfaces. - Linear learning methods have nice theoretical properties

1980s - Almost all learning methods learned linear decision surfaces. - Linear learning methods have nice theoretical properties

1990’s - Efficient learning algorithms for non-linear functions based on

computational learning theory developed - Nice theoretical properties.

History


Two independent developments within last decade

– Computational learning theory– New efficient separability of non-linear functions that use “kernel

functions”

The resultant learning algorithm is optimization algorithm rather than a greedy search.

What is greedy search ?

History


Find largest sum by traversing through path.

Greedy search



Greedy search


A system receives data as input. Output a function that can be predict some

features of future data.

Learning Theory

f x yest

f(x,w,b) = sign(w. x - b)


Not affected by local minima. Do not suffer from the curse of dimensionality. Have modular design that allows one to

separately implement and design other component.

Various properties of the SVM solution help avoid over fitting, even in very high dimensional feature spaces

Features of SVM's


Support vectors are data points that lie closes to the decision surface.

But they are difficult to classify. They have direct bearing of optimum location

on the surface.

Support Vectors


d1 and d2 are 2 vectors. And sum of their distance is d1+d2=q.

d1=2x+5y and d2=3x+2y q=d1+d2=5x+7y

Vector Space – Primer


Computing the inner products between the vectors in the featured space.

Kernal – Primer


Linear Classifiers

denotes +1

denotes -1

How would you classify this data?


denotes +1

denotes -1


Linear Classifiers


denotes +1

denotes -1

Any of these would be fine..

..but which is best?

Linear Classifiers


Classifier Margin

denotes +1

denotes -1 Define the margin of a linear classifier as the width that the boundary could be increased by before hitting a datapoint.


Maximum Margin

denotes +1

denotes -1 The maximum margin linear classifier is the linear classifier with the maximum margin.

This is the simplest kind of SVM (Called an LSVM)

Linear SVM17. Support Vector Machine 19

Formulating SVM


Idea: map to higher dimensional feature space

Kernal - polynomial


License Plate Recognition


Pre-process the image of number plate. Segment the image into several parts of

which each contains only a single character.



Extract the feature vector of each normalized candidate

Recognizes the single character (a digit or a letter) by the set of SVMs trained in advance.




If there are no more unclassified samples, then STOP. Otherwise, then repeat the process of recognition of character.

Add these test samples into their corresponding database for further training.

Recognize number plate by bringing all characters used together


SVM is widely used as classify spam detection in the market.

It supports for Linear and Non-Linear spectrum.

Conclusion


http://www.cs.ucf.edu/courses/cap6412/fall2009/papers/Berwick2003.pdf

http://physiology.med.cornell.edu/people/banfelder/qbio/resources_2011/2011_Leslie.pdf

http://physiology.med.cornell.edu/people/banfelder/qbio/resources_2011/2011_Leslie.pdf

http://www.cs.columbia.edu/~kathy/cs4701/documents/jason_svm_tutorial.pdf

http://www.slideshare.net/wltongxing/svm-12978262


support vector machine

Technology