introduction of svmstruct hung-yi...
TRANSCRIPT
SVMstruct
โข API for implementing different kinds of structured learning tasks by structured SVMโข C:
https://www.cs.cornell.edu/people/tj/svm_light/svm_struct.html
โข Python: http://tfinley.net/software/svmpython2/
โข MATLAB: http://www.robots.ox.ac.uk/~vedaldi/svmstruct.html
โข Structured SVM with Hidden Informationโข http://www.cs.cornell.edu/~cnyu/latentssvm/
SVMstruct
โข Python version in these slides
โข Download from http://tfinley.net/software/svmpython2/svm-python-v204.tgz
โข To build this, a simple make should work.
โข svm_python_learn: for learning a model and
โข svm_python_classify: for testing with a learned model
svm_python_learn
๐ฅ1, ๐ฆ1 , ๐ฅ2, ๐ฆ2 , โฏ , ๐ฅ๐, ๐ฆ๐ = read_examples(TrainingData)
init_model()
w โ 0, working set ๐ธ1 โ ๐๐ข๐๐, ๐ธ2 โ ๐๐ข๐๐,โฏ , ๐ธ๐ โ ๐๐ข๐๐Repeat
๐ค โSolve a QP with Working Set ๐ธ1, ๐ธ2, โฏ , ๐ธ๐
argmax๐ฆ
โ ๐ฆ๐, ๐ฆ + ๐ค โ ๐ ๐ฅ๐, ๐ฆ
๐ฆ๐ =
For each training data:
Until the stopping criterion fulfilled
Return w
Update working set ๐ธ๐ โ ๐ธ๐ โช ๐ฆ๐
What you have to implement by yourself
find_most_violated_constraint(๐ฅ๐, ๐ฆ๐)
svm_python_learn - QP
Find ๐ค, ๐1โฆ๐๐ minimizing 1
2๐ค 2 + ๐
๐=1
๐
๐๐QP:
๐ค โ ๐ ๐ฅ๐, ๐ฆ๐ โ ๐ ๐ฅ๐, ๐ฆ โฅ โ ๐ฆ๐, ๐ฆ โ๐๐ , ๐๐ โฅ 0
For โ๐ฆ โ ๐ธ๐:
For โ๐:
psi(๐ฅ, ๐ฆ) loss( ๐ฆ๐, ๐ฆ)
Usage: ./svm_python_learn โc 1000 TrainingData Model
-e value (smaller, training time longer)
-k int (number of new constraints to accumulate before recomputing the QP)
svm_python_classify
Usage: ./svm_python_classify TestingData Model Result
For each test data:
๐ฆโฒ โ ๐ผ๐๐๐๐๐๐๐๐ ๐ฅโฒ, ๐คclassify_example (๐ฅ๐) Inference
argmax๐ฆ
๐ค โ ๐ ๐ฅ๐, ๐ฆ
svmstruct.py
โข Put your code in svmstruct.py
Using multi-class classification to show how to implement each function
http://tfinley.net/software/svmpython2/multiclass_code.html
read_examples(filename)
init_model()
find_most_violated_constraint(๐ฅ๐, ๐ฆ๐)
psi(๐ฅ, ๐ฆ) loss( ๐ฆ๐, ๐ฆ)
classify_example (๐ฅ๐)
read_examples
โข Format of โexamplesโ (training data)
โข The whole โexamplesโ (training data) is a list
โข Each data is a tuple with two element
๐๐ฅ๐๐๐๐๐๐ = [ ๐ฅ1, ๐ฆ1 , โฆ ๐ฅ๐, ๐ฆ๐ , โฆ ๐ฅ๐, ๐ฆ๐ ]
Input object output object
a scalar
[(3,6.55),(7,-1.01),โฆ(10,0.4)]
Sparse representation of a vector
init_model
โข Tell SVMstruct the feature dimension of w
e.g. K classes, feature representation of x is M dimensions
sm.size_psi = K * M
feature dimension of w
Model
[(3+(k-1)*M,6.55),(7+(k-1)*M,-1.01),โฆ(10+(k-1)*M,0.4)]
psi
0
0
๐ฅ
โฏโฏ
๐ ๐ฅ, ๐ฆ =
๐พ ร ๐ dims
K blocks
Each has M dimensions
[(3,6.55),(7,-1.01),โฆ(10,0.4)]
vec = svmapi.Sparse( )
return vec
If x is class k, put ๐ฅ at kth block
y = k (class k)
x:
0 for other dimensions
find_most_violated_constraint ๐ฆ๐ = argmax
๐ฆโ ๐ฆ๐, ๐ฆ + ๐ค โ ๐ ๐ฅ๐, ๐ฆ
w = sm.w
vec = psi(x,y)
for k, v in vec: The value at k dimension is v.
score = 0
score = score + v * w[k]
๐ค โ ๐ ๐ฅ๐, ๐ฆ