COLOR-ATTRIBUTES-RELATED IMAGE RETRIEVAL

WEEK 4

Student: Kylie Gorman

Mentor: Yang Zhang

GMM AND FISHER VECTOR CODE

STEP ONE: CONCATENATE THE FEATURE MATRICES OF EACH IMAGE

STEP TWO: APPLY GMM FUNCTION

Generate mean, covariance, and prior mode probabilities

Mean

Covariance

Prior Mode Probabilities

STEP THREE: CREATE FISHER VECTORS

encoding = vl_fisher(new', means, covariances, priors);

REPEAT PROCESS

FINAL STEPS

Calculate feature matrix of each image, isolating the object first

Concatenate matrices Apply PCA function to preprocess data Multiply each individual feature matrix by result Concatenate output into 1 matrix Apply GMM function and obtain mean,

covariance, and prior mode probabilities Apply Fisher Vector to each individual result to

obtain vectors that are the same size Use those fisher vectors for 11 SVM’s (one for

each color)

COMPLETE STEPS Using Ebay Data (omitting binary images) Use all Google Data (from 30 to 100 images per color) Increase cluster size in GMM from 10 to 128

SVM

LINEAR SVM

First tried it with libsvm code MATLAB Function: svmtrain (Training, Group) Training: Data to be processed (transpose

matrix) Group: Specifies +1 or -1 data Use SVM for each color (black, blue, brown,

green, grey, orange, pink, purple, red, white, yellow)

Changed to fitcsvm(X,Y)

SVM TRAIN OUTPUT

FITCSVM OUTPUT

CLASSIFY DATA

MATLAB Function: svmclassify(SVMStruct,Sample)

Use SVMStruct from svmtrain (from each color)

Sample: Concatenated Ebay Fisher Vectors Changed to predict(SVMModel, X) SVMModel from fitcsvm

SVM CLASSIFY OUTPUTColumn vector with the same number of rows

as Sample. Each entry (row) in Group represents the class of the corresponding row

of Sample.

PREDICT OUTPUTReturns Label

and Score

CURRENT PROGRESS

CALCULATE PRECISION

Calculate 12 highest scores for each color, using first column only

Determine if each score is a correct match by checking indices

Calculate each color’s precision

FUTURE GOALS

Fix Binary function Try process with new data set

Data set available: Fahad Shahbaz Khan, Rao Muhammad Anwer, Joost van de Weijer, Andrew D. Bagdanov, Maria Vanrell, Antonio M. Lopez

Image retrieval test Object Detection