cse847 project description

CSE 847 Project: Large Scale Image Classification 1. Introduction The objective of this project is to build large scale image classifiers. You are required to build programs that efficiently learn classification models from one million high dimensional training examples, and apply the learned classifiers to make predict for around 200,000 test examples. Although you are allowed to use the off shelf tools, you are encouraged to develop your own classification algorithms and learning programs. The course project will be evaluated in three aspects: the classification performance of your algorithms (70%), your presentation (20%) and your final report (10%). To evaluate the performance of your algorithms and programs, you are required to submit the classification results of the testing data, which will be evaluated by the instructor using the metric described in section 4. The ranking of the evaluation results will be released in the final presentation. In your presentation, you need to report the running times of your programs for training and testing, and the maximum memory used in training and testing. You may also include the special efforts you put into the course project to improve the efficiency and the accuracy of your learning programs. For instance, you can explain the strategy you used to efficiently train a classifier from a large number of training examples using only a limited amount of memory. 2. Dataset The dataset used in this project is modified from the ImageNet Large Scale Visual Recognition Challenge 2010 (ILSVRC2010). For more details of the original dataset, you can visit the ILSVRC2010 website http://www.image‐net.org/challenges/LSVRC/2010/ The dataset used in the course project consists of 1,262,106, images that are distributed over 164 classes. Some of the classes are directly from the ImageNet dataset, while the others are generated by merging multiple classes in order to make it more challenging. Each image in the dataset is represented by a vector of 900 dimensions, and is assigned to one of the 164 classes. All the features are integers. We randomly choose 1,000,000 images from the dataset to form the training set, and use the rest 262,106 images as the testing set. Furthermore, we randomly select 125,000 images from the training dataset to create a small development set, which will be used for algorithm development. For each set, the image features and the corresponding class assignments are saved in two plain text files, named as xxx.txt and xxx_label.txt, respectively. Of course, test_label.txt file is unavailable. Each line in xxx.txt is the feature vector of an image and the

Upload: ashwini-kumar-maurya

Post on 24-Sep-2015

212 views

Category:

Documents

0 download

Report

Download

Embed Size (px):

TRANSCRIPT

CSE847Project:LargeScaleImageClassification

1. IntroductionTheobjectiveofthisprojectistobuildlargescaleimageclassifiers.Youarerequiredtobuildprogramsthatefficientlylearnclassificationmodelsfromonemillionhighdimensionaltrainingexamples,andapplythelearnedclassifierstomakepredictforaround200,000testexamples.Althoughyouareallowedtousetheoffshelftools,youareencouragedtodevelopyourownclassificationalgorithmsandlearningprograms.Thecourseprojectwillbeevaluatedinthreeaspects:theclassificationperformanceofyouralgorithms(70%),yourpresentation(20%)andyourfinalreport(10%).

Toevaluatetheperformanceofyouralgorithmsandprograms,youarerequiredtosubmittheclassificationresultsofthetestingdata,whichwillbeevaluatedbytheinstructorusingthemetricdescribedinsection4.Therankingoftheevaluationresultswillbereleasedinthefinalpresentation.Inyourpresentation,youneedtoreporttherunningtimesofyourprogramsfortrainingandtesting,andthemaximummemoryusedintrainingandtesting.Youmayalsoincludethespecialeffortsyouputintothecourseprojecttoimprovetheefficiencyandtheaccuracyofyourlearningprograms.Forinstance,youcanexplainthestrategyyouusedtoefficientlytrainaclassifierfromalargenumberoftrainingexamplesusingonlyalimitedamountofmemory.

2. DatasetThedatasetusedinthisprojectismodifiedfromtheImageNetLargeScaleVisualRecognitionChallenge2010(ILSVRC2010).Formoredetailsoftheoriginaldataset,youcanvisittheILSVRC2010websitehttp://www.imagenet.org/challenges/LSVRC/2010/Thedatasetusedinthecourseprojectconsistsof1,262,106,imagesthataredistributedover164classes.SomeoftheclassesaredirectlyfromtheImageNetdataset,whiletheothersaregeneratedbymergingmultipleclassesinordertomakeitmorechallenging.Eachimageinthedatasetisrepresentedbyavectorof900dimensions,andisassignedtooneofthe164classes.Allthefeaturesareintegers.Werandomlychoose1,000,000imagesfromthedatasettoformthetrainingset,andusetherest262,106imagesasthetestingset.Furthermore,werandomlyselect125,000imagesfromthetrainingdatasettocreateasmalldevelopmentset,whichwillbeusedforalgorithmdevelopment.Foreachset,theimagefeaturesandthecorrespondingclassassignmentsaresavedintwoplaintextfiles,namedasxxx.txtandxxx_label.txt,respectively.Ofcourse,test_label.txtfileisunavailable.Eachlineinxxx.txtisthefeaturevectorofanimageandthe
valuesineachfeaturevectorareseparatedbyspaces.Thelinenumbersareusedastheindexidsforimages.Forexample,theimagewhichfeaturesareonthefirstlinehasindexid1.Eachlineinxxx_label.txtistheclasslabelforthecorrespondingimageinthexxx.txt.

Inthecourseproject,thedevelopmentsetwillbedistributedon03/14/2013.Itcanbedownloadedfromhttp://www.cse.msu.edu/~cse847/project/development.rar.Boththetrainingsetandtestingsetwillbeavailableon04/04/2013andcanbedownloadedfromhttp://www.cse.msu.edu/~cse847/project/training.rarandhttp://www.cse.msu.edu/~cse847/project/testing.rar.Youneedtosendyoupredictionresultsforthetestingsetbyemailtoyourinstructoron04/17/2011(11:59pm).

3. SubmissionsForeachclass,youneedtoreturnalistoftheindicesfor100testimages,inthedescendingorderoftheclassificationscores,i.e.,thefirstimageindexintherankinglistshouldbetheonethatismostlikelytobeassignedtotheclassandetc.Pleaseusethefollowingformatforeachlineinthesubmittedfile:

Classlabel imageindex

whereclasslabelisthelabelofthepredictedclass,variedfrom1to164;imageindexistheindexofatestimage.ThetwofieldsareseparatedbyaTab.Pleaseputthe100imageindicesofclass1,orderedbytheclassificationscores,atthetopofthefile,followedbythe100imageindicesoftheclass2andsoon.Belowisanexampleofthefile:

1 165

1 32464

164 3332

164 8476

100imageindicesforfirstclass1

100imageindicesforthelastclass164
4. Evaluationmetric:

TheMeanAveragePrecision(MAP)isusedtoevaluatetheperformance,whichiscomputedasthefollowing:

1

1

Whereisthetotalnumberofclasses(i.e.,164),isthenumberofimagesreturnedforeachclass(i.e.,100).

iscalledprecision,andisdefinedasthepercentageofthefirstktestimages,returnedbyyourprograms,thatbelongtoclassi.