computer vision for card games -...

Printing:Thisposteris48”wideby36”high.It’sdesignedtobeprintedonalargeformatprinter.

CustomizingtheContent:Theplaceholdersinthisposterareformattedforyou.Typeintheplaceholderstoaddtext,orclickanicontoaddatable,chart,SmartArtgraphic,pictureormultimediafile.Toaddorremovebulletpointsfromtext,justclicktheBulletsbuttonontheHometab.Ifyouneedmoreplaceholdersfortitles,contentorbodytext,justmakeacopyofwhatyouneedanddragitintoplace.PowerPoint’sSmartGuideswillhelpyoualignitwitheverythingelse.Wanttouseyourownpicturesinsteadofours?Noproblem!JustrightpictureandchooseChangePicture.Maintaintheproportionofpicturesasyouresizebydraggingacorner.

ComputerVisionforCardGamesMatiasCastillo,BenjaminGoeing,JesperWestell

ProjectObjective

DataAugmentationandEngineering

Wetestedtwoclassicalmachinelearningmethods,usingscikit-learn:● Multi-classOVAclassification,usinglogloss,whereweproduceda

listofclassifiersand

● Multi-classOVAclassificationwithL2Regularization,usinglambda0.1

● Multi-classSVM,usinghingeloss● Multi-classSVMwithL2Regularization,usinglambda0.1

ClassicalMachineLearningMethods

Resultsfromclassicalmethods

● For the purpose of this project, we have created our owndataset by taking 10 pictures of each playing card lying on atable

● We then wrote a script to augment every image, usingdifferent combinations of zoom, brightness, contrast, rotationetc.

● This resulted in 1,050 examples per class for a total of 54,600images, which we divided into 90% train, 5% dev, and 5% test

● As the training set was too large to fit into memory, wetrained using random mini-batches, that were sequentiallyread into memory and also decided to downsize the images

ResultsConvolutionalNeuralNet

FUTUREWORK

TrainingacustomCNNfromscratch:● For our joint CS229/CS221 project, we are developing a

program that can play real world card games against a humanplayer

● The goal of our 229 project is to develop a computer visionmethod, that can recognize playing cards lying on a tablewith 100% accuracy

● This is a 52-class classification problem

● In order to reach our goal of achieving 100% accuracy, wedecided to design and train a custom CNN using Tensorflowfrom scratch:

● Allofourcardimagesarecomingfromthesamedeckthatwepurchased● Infuturework,weplantousemorevariedcardsets(e.g.withdifferentthemes)● Inaddition,weareplanningtotrainamodelthatcanrecognizemultiplecardsatthesametime

● We managed to achieve 100.0% train, dev andtest accuracy with the CNN after 17 epochs

● We decided to use a ReLu activation function, maxpooling with window sizes 6x6 and 4x4 and strides 6and 4 respectively, as well as a “same” padding

● Filters used are [4,4,3,8] and [2,2,8,16]

● We used the cross-entropy loss as a cost function:

References● DeepLearning.ai/CS230coursematerials

● Themulti-classOVAperformsbetterthantheSVMmodel(85%vs.82%devsetaccuracy)

● Therearestillnoticeableoverfittingissueswithbothmodels● ApplyingL2regularizationimproveddevsetaccuracyinboth

models(from85%to89%inOVAandfrom82%to85%inSVM).