imagenet classification using binary convolutional neural...
TRANSCRIPT
![Page 1: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks](https://reader035.vdocument.in/reader035/viewer/2022063007/5fb9fe0d7380fe21660c3bf6/html5/thumbnails/1.jpg)
XNOR-NetImageNet Classification Using Binary
Convolutional Neural Networks
Mohammad Rastegari Vicente OrdonezJoseph Redmon
Ali Farhadi
Presentation by Naveen
![Page 2: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks](https://reader035.vdocument.in/reader035/viewer/2022063007/5fb9fe0d7380fe21660c3bf6/html5/thumbnails/2.jpg)
Deep Neural Networks are Complicated(And Huge!)
Remember HW2 - Size of AlexNet?
![Page 3: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks](https://reader035.vdocument.in/reader035/viewer/2022063007/5fb9fe0d7380fe21660c3bf6/html5/thumbnails/3.jpg)
CPU vs GPU
SmallWeak
Scrawny
BigStrong
Powerful
![Page 4: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks](https://reader035.vdocument.in/reader035/viewer/2022063007/5fb9fe0d7380fe21660c3bf6/html5/thumbnails/4.jpg)
Possible ApproachesShallow Approximation Compression
![Page 5: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks](https://reader035.vdocument.in/reader035/viewer/2022063007/5fb9fe0d7380fe21660c3bf6/html5/thumbnails/5.jpg)
Binary-Weights Network
Basic Idea: Too much information in each convolutional layer. Can we store less?
![Page 6: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks](https://reader035.vdocument.in/reader035/viewer/2022063007/5fb9fe0d7380fe21660c3bf6/html5/thumbnails/6.jpg)
I * W ~ (I W) a~
I = Input TensorW = Weightsa = scaling factor
Binary-Weights Network
![Page 7: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks](https://reader035.vdocument.in/reader035/viewer/2022063007/5fb9fe0d7380fe21660c3bf6/html5/thumbnails/7.jpg)
Training
Binarize weights in forward pass and backward
propagation
Use real valued weights in gradient descent (Why?)
Also, if we are using real valued weights somewhere, what’s the
point?!
![Page 8: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks](https://reader035.vdocument.in/reader035/viewer/2022063007/5fb9fe0d7380fe21660c3bf6/html5/thumbnails/8.jpg)
XNOR-Net
![Page 9: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks](https://reader035.vdocument.in/reader035/viewer/2022063007/5fb9fe0d7380fe21660c3bf6/html5/thumbnails/9.jpg)
Training
BinActivComputes the K and sign(I)
BinConvPerform earlier Binary
Convolution
![Page 10: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks](https://reader035.vdocument.in/reader035/viewer/2022063007/5fb9fe0d7380fe21660c3bf6/html5/thumbnails/10.jpg)
Experiments
Efficiency
58x CPU Speedups
![Page 11: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks](https://reader035.vdocument.in/reader035/viewer/2022063007/5fb9fe0d7380fe21660c3bf6/html5/thumbnails/11.jpg)
ExperimentsAccuracy
Cifar-10
Binary-Weight Network: 9.88% Error XNOR-Net: 10.17% Error
![Page 12: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks](https://reader035.vdocument.in/reader035/viewer/2022063007/5fb9fe0d7380fe21660c3bf6/html5/thumbnails/12.jpg)
ExperimentsAccuracy
![Page 13: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks](https://reader035.vdocument.in/reader035/viewer/2022063007/5fb9fe0d7380fe21660c3bf6/html5/thumbnails/13.jpg)
Questions?