efficient neural network architecture for image classfication
TRANSCRIPT
![Page 1: Efficient Neural Network Architecture for Image Classfication](https://reader036.vdocument.in/reader036/viewer/2022081513/587354c61a28ab56378b711d/html5/thumbnails/1.jpg)
Efficient Convolutional Neural Network Architecture
for Image Classification
Yogendra TamangMSCS-070-670
Supervisor:Prof. Dr. Sashidhar Ram Joshi
Presented By:
![Page 2: Efficient Neural Network Architecture for Image Classfication](https://reader036.vdocument.in/reader036/viewer/2022081513/587354c61a28ab56378b711d/html5/thumbnails/2.jpg)
Outline• Background• Convolutional Neural Network• Objectives•Methodology•Work Accomplished•Work Remaining• References
![Page 3: Efficient Neural Network Architecture for Image Classfication](https://reader036.vdocument.in/reader036/viewer/2022081513/587354c61a28ab56378b711d/html5/thumbnails/3.jpg)
Background• Learning• Supervised• Unsupervised
• AI Tasks• Classification and Regression• Clustering
Machine Learning Problem
Supervised
RegressionClassfication
Unsupervised
Clustering
![Page 4: Efficient Neural Network Architecture for Image Classfication](https://reader036.vdocument.in/reader036/viewer/2022081513/587354c61a28ab56378b711d/html5/thumbnails/4.jpg)
Background•Classification• Classifies data into one of discrete classes• Eg. Classifying digits• Cost Function for Classification Task may be Logistic Regression or Log-
likelihood
• Regression• Predicts continuous real valued output• Eg. Stock Price Predictions• Cost function for regression type problem are MSE(Mean Squared Error)
![Page 5: Efficient Neural Network Architecture for Image Classfication](https://reader036.vdocument.in/reader036/viewer/2022081513/587354c61a28ab56378b711d/html5/thumbnails/5.jpg)
Multi Layerd Perceptrons (MLPs)
Input Layer Hidden LayerOutput Layer
![Page 6: Efficient Neural Network Architecture for Image Classfication](https://reader036.vdocument.in/reader036/viewer/2022081513/587354c61a28ab56378b711d/html5/thumbnails/6.jpg)
Convolutional Neural Networks•One or more convolutional layer• Followed by one or more fully connected layer•Resulting in easy to train networks with many fewer
parameters.
![Page 7: Efficient Neural Network Architecture for Image Classfication](https://reader036.vdocument.in/reader036/viewer/2022081513/587354c61a28ab56378b711d/html5/thumbnails/7.jpg)
Objectives• To classify images using CNN• To design effective architecture of CNN for image classification task.
![Page 8: Efficient Neural Network Architecture for Image Classfication](https://reader036.vdocument.in/reader036/viewer/2022081513/587354c61a28ab56378b711d/html5/thumbnails/8.jpg)
Convolutional Neural Networks
• Receptive fields(RFs)• Apply filter to image.• Pooling and
subsampling layers
![Page 9: Efficient Neural Network Architecture for Image Classfication](https://reader036.vdocument.in/reader036/viewer/2022081513/587354c61a28ab56378b711d/html5/thumbnails/9.jpg)
Convolution Neural Network
![Page 10: Efficient Neural Network Architecture for Image Classfication](https://reader036.vdocument.in/reader036/viewer/2022081513/587354c61a28ab56378b711d/html5/thumbnails/10.jpg)
MethodologyTraining Set Validation
Set
Testing Set
![Page 11: Efficient Neural Network Architecture for Image Classfication](https://reader036.vdocument.in/reader036/viewer/2022081513/587354c61a28ab56378b711d/html5/thumbnails/11.jpg)
Methodology• Convolution Layer Design
![Page 12: Efficient Neural Network Architecture for Image Classfication](https://reader036.vdocument.in/reader036/viewer/2022081513/587354c61a28ab56378b711d/html5/thumbnails/12.jpg)
Methodology• Pooling Layer Design
![Page 13: Efficient Neural Network Architecture for Image Classfication](https://reader036.vdocument.in/reader036/viewer/2022081513/587354c61a28ab56378b711d/html5/thumbnails/13.jpg)
MethodologyExample CNN Architecture
Learning a Classifier• Gradient Descent Algorithm• Calculate Cost Function or Lost Function J(s)• Calculate Gradient • Update Weights
• Stochastic Gradient Descent: Updates Adjust after example.• Minibatch SGD: Updates after batches.
![Page 14: Efficient Neural Network Architecture for Image Classfication](https://reader036.vdocument.in/reader036/viewer/2022081513/587354c61a28ab56378b711d/html5/thumbnails/14.jpg)
Learning a Classifier- Negative Log likelihood
𝑁𝐿𝐿 (𝜃 ,𝒟 )=− ∑𝑖=0
¿ 𝒟∨¿log 𝑃(𝑌=𝑦 ( 𝑖)∨𝑥 ( 𝑖) ,𝜃 )¿
¿Where is Dataset is weight parameter is ith training data. Y is target data.
![Page 15: Efficient Neural Network Architecture for Image Classfication](https://reader036.vdocument.in/reader036/viewer/2022081513/587354c61a28ab56378b711d/html5/thumbnails/15.jpg)
Work Accompolished1. GPU Configuration to support CUDA.
![Page 16: Efficient Neural Network Architecture for Image Classfication](https://reader036.vdocument.in/reader036/viewer/2022081513/587354c61a28ab56378b711d/html5/thumbnails/16.jpg)
2. CNN Architecture for CIFAR-10 dataset
![Page 17: Efficient Neural Network Architecture for Image Classfication](https://reader036.vdocument.in/reader036/viewer/2022081513/587354c61a28ab56378b711d/html5/thumbnails/17.jpg)
3. CNN Architecture for MNIST-10 datasetINPUT-> CONV ->MAXPOOL-> CONV -> MAXPOOL-> FULL -> OUTPUT
![Page 18: Efficient Neural Network Architecture for Image Classfication](https://reader036.vdocument.in/reader036/viewer/2022081513/587354c61a28ab56378b711d/html5/thumbnails/18.jpg)
MNIST Dataset Training and Output
![Page 19: Efficient Neural Network Architecture for Image Classfication](https://reader036.vdocument.in/reader036/viewer/2022081513/587354c61a28ab56378b711d/html5/thumbnails/19.jpg)
Training Loss, Validation Loss, Validation Accuracy on MNIST Dataset
1 2 3 4 5 6 7 8 9 100
0.2
0.4
0.6
0.8
1
1.2
CNN running over mnist dataset
Training LossValidation lossValidation accuracy
Epochs
Trai
ning
Loss
/Val
idati
on
Loss
/Val
idati
on A
ccur
acy
![Page 20: Efficient Neural Network Architecture for Image Classfication](https://reader036.vdocument.in/reader036/viewer/2022081513/587354c61a28ab56378b711d/html5/thumbnails/20.jpg)
Work Remaining• Dropout Implementation• Parameter Changing
![Page 21: Efficient Neural Network Architecture for Image Classfication](https://reader036.vdocument.in/reader036/viewer/2022081513/587354c61a28ab56378b711d/html5/thumbnails/21.jpg)
Time Schedule
![Page 22: Efficient Neural Network Architecture for Image Classfication](https://reader036.vdocument.in/reader036/viewer/2022081513/587354c61a28ab56378b711d/html5/thumbnails/22.jpg)
References[1] A. D. J. J. J. B. Eugenio Culurciello, “An Analysis of the Connections Between Layers of Deep Neural Networks,” arXiv, 2013.[2] B. K. A.-r. M. B. R. Tara N. Sainath, “Learning Filter Banks within a Deep Neural Network Framework,”
in IEEE, 2013. [3] A.-r. M. G. H. Alex Graves, “Speech Recognition with Deep Recurrent Neural Networks,” University of
Toronto.[4] A. Graves, “Generating Sequences with Recurrent Neural Networks,” arXiv, 2014.[5] Q. V. Oriol Vinyals, “A Neural Conversational Model,” arXiv, 2015.[6] J. D. T. D. J. M. Ross Grishick, “Rich Features Hierarchies for accurate object detection and semantic
segmentation.,” UC Berkeley.[7] A. Karpathy, “CS231n Convolutional Neural Networks for Visual Recognition,” Stanford University, [Online]. Available: http://cs231n.github.io/convolutional-networks/.[8] I. Sutskever, “Training Recurrent Neural Networks,” University of Toronto, 2013.[9] “Convolutional Neural Networks (LeNet),” [Online]. Available: http://deeplearning.net/tutorial/lenet.html.[10] I. S. E. H. Alex Krizhevsky, “ImageNet Classification with Deep Convolutional Neural Networks,” 2012.
![Page 23: Efficient Neural Network Architecture for Image Classfication](https://reader036.vdocument.in/reader036/viewer/2022081513/587354c61a28ab56378b711d/html5/thumbnails/23.jpg)
References[11] R. F. Matthew D Zeiler, “Visualizing and Understanding Convolutional Networks,” arXiv, 2013.[12] A. K. a. L. Fie-Fie, “Deep Visual Alignment for Generating Image Descriptions,” Standford University, 2014.[13] A. T. S. B. D. E. O. Vinyals, “Show and Tell: A Neural Image Caption Generator.,” Google Inc., 2014.[14] J. M. G. H. IIya Sutskever, “Generating Text with Recurrent Neural Networks,” in 28th International Conference on Machine Learning, Bellevue, 2011. [15] M. A. Nielsen, “Neural Networks and Deep Learning,” Determination Press, 2014.[16] J. Martens, “Deep Learning via Hessian-Free Optimization,” in Procedings of 27th International Conference on Machine Learning, 2010.