presentation2 - copy · microsoft powerpoint - presentation2 - copy.pptx author: morteza created...
TRANSCRIPT
![Page 1: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/1.jpg)
Convolutional and Recurrent Neural Networks
Part 2
Morteza ChehreghaniChalmers University of Technology
![Page 2: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/2.jpg)
References
• The slides have been prepared based on• Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, Joel S. Emer, Efficient Processing of Deep Neural
Networks: A Tutorial and Survey. Proceedings of the IEEE 105(12): 2295-2329 (2017)and the slides at: http://www.rle.mit.edu/eems/wp-content/uploads/2017/06/ISCA-2017-
Hardware-Architectures-for-DNN-Tutorial.pdf
• https://github.com/hunkim/PyTorchZeroToAll [for PyTorch]
![Page 3: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/3.jpg)
The first DNN application
• Image Classification Task:• 1.2M training images• 1000 classes
• Object Detection Task:• 456k training images• 200 classes
![Page 4: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/4.jpg)
The first DNN application
![Page 5: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/5.jpg)
The first DNN application
![Page 6: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/6.jpg)
LeNet-5
![Page 7: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/7.jpg)
AlexNet
![Page 8: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/8.jpg)
AlexNet Convolutional Layer Configurations
![Page 9: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/9.jpg)
VGG-16CONV Layers: 13Fully Connected Layers: 3Weights: 138MMACs: 15.5G Also, 19 layer version
Image Source: http://www.cs.toronto.edu/~frossard/post/vgg16/
![Page 10: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/10.jpg)
GoogLeNet (v1)
[Szegedy et al., arXiv 2014, CVPR 2015]
CONV Layers: 21 (depth), 57 (total)Fully Connected Layers: 1Weights: 7.0MMACs: 1.43G
Also, v2, v3 and v4ILSVRC14 Winner
![Page 11: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/11.jpg)
GoogLeNet (v1)
CONV Layers: 21 (depth), 57 (total)Fully Connected Layers: 1Weights: 7.0MMACs: 1.43G
Also, v2, v3 and v4ILSVRC14 Winner
![Page 12: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/12.jpg)
ResNet-50CONV Layers: 49Fully Connected Layers: 1Weights: 25.5MMACs: 3.9G
Also, 34,152 and 1202 layer versionsILSVRC15 Winner
[He et al., arXiv 2015, CVPR 2016]
![Page 13: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/13.jpg)
Revolution of Depth
http://icml.cc/2016/tutorials/icml2016_tutorial_deep_residual_networks_kaiminghe.pdf
![Page 14: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/14.jpg)
Summary of Popular DNNs
![Page 15: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/15.jpg)
Image Classification Datasets
Image Classification/Recognition– Given an entire image -> Select 1 of N classes– No localization (detection)
Image Source: Stanford cs231n
![Page 16: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/16.jpg)
MNIST
Digit Classification28x28 pixels (B&W)10 Classes60,000 Training10,000 Testing
LeNet in 1998(0.95% error)
ICML 2013(0.21% error)
http://yann.lecun.com/exdb/mnist/
![Page 17: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/17.jpg)
CIFAR-10/CIFAR-100
Image Source: http://karpathy.github.io/
Object Classification
32x32 pixels (color)10 or 100 Classes50,000 Training10,000 Testing
![Page 18: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/18.jpg)
ImageNet
Object Classification
~256x256 pixels (color)1000 Classes1.3M Training100,000 Testing (50,000 Validation)
http://www.image-net.org/challenges/LSVRC/
![Page 19: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/19.jpg)
ImageNet
Fine grained Classes(120 breeds)
Winner 2012Top-5 Error (16.42% error)
Winner 2016Top-5 Error (2.99% error)
![Page 20: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/20.jpg)
Image Classification Datasets - Summary
http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html
![Page 21: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/21.jpg)
Next Tasks: Localization and Detection
[Russakovsky et al., IJCV, 2015]
![Page 22: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/22.jpg)
Other Popular Datasets
Pascal VOC– 11k images– Object Detection– 20 classeshttp://host.robots.ox.ac.uk/pascal/VOC/
• MS COCO– 300k images– Detection, Segmentation– Recognition in contexthttp://mscoco.org/
![Page 23: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/23.jpg)
Recently Introduced Datasets
• Google Open Images (~9M images)• https://github.com/openimages/dataset
• Youtube-8M (8M videos)• https://research.google.com/youtube8m/
• AudioSet (2M sound clips)• https://research.google.com/audioset/index.html
![Page 24: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/24.jpg)
Summary of Deep Learning for Images
Image Classification Object Localization Object Detection
• Image Segmentation• Action Recognition• Image Generation
![Page 25: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/25.jpg)
Deep Learning for Speech Speech Recognition Natural Language Processing Speech Translation Audio Generation
![Page 26: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/26.jpg)
Deep Learning on Games
Google DeepMind AlphaGo
![Page 27: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/27.jpg)
Medical Applications of Deep Learning
Brain Cancer Detection Image Source: [Jermyn et al., JBO 2016]
![Page 28: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/28.jpg)
Deep Learning for Self-driving Cars
![Page 29: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/29.jpg)
Mature Applications
• Image• Classification: image to object class• Recognition: same as classification (except for faces)• Detection: assigning bounding boxes to objects• Segmentation: assigning object class to every pixel
• Speech & Language• Speech Recognition: audio to text• Translation• Natural Language Processing: text to meaning• Audio Generation: text to audio
• Games
![Page 30: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/30.jpg)
Emerging Applications
• Medical (Cancer Detection, Pre-Natal)• Finance (Trading, Energy Forecasting, Risk)• Infrastructure (Structure Safety and Traffic)• Weather Forecasting and Event Detection
http://www.nextplatform.com/2016/09/14/next-wave-deep-learning-applications/
![Page 31: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/31.jpg)
Opportunities
• $500B Market over 10 Years!
![Page 32: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/32.jpg)
Frameworks
![Page 33: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/33.jpg)
Benefits of Frameworks
• Rapid development• Sharing models• Workload profiling• Network hardware co-design
![Page 34: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/34.jpg)
• PyTorch is a python package that provides two high-level features:• Tensor computation (like numpy) with strong GPU acceleration• Deep Neural Networks built on a tape-based autograd system
• http://pytorch.org/about/
• To learn basics: https://github.com/hunkim/PyTorchZeroToAll
![Page 35: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/35.jpg)
Why
• More Pythonic (imperative)• Flexible• Intuitive and cleaner code• Easy to debug
• More Neural Networkic• Write code as the network works• forward/backward
![Page 36: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/36.jpg)
STEPS
![Page 37: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/37.jpg)
![Page 38: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/38.jpg)
![Page 39: Presentation2 - Copy · Microsoft PowerPoint - Presentation2 - Copy.pptx Author: morteza Created Date: 4/24/2018 4:06:19 PM](https://reader034.vdocument.in/reader034/viewer/2022042319/5f083b147e708231d420fb95/html5/thumbnails/39.jpg)
s