Download - Deep Learning at Scale
![Page 1: Deep Learning at Scale](https://reader034.vdocument.in/reader034/viewer/2022051709/586f84681a28ab54768b4d41/html5/thumbnails/1.jpg)
Proprietary and confidential. Do not distribute.
Nervana’s Deep Learning Platform
MAKING MACHINES SMARTER.™
Hanlin Tang, PhDAlgorithms Engineer
![Page 2: Deep Learning at Scale](https://reader034.vdocument.in/reader034/viewer/2022051709/586f84681a28ab54768b4d41/html5/thumbnails/2.jpg)
Facebook DeepMask
Silver et al, 2016
The Atlantic, March 2016
“The error rate has been cut by a factor of two in all the languages, more than a factor of two in many cases. That’s mostly due to deep learning and the way we have optimized it …”
Alex Acero, Siri Senior Director, AppleArticle in Backhannel/WIRED, Aug 2016
Deep Learning
![Page 3: Deep Learning at Scale](https://reader034.vdocument.in/reader034/viewer/2022051709/586f84681a28ab54768b4d41/html5/thumbnails/3.jpg)
neon deep learning
framework
train deployexplore
nervanaengine
Fastest deep learning framework
cloudn
![Page 4: Deep Learning at Scale](https://reader034.vdocument.in/reader034/viewer/2022051709/586f84681a28ab54768b4d41/html5/thumbnails/4.jpg)
• Unprecedented computing power• 10x speedup over current Maxwell GPUs (~55 TeraOps)
• 32 GB High-Bandwidth Memory
• Six bi-directional high-bandwidth links for 3D torus interconnect• 8 chips in a box, seamlessly scale to multiple chassis
![Page 5: Deep Learning at Scale](https://reader034.vdocument.in/reader034/viewer/2022051709/586f84681a28ab54768b4d41/html5/thumbnails/5.jpg)
https://github.com/NervanaSystems/neon
![Page 6: Deep Learning at Scale](https://reader034.vdocument.in/reader034/viewer/2022051709/586f84681a28ab54768b4d41/html5/thumbnails/6.jpg)
• https://github.com/NervanaSystems/ModelZoo• Pre-trained weights and models
SegNet
Deep Speech 2
Skip-thought
Autoencoders
Deep Dream
![Page 7: Deep Learning at Scale](https://reader034.vdocument.in/reader034/viewer/2022051709/586f84681a28ab54768b4d41/html5/thumbnails/7.jpg)
Badrinarayanan et al., 2015
![Page 8: Deep Learning at Scale](https://reader034.vdocument.in/reader034/viewer/2022051709/586f84681a28ab54768b4d41/html5/thumbnails/8.jpg)
Neon (ms) Caffe (ms) Speed-upForward 101 719 7.1x
Backward 164 746 4.5xTotal 265 1455 5.5x
![Page 9: Deep Learning at Scale](https://reader034.vdocument.in/reader034/viewer/2022051709/586f84681a28ab54768b4d41/html5/thumbnails/9.jpg)
neon v1.6 + mgpu v1.6
neon v2.0Modular dataloader (aeon)Neural machine translation model
neon v3.0•Nervana Graph•Tensorflow inter-operability•Graph-enabled models•Distributed computing
![Page 10: Deep Learning at Scale](https://reader034.vdocument.in/reader034/viewer/2022051709/586f84681a28ab54768b4d41/html5/thumbnails/10.jpg)
![Page 11: Deep Learning at Scale](https://reader034.vdocument.in/reader034/viewer/2022051709/586f84681a28ab54768b4d41/html5/thumbnails/11.jpg)
“Training neural networks is a dark art.”Hyperparameters:•Number and type of units/layers•Convolution filter size•Weight Initialization•Optimization method•Learning Rate schedule
![Page 12: Deep Learning at Scale](https://reader034.vdocument.in/reader034/viewer/2022051709/586f84681a28ab54768b4d41/html5/thumbnails/12.jpg)
![Page 13: Deep Learning at Scale](https://reader034.vdocument.in/reader034/viewer/2022051709/586f84681a28ab54768b4d41/html5/thumbnails/13.jpg)
Command Line client Web Interface
![Page 14: Deep Learning at Scale](https://reader034.vdocument.in/reader034/viewer/2022051709/586f84681a28ab54768b4d41/html5/thumbnails/14.jpg)
Nervana in actionHealthcare: Tumor detection
Automotive: Speech interfacesFinance: Time-series search engine
Positive:
Negative:
Agricultural Robotics Oil & Gas
Positive:
Negative:
Proteomics: Sequence analysis
Query:
Results:
![Page 15: Deep Learning at Scale](https://reader034.vdocument.in/reader034/viewer/2022051709/586f84681a28ab54768b4d41/html5/thumbnails/15.jpg)
+ n