deep learning software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · deep...
TRANSCRIPT
![Page 1: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/1.jpg)
Deep Learning SoftwareSpring 2020
![Page 2: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/2.jpg)
Today● Homework 1 is out, due Feb 6● Conda and Jupyter Notebook● Deep Learning Software
● Keras● Tensorflow● Numpy
● Google Cloud Platform
![Page 3: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/3.jpg)
Conda● A package management system for Python
![Page 4: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/4.jpg)
Jupyter notebook● A web application that
where you can code, interact, record and plot.
● Allow for remote interaction when you are working on the cloud
● You will be using it for HW1
![Page 5: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/5.jpg)
Deep Learning Software
![Page 6: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/6.jpg)
Deep Learning SoftwareCaffe(UCB) Caffe2(Facebook)
Torch(NYU/Facebook) PyTorch(Facebook)
Theano(U Montreal) TensorFlow(Google)
Keras (High Level Wrapper)
Paddle (Baidu)
CNTK(Microsoft)
MXNet(Amazon)
![Page 7: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/7.jpg)
Deep Learning Software: Most PopularCaffe(UCB) Caffe2(Facebook)
Torch(NYU/Facebook) PyTorch(Facebook)
Theano(U Montreal) TensorFlow(Google)
Keras (High Level Wrapper)
Paddle (Baidu)
CNTK(Microsoft)
MXNet(Amazon)
![Page 8: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/8.jpg)
Deep Learning Software: TodayCaffe(UCB) Caffe2(Facebook)
Torch(NYU/Facebook) PyTorch(Facebook)
Theano(U Montreal) TensorFlow(Google)
Keras (High Level Wrapper)
Paddle (Baidu)
CNTK(Microsoft)
MXNet(Amazon)
![Page 9: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/9.jpg)
Mobile Platform ● Tensorflow Lite:
○ Released last November
![Page 10: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/10.jpg)
Why do we use deep learning frameworks?● Easily build big computational graphs
○ Not the case in HW1
● Easily compute gradients in computational graphs● GPU support (cuDNN, cuBLA...etc)
○ Not required in HW1
![Page 11: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/11.jpg)
Keras● A high-level deep learning framework● Built on other deep-learning frameworks
○ Theano○ Tensorflow○ CNTK
● Easy and Fun!
![Page 12: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/12.jpg)
Keras: A High-level Wrapper● Pass on a layer of instances in the constructor
● Or: simply add layers. Make sure the dimensions match.
![Page 13: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/13.jpg)
Keras: Compile and train!
Epoch: 1 epoch means going through all the training dataset once
![Page 14: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/14.jpg)
Numpy● The fundamental package in Python for:
○ Scientific Computing○ Data Science
● Think in terms of vectors/Matrices○ Refrain from using for loops!○ Similar to Matlab
![Page 15: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/15.jpg)
Numpy● Basic vector operations
○ Sum, mean, argmax….
● Linear Algebra operations
![Page 16: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/16.jpg)
Numpy● Indexing, Slicing, Iterating
![Page 17: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/17.jpg)
Numpy ● Broadcasting
![Page 18: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/18.jpg)
Numpy Example● Find the nearest value from a given value in an array
![Page 19: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/19.jpg)
Computational Graphs● f(x,y,z) = sum(x*y + z)● x,y,z can be scalars, vectors, matrices,
tensors.
![Page 20: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/20.jpg)
Computational Graphs- Numpy
![Page 21: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/21.jpg)
Tensorflow
Define Variables
Define New Variables(Gradients)
Define Functions
![Page 22: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/22.jpg)
Comparison
Framework Pros/Cons
Theano Development completed, No development in progress, Static
Tensorflow Actively developed, big community, Static
PyTorch Better for Research, relatively new, Dynamic
Keras High-level, Easy, Not flexible
![Page 23: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/23.jpg)
Resources● Great Documentation on all of the DL software● Deeplearning.ai● State-of-art result for machine learning problems
○ https://github.com/RedditSota/state-of-the-art-result-for-machine-learning-problems
![Page 24: Deep Learning Software - archive.ece.cmu.eduece739/lectures/18739-2020-spring-lecture-04... · Deep Learning Software ... Tensorflow Actively developed, big community, Static PyTorch](https://reader034.vdocument.in/reader034/viewer/2022042312/5edad8ae09ac2c67fa686706/html5/thumbnails/24.jpg)
Acknowledgment
● Based on material from○ Spring 2019 Course