![Page 1: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/1.jpg)
1© 2015 The MathWorks, Inc.
Demystifying Deep Learning“Let the computers do the hard work”
Jérémy Huard
![Page 2: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/2.jpg)
2
![Page 3: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/3.jpg)
3
Why MATLAB for Deep Learning?
▪ MATLAB is Productive
▪ MATLAB is Fast
▪ MATLAB Integrates with Open Source
![Page 4: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/4.jpg)
4
What is Deep Learning?
![Page 5: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/5.jpg)
5
Deep Learning
Model learns to perform tasks directly from data.
x1000
x1000
x1000
x1000
x1000
Deep
Learning
Model
Image
Classifier
![Page 6: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/6.jpg)
6
Data Types for Deep Learning
Signal ImageText
![Page 7: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/7.jpg)
7
Deep Learning is Versatile
Iris Recognition – 99.4% accuracy2
Rain Detection and Removal1Detection of cars and road in autonomous driving systems
1. Deep Joint Rain Detection and Removal from a Single Image" Wenhan Yang,
Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan
2. Source: An experimental study of deep convolutional features for iris recognition
Signal Processing in Medicine and Biology Symposium (SPMB), 2016 IEEE
Shervin Minaee ; Amirali Abdolrashidiy ; Yao Wang; An experimental study of
deep convolutional features for iris recognition
![Page 8: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/8.jpg)
8
How is deep learning performing so well?
![Page 9: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/9.jpg)
9
Deep Learning Uses a Neural Network Architecture
Input
Layer Hidden Layers (n)
Output
Layer
![Page 10: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/10.jpg)
10
Deep Learning in 6 Lines of MATLAB Code
![Page 11: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/11.jpg)
11
Why MATLAB for Deep Learning?
▪ MATLAB is Productive
▪ MATLAB is Fast
▪ MATLAB integrates with Open Source
![Page 12: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/12.jpg)
12
“I love to label and
preprocess my data”
~ Said no engineer, ever.
![Page 13: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/13.jpg)
13
Caterpillar Case Study
▪ World’s leading manufacturer of
construction and mining
equipment.
▪ Similarity between these
projects?
– Autonomous haul trucks
– Pedestrian detection
– Equipment classification
– Terrain mapping
![Page 14: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/14.jpg)
14
Computer Must Learn from Lots of Data
▪ ALL data must first be labeled to create these autonomous systems.
“We were spending way too much time ground-truthing [the data]”
--Larry Mianzo, Caterpillar
![Page 15: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/15.jpg)
15
How Did Caterpillar Do with Our Tools?
▪ Semi-automated labeling process– “We go from having to label 100 percent of our data to only having to
label about 80 to 90 percent”
▪ Used MATLAB for entire development workflow.
– “Because everything is in MATLAB, development time is short”
![Page 16: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/16.jpg)
16
How Does MATLAB Come into Play?
![Page 17: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/17.jpg)
17
![Page 18: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/18.jpg)
18
Labeling Videos: Ground Truth Labeler App
![Page 19: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/19.jpg)
19
Labeling pixels
![Page 20: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/20.jpg)
20
![Page 21: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/21.jpg)
21
MATLAB is Productive
▪ Image Labeler App semi-automates labeling workflow
▪ Improve automatic labeling by updating algorithm as you
label more images correctly.
▪ Easy to load metadata even when labeling manually
![Page 22: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/22.jpg)
22
MATLAB is Fast
Performance
Training Deployment
![Page 23: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/23.jpg)
23
Speech Recognition Example
Audio signal → Spectrogram → Image Classification algorithm
Time Time
Am
plit
ude
Fre
quency
![Page 24: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/24.jpg)
24
Another Network for Signals - LSTM
▪ LSTM = Long Short Term Memory (Networks)
– Signal, text, time-series data
– Use previous data to predict new information
▪ I live in France. I speak ___________.
c0 C1 Ct
![Page 25: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/25.jpg)
25
1. Create Datastore
▪ Datastore creates
reference for data
▪ Do not have to load in
all objects into memory
![Page 26: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/26.jpg)
26
2. Compute Speech Spectrograms
Am
plit
ude
Fre
quency
Time
![Page 27: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/27.jpg)
27
3. Split datastores
Training Validation Test
70% 15% 15%
• Trains the model
• Computer “learns”
from this data
• Checks accuracy
of model during
training
• Tests model accuracy
• Not used until validation
accuracy is good
![Page 28: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/28.jpg)
28
4. Define Architecture and Parameters
Neural Network Architecture
Training Parameters
![Page 29: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/29.jpg)
29
4. How to choose the right structure to start from?
Pretrained Models for Transfer learning
▪ Access pretrained models from top
researchers with a single line of code
Import Models from OSS Frameworks
▪ Caffe Model Importer
▪ TensorFlow/Keras Model Importer
net = alexnet;
net = vgg16;
net = inceptionv3;
net = googlenet;
…
Start from scratch
▪ Verify structure with Network Analyzer App
![Page 30: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/30.jpg)
30
5. Train Network
![Page 31: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/31.jpg)
31
Deep Learning on CPU, GPU, Multi-GPU and Clusters
Single CPU
Single CPUSingle GPU
HOW TO TARGET?
Single CPU, Multiple GPUs
On-prem server with GPUs
Cloud GPUs(AWS)
![Page 32: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/32.jpg)
32
Training Performance
TensorFlow
MATLAB
MXNet
Batch size 32
Seconds / Epoch Lower is better
![Page 33: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/33.jpg)
33
MATLAB is Fast for Deployment
▪ Target a GPU for optimal
performance
▪ NVIDIA GPUs use CUDA
code
▪ We only have MATLAB code.
Can we translate this?
![Page 34: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/34.jpg)
34
GPU Coder
▪ Automatically generates CUDA Code from MATLAB Code
– can be used on NVIDIA GPUs
▪ CUDA extends C/C++ code with constructs for parallel computing
![Page 35: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/35.jpg)
35
GPU Coder Performance
![Page 36: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/36.jpg)
36
Why MATLAB?
▪ MATLAB is Productive
▪ MATLAB is Fast
▪ MATLAB Integrates with Open Source
![Page 37: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/37.jpg)
37
Used MATLAB and Open Source Together
1. Deep Joint Rain Detection and Removal from a Single
Image" Wenhan Yang, Robby T. Tan, Jiashi Feng,
Jiaying Liu, Zongming Guo, and Shuicheng Yan
▪ Used Caffe and MATLAB
together
▪ Achieved significantly
better results than an
engineered rain model.
▪ Use our tools where it
makes your workflow
easier!
![Page 38: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/38.jpg)
38
MATLAB Integrates with Open Source Frameworks
▪ Access to many pretrained models through add-ons
▪ Users wanted to import latest models
▪ Import models directly from Tensorflow or Caffe
– Allows for improved collaboration
![Page 39: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/39.jpg)
39
Keras-Tensorflow Importer
![Page 40: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/40.jpg)
40
MATLAB Integrates with Open Source
Frameworks
▪ MATLAB supports entire deep learning workflow
– Use when it is convenient for your workflow
▪ Access to latest models
▪ Improved collaboration with other users
![Page 41: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/41.jpg)
41
Why MATLAB for Deep Learning?
▪ MATLAB is Productive
▪ MATLAB is Fast
▪ MATLAB Integrates with Open Source
![Page 42: “Let the computers do the hard work” · Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan 2. Source: An experimental study of deep convolutional features](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea26aa9554bc750d1030c13/html5/thumbnails/42.jpg)
42