![Page 1: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/1.jpg)
1© 2019 The MathWorks, Inc.
Introduction to Deep Learning in Signal Processing &
Communications with MATLAB
Dr. Amod Anandkumar
Pallavi Kar
Application Engineering Group, Mathworks India
![Page 2: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/2.jpg)
2
Different Types of Machine Learning
Machine
Learning
Supervised
Learning
Classification
Regression
Unsupervised
LearningClustering
Discover an internal representation from
input data only
Develop predictivemodel based on bothinput and output data
Type of Machine Learning Categories of Algorithms
• No output - find natural groups and
patterns from input data only
• Output is a real number
(temperature, stock prices)
• Output is a choice between classes
(True, False) (Red, Blue, Green)
![Page 3: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/3.jpg)
3
What is Deep Learning?
![Page 4: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/4.jpg)
4
Deep learning is a type of supervised machine learning in which a
model learns to perform classification tasks directly from images, text,
or sound.
Deep learning is usually implemented using a neural network.
The term “deep” refers to the number of layers in the network—the
more layers, the deeper the network.
![Page 5: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/5.jpg)
5
Why is Deep Learning So Popular Now?
Source: ILSVRC Top-5 Error on ImageNet
Human
Accuracy
![Page 6: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/6.jpg)
6
Vision applications have been driving the progress in deep learning
producing surprisingly accurate systems
![Page 7: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/7.jpg)
7
Deep Learning success enabled by:
• Labeled public datasets
• Progress in GPU for acceleration
• World-class models and
connected community
AlexNetPRETRAINED
MODEL
CaffeI M P O R T E R
ResNet-50PRETRAINED MODEL
TensorFlow-
KerasI M P O R T E R
VGG-16PRETRAINED
MODEL
GoogLeNet PRETRAINED
MODEL
ONNX ConverterMODEL CONVERTER
Inception-v3M O D E L S
![Page 8: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/8.jpg)
8
Deep Learning is Versatile MATLAB Examples Available Here
![Page 9: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/9.jpg)
9
Directed Acyclic Graph Network
ResNet
R-CNN
Many Network Architectures for Deep Learning
Series Network
AlexNet
YOLO
Recurrent Network
LSTM
and more
(GAN, DQN,…)
![Page 10: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/10.jpg)
10
Convolutional Neural Networks
ObjectsShapes
Edges
Convolution layers Fully-connected
layers
A lot of data
is required
![Page 11: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/11.jpg)
11
Deep Learning Inference in 4 Lines of Code
• >> net = alexnet;
• >> I = imread('peacock.jpg')
• >> I1 = imresize(I,[227 227]);
• >> classify(net,I1)
• ans =
• categorical
• peacock
![Page 12: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/12.jpg)
12
Understanding network behavior using visualizations
▪ Custom visualizations
– Example: Class Activation Maps
(See blog post)
Filters…
Activations
Deep Dream
![Page 13: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/13.jpg)
13
Visualization Technique – Deep Dream
deepDreamImage(...
net, ‘fc8', channel,
'NumIterations', 50, ...
'PyramidLevels', 4,...
'PyramidScale', 1.25);
Synthesizes images that strongly activate
a channel in a particular layer
Example Available Here
![Page 14: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/14.jpg)
14
What is Training?
Labels
Network
Training
Images
Trained Deep
Neural Network
During training, neural network architectures learn features directly
from the data without the need for manual feature extraction
Large Data Set
![Page 15: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/15.jpg)
15
What Happens During Training?AlexNet Example
Layer weights are learned
during training
Training Data Labels
![Page 16: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/16.jpg)
16
Visualize Network Weights During TrainingAlexNet Example
Training Data Labels
First Convolution
Layer
![Page 17: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/17.jpg)
17
Visualize Features Learned During TrainingAlexNet Example
Sample Training Data Features Learned by Network
![Page 18: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/18.jpg)
18
Visualize Features Learned During TrainingAlexNet Example
Sample Training Data Features Learned by Network
![Page 19: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/19.jpg)
19
Deep Learning Workflow
Files
Databases
Sensors
ACCESS AND EXPLORE
DATA
DEVELOP PREDICTIVE
MODELS
Hardware-Accelerated
Training
Hyperparameter Tuning
Network Visualization
LABEL AND PREPROCESS
DATA
Data Augmentation/
Transformation
Labeling Automation
Import Reference
Models
INTEGRATE MODELS WITH
SYSTEMS
Desktop Apps
Enterprise Scale Systems
Embedded Devices and
Hardware
![Page 20: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/20.jpg)
20
Deep Learning Challenges
Data
▪ Handling large amounts of data
▪ Labeling thousands of signals, images & videos
▪ Transforming, generating, and augmenting data (for different domains)
Training and Testing Deep Neural Networks
▪ Accessing reference models from research
▪ Understanding network behaviour
▪ Optimizing hyperparameters
▪ Training takes hours-days
Rapid and Optimized Deployment
▪ Desktop, web, cloud, and embedded hardware
![Page 21: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/21.jpg)
21
Working with Signal Data
Signals Transform Train
Convolutional
Network (CNN)
Trained Model
Labels
Signals
Labels
Sequence
Layers
Fully Connected
Layers Trained Model
![Page 22: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/22.jpg)
22
Long Short Term Memory Networks from RNNs
▪ Recurrent Neural Network that carries a memory cell throughout the process
▪ Sequence Problems – Long term dependency does not work well with RNNs
c0 C1 Ct
![Page 23: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/23.jpg)
23
RNN to LSTM
Forget gateInput gate
Update
Output gate
Cell state
![Page 24: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/24.jpg)
24
Example: Speaker Gender Recognition Using Deep Learning
LSTM Network for Audio based Speaker Classification
Mozilla Common
Voice Dataset
audioDatastore
Time-varying
features:
- MFCC
- Pitch
- ...
− Male
− Female
− Male
− Female
Tra
inin
g D
ata
Test D
ata
1. Train
2. Predict
![Page 25: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/25.jpg)
27
Some audio and speech applications following CNN workflows
![Page 26: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/26.jpg)
28
https://www.isca-speech.org/archive/interspeech_2015/papers/i15_1478.pdf
![Page 27: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/27.jpg)
29
Solution2: Speech Command Recognition with Deep Learning(MATLAB)
Raw DataPreprocessed
DataModels and Algorithms
. . .
Integrated Analytic
Systems
![Page 28: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/28.jpg)
43
![Page 29: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/29.jpg)
44
![Page 30: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/30.jpg)
45
![Page 31: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/31.jpg)
46
![Page 32: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/32.jpg)
50
Data augmentation allows training more advanced networks and
generating more robust models
Original
DatasetSimulate:- lower voice
- varying level of environmental interference
- recording hardware effects
- distortions due to use of different microphones
- different speaker vs microphone positions or room sizes in
enclosed environments
- ...
![Page 33: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/33.jpg)
51
Se
ria
l
% Tall array and "implicit" parallel processing patternT = tall(ads);
dcOffsets = cellfun(@(x) mean(x), T,'UniformOutput',false);gather(dcOffsets);
% Partition and explicit parfor parallel processing pattern N = numpartitions(ads);
parfor index=1:Nsubads = partition(ads,N,index);while hasdata(subads)
data = read(subads);% do something with data
endend
% Cycle continuously and automatically through files in datastoremfccFile = zeros(numel(ads.Files),numMfcc)while hasdata(ads)
[data,info] = read(ads);% do something with data
end
Pa
ralle
l
![Page 34: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/34.jpg)
52
Package Speech Recognition App
![Page 35: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/35.jpg)
53
![Page 36: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/36.jpg)
54
BREAK
![Page 37: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/37.jpg)
55
Data
✓ Handling large amounts of data
▪ Labeling thousands of signals, images & videos
▪ Transforming, generating, and augmenting data (for different domains)
Training and Testing Deep Neural Networks
✓ Understanding network behavior
▪ Accessing reference models from research
▪ Optimizing hyperparameters
▪ Training takes hours-days
Rapid and Optimized Deployment
▪ Desktop, web, cloud, and embedded hardware
Deep Learning Challenges
Not a deep learning expert
![Page 38: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/38.jpg)
56
BREAK
![Page 39: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/39.jpg)
57
![Page 40: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/40.jpg)
58
Speech 2 text intro
![Page 41: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/41.jpg)
59
![Page 42: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/42.jpg)
60
![Page 43: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/43.jpg)
61
![Page 44: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/44.jpg)
62
![Page 45: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/45.jpg)
63
Audio Labeler
▪ Work on collections of
recordings or record new audio
directly within the app
▪ Navigate dataset and playback
interactively
▪ Define and apply labels to
– Entire files
– Regions within files
▪ Import and export audio folders,
label definitions and datastores
![Page 46: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/46.jpg)
64
Apps for Data Labeling
![Page 47: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/47.jpg)
65
Label Images Using Image Labeler App
![Page 48: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/48.jpg)
66
Accelerate Labeling With Automation Algorithms
Learn More
![Page 49: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/49.jpg)
67
Ground Truth
Train
Network
Labeling Apps
Images
Videos
Automation
Algorithm
Propose LabelsMore
Ground TruthImprove Network
Perform Bootstrapping to Label Large Datasets
![Page 50: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/50.jpg)
68
Example – Semantic Segmentation
▪ Classify pixels into 11 classes
– Sky, Building, Pole, Road,
Pavement, Tree, SignSymbol,
Fence, Car, Pedestrian, Bicyclist
▪ CamVid dataset Brostow, Gabriel J., Julien Fauqueur, and Roberto Cipolla. "Semantic object classes in
video: A high-definition ground truth database." Pattern Recognition Letters Vol 30, Issue
2, 2009, pp 88-97.
Available Here
![Page 51: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/51.jpg)
69
Example: Singing Voice Separation
▪ Source separation
▪ Based on U-Net architecture
![Page 52: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/52.jpg)
70
Synthetically generating labeled data
![Page 53: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/53.jpg)
71
▪ Generate synthetic modulated signals
▪ Apply channel impairments
▪ Train a CNN to classify modulation types
Modulation Classification with Deep Learning
![Page 54: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/54.jpg)
72
Deep Learning Challenges
Data
✓ Handling large amounts of data
✓ Labeling thousands of signals, images & videos
✓ Transforming, generating, and augmenting data (for different domains)
Training and Testing Deep Neural Networks
✓ Understanding network behavior
▪ Accessing reference models from research
▪ Optimizing hyperparameters
▪ Training takes hours-days
Rapid and Optimized Deployment
▪ Desktop, web, cloud, and embedded hardware
![Page 55: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/55.jpg)
73
Transfer Learning8 lines of MATLAB Code
%% Create a datastore
imds = imageDatastore(‘Data’,...
'IncludeSubfolders',true,'LabelSource','foldernames');
num_classes = numel(unique(imds.Labels));
%% Load Referece Network
net = alexnet;
layers = net.Layers
%% Replace Final Layers
layers(end-2) =
fullyConnectedLayer(num_classes,'Name',[‘fc5']);
layers(end) = classificationLayer('Name','classOut');
%% Set Training Options & Train Network
trainOpts = trainingOptions('sgdm’,...
'MiniBatchSize', 64);
net = trainNetwork(imds, layers, trainOpts);
Load Training Data
Load Pre-Trained Network
Replace Final Layers
Train Network on New Data
1
2
3
4
5
6
7
8
![Page 56: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/56.jpg)
74
Tune Hyperparameters to Improve Training
Many hyperparameters
▪ depth, layers, solver options,
learning rates, regularization,
…
Techniques
▪ Parameter sweep
▪ Bayesian optimization
![Page 57: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/57.jpg)
75
Keras-Tensorflow Importer
![Page 58: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/58.jpg)
76
Model Exchange and Co-execution
Model Exchange
MATLAB Python
Co-execution
![Page 59: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/59.jpg)
77
Model Exchange With Deep Learning Frameworks
ONNX
PyTorch
MATLABMXNet
Caffe2 TensorFlow
ChainerCore MLCognitive
Toolkit
ONNX = Open Neural Network Exchange Format
TensorFlow-
Keras
Caffe
![Page 60: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/60.jpg)
78
Interoperate With Deep Learning Frameworks – Use Cases
ONNX
PyTorch
MATLABMXNet
Caffe2 TensorFlow
Core ML
Export
Import
Deployment
Augmentation
and
Optimization
Visualization
and
Debugging
ChainerCognitive
Toolkit
ONNX = Open Neural Network Exchange Format
![Page 61: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/61.jpg)
79
Model Exchange With Deep Learning Frameworks
Caffe Model Importer
▪ importCaffeLayers
▪ importCaffeNetwork
TensorFlow-Keras Model Importer
▪ importKerasLayers
▪ importKerasNetwork
ONNX Converter
▪ importONNXNetwork
▪ exportONNXNetworkONNX Converter
![Page 62: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/62.jpg)
80
Model Exchange and Co-execution
Model Exchange
MATLAB Python
Co-execution
![Page 63: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/63.jpg)
81
MATLAB-Python Co-Execution – The ‘How’
TF_code.py
Call Python file from
MATLAB
Call TensorFlow commands from
MATLAB
![Page 64: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/64.jpg)
82
MATLAB-Python Co-Execution – Method A
TF_code.py
1. Copy the code into a .PY file
2. Wrap entry point in a function
3. Add module to Python path and then call
from MATLAB via:
py.myModule.myTFCode();
import tensorflow as tf
from tf import keras
def myTFCode():
for x in y:
a.foo()
def foo(x):
return x + 1
![Page 65: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/65.jpg)
83
Deep Learning on CPU, GPU, Multi-GPU and Clusters
Single CPU
Single CPUSingle GPU
HOW TO TARGET?
Single CPU, Multiple GPUs
On-prem server with GPUs
Cloud GPUs(AWS)
![Page 66: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/66.jpg)
84
MATLAB Containers for NVIDIA GPU Cloud & DGX
![Page 67: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/67.jpg)
85
Deep Learning Challenges
Data
✓ Handling large amounts of data
✓ Labeling thousands of signals, images & videos
✓ Transforming, generating, and augmenting data (for different domains)
Training and Testing Deep Neural Networks
✓ Accessing reference models from research
✓ Understanding network behaviour
✓ Optimizing hyperparameters
✓ Training takes hours-days
Rapid and Optimized Deployment
▪ Desktop, web, cloud, and embedded hardware
![Page 68: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/68.jpg)
86
Deploying Deep Learning Application
Embedded Hardware Desktop, Web, CloudApplication
logic
Application
DeploymentCode
Generation
![Page 69: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/69.jpg)
87
MATLAB Production Server is an application server that publishes
MATLAB code as APIs that can be called by other applications
Enterprise
Application
Mobile / Web
Application
Analytics Development
MATLABMATLAB
Compiler SDK
< >
Package Code / test
Data sources /
applications
3rd party
dashboardScale and secure
MATLAB Production Server
Request
Broker
Worker processes
Access
Integrate
Deploy
![Page 70: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/70.jpg)
88
MATLAB support for Cloud
![Page 71: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/71.jpg)
89
![Page 72: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/72.jpg)
90
IAAS to PAAS
![Page 73: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/73.jpg)
91
Automatic Cluster Creation
![Page 74: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/74.jpg)
92
Get Public IP for access
![Page 75: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/75.jpg)
93
Server Machine VM access• MPS console endpoint: https://xxx.xx.xx.xx
![Page 76: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/76.jpg)
94
Deploying Deep Learning Application
Embedded Hardware Desktop, Web, CloudApplication
logic
Application
Deployment
Code
Generation
![Page 77: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/77.jpg)
95
Solution- GPU/MATLAB Coder for Deep Learning Deployment
GPU Coder
Target Libraries
ARM
Compute
Library
Intel
MKL-DNN
Library
Application
logic
TensorRT &
cuDNN
Libraries
MATLAB
Coder
![Page 78: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/78.jpg)
96
Deep Learning Deployment Workflows
Pre-
processing
Post-
processing
codegen
Portable target code
INTEGRATED APPLICATION DEPLOYMENT
cnncodegen
Portable target code
INFERENCE ENGINE DEPLOYMENT
Trained
DNN
![Page 79: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/79.jpg)
97
Single Image Inference on Titan Xp using cuDNN
Intel® Xeon® CPU 3.6 GHz - NVIDIA libraries: CUDA9 - cuDNN 7
PyTorch (0.3.1)
MXNet (1.2.1)
GPU Coder (R2018b)
TensorFlow (1.8.0)
![Page 80: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/80.jpg)
98
NVIDIA Hardware Support Package (HSP)Simple out-of-box targeting for NVIDIA boards:
Jetson Drive platform
cnncodegen
Portable target code
INFERENCE ENGINE DEPLOYMENT
Trained
DNN
![Page 81: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/81.jpg)
99
Summary- MATLAB Deep Learning Framework
Access Data Design + Train
▪ Manage large image sets
▪ Automate image labeling
▪ Easy access to models
▪ Acceleration with GPU’s
▪ Scale to clusters
NVIDIA
TensorRT &
cuDNN
Libraries
Intel
MKL-DNN
Library
DEPLOYMENT
ARM
Compute
Library
![Page 82: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/82.jpg)
100
Summary
✓Create and validate Deep learning models
✓Automate ground truth labeling
✓Access large amount of data from cluster/cloud
✓ Interoperability with Deep learning frameworks
✓Visualization and hyperparameter tuning
✓Seamlessly scale training to GPUs, clusters and cloud
✓Deployment on embedded targets and web services
![Page 84: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/84.jpg)
102
![Page 85: Introduction to Deep Learning in Signal Processing ...ncc2019/tutorial_slides/mathworks/DL_NCC2019_final.pdf · 4 Deep learning is a type of supervised machine learning in which a](https://reader030.vdocument.in/reader030/viewer/2022040212/5e818892c9f14103e10c3e3c/html5/thumbnails/85.jpg)
103