![Page 1: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/1.jpg)
DEEP LEARNING IN PYTHON
Introduction to deep learning
![Page 2: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/2.jpg)
Deep Learning in Python
Imagine you work for a bank● You need to predict how many transactions each customer
will make next year
![Page 3: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/3.jpg)
Deep Learning in Python
Example as seen by linear regressionAge
Bank Balance
Retirement Status
…
Number of Transactions
![Page 4: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/4.jpg)
Deep Learning in Python
Example as seen by linear regression
Predicted Transactions
Bank Balance
Retired
Not Retired
Model with no interactions
Predicted Transactions
Bank Balance
Retired
Not Retired
Model with interactions
![Page 5: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/5.jpg)
Deep Learning in Python
Interactions● Neural networks account for interactions really well
● Deep learning uses especially powerful neural networks
● Text
● Images
● Videos
● Audio
● Source code
![Page 6: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/6.jpg)
Deep Learning in Python
Course structure● First two chapters focus on conceptual knowledge
● Debug and tune deep learning models on conventional prediction problems
● Lay the foundation for progressing towards modern applications
● This will pay off in the third and fourth chapters
![Page 7: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/7.jpg)
Deep Learning in Python
Build deep learning models with kerasIn [1]: import numpy as np
In [2]: from keras.layers import Dense
In [3]: from keras.models import Sequential
In [4]: predictors = np.loadtxt('predictors_data.csv', delimiter=',')
In [5]: n_cols = predictors.shape[1]
In [6]: model = Sequential() In [7]: model.add(Dense(100, activation='relu', input_shape = (n_cols,)))
In [8]: model.add(Dense(100, activation='relu')
In [9]: model.add(Dense(1))
![Page 8: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/8.jpg)
Deep Learning in Python
Deep learning models capture interactions
Age
Bank Balance
Retirement Status
…
Number of Transactions
![Page 9: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/9.jpg)
Deep Learning in Python
Interactions in neural network
Age
Number of Transactions
Income
# Accounts
Input LayerHidden Layer
Output Layer
![Page 10: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/10.jpg)
DEEP LEARNING IN PYTHON
Let’s practice!
![Page 11: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/11.jpg)
DEEP LEARNING IN PYTHON
Forward propagation
![Page 12: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/12.jpg)
Course Title
Bank transactions example● Make predictions based on:
● Number of children
● Number of existing accounts
![Page 13: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/13.jpg)
Deep Learning in Python
Forward propagation
3
21
1
-1
1
2
-1
Input Hidden Layer
Output
5
1
9 # Transactions
# Accounts
# Children
![Page 14: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/14.jpg)
Deep Learning in Python
Forward propagation
3
21
1
-1
1
2
-1
Input Hidden Layer
Output
5
1
9
# Children
# Accounts
# Transactions
![Page 15: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/15.jpg)
Deep Learning in Python
Forward propagation
3
21
1
-1
1
2
-1
Input Hidden Layer
Output
5
1
9
# Children
# Accounts
# Transactions
![Page 16: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/16.jpg)
Deep Learning in Python
Forward propagation
3
21
1
-1
1
2
-1
Input Hidden Layer
Output
5
1
9
# Children
# Accounts
# Transactions
![Page 17: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/17.jpg)
Course Title
Forward propagation● Multiply - add process
● Dot product
● Forward propagation for one data point at a time
● Output is the prediction for that data point
![Page 18: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/18.jpg)
Deep Learning in Python
In [1]: import numpy as np
In [2]: input_data = np.array([2, 3])
In [3]: weights = {'node_0': np.array([1, 1]), ...: 'node_1': np.array([-1, 1]), ...: 'output': np.array([2, -1])}
In [4]: node_0_value = (input_data * weights['node_0']).sum()
In [5]: node_1_value = (input_data * weights['node_1']).sum()
Forward propagation code
2
-1
Input Hidden Layer Output
3 1
51 2
-11
1
![Page 19: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/19.jpg)
Deep Learning in Python
In [6]: hidden_layer_values = np.array([node_0_value, node_1_value])
In [7]: print(hidden_layer_values) [5, 1]
In [8]: output = (hidden_layer_values * weights['output']).sum()
In [9]: print(output) 9
Forward propagation code
2
-1
Input Hidden Layer Output
3 1
5
9
1 2
-11
1
![Page 20: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/20.jpg)
DEEP LEARNING IN PYTHON
Let’s practice!
![Page 21: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/21.jpg)
DEEP LEARNING IN PYTHON
Activation functions
![Page 22: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/22.jpg)
Deep Learning in Python
Linear vs Nonlinear Functions
Linear Functions Nonlinear Functions
![Page 23: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/23.jpg)
Deep Learning in Python
Activation functions● Applied to node inputs to produce node output
![Page 24: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/24.jpg)
Deep Learning in Python
Improving our neural network
3
21
1
-1
1
2
-1
Input Hidden Layer
Output
5
1
9
![Page 25: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/25.jpg)
Deep Learning in Python
Activation functions
3
21
1
-1
1
2
-1
Input Hidden Layer
Output
tanh(2+3)
9
tanh(-2+3)
![Page 26: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/26.jpg)
Deep Learning in Python
ReLU (Rectified Linear Activation)Rectifier
![Page 27: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/27.jpg)
Deep Learning in Python
Activation functionsIn [1]: import numpy as np
In [2]: input_data = np.array([-1, 2])
In [3]: weights = {'node_0': np.array([3, 3]), ...: 'node_1': np.array([1, 5]), ...: 'output': np.array([2, -1])}
In [4]: node_0_input = (input_data * weights['node_0']).sum()
In [5]: node_0_output = np.tanh(node_0_input)
In [6]: node_1_input = (input_data * weights['node_1']).sum()
In [7]: node_1_output = np.tanh(node_1_input)
In [8]: hidden_layer_outputs = np.array([node_0_output, node_1_output])
In [9]: output = (hidden_layer_output * weights['output']).sum()
In [10]: print(output) 1.2382242525694254
![Page 28: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/28.jpg)
DEEP LEARNING IN PYTHON
Let’s practice!
![Page 29: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/29.jpg)
DEEP LEARNING IN PYTHON
Deeper networks
![Page 30: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/30.jpg)
Deep Learning in Python
Multiple hidden layersAge
5
32
4
4
-5
-1
1
2
2
-3
7
Calculate with ReLU Activation Function
5
![Page 31: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/31.jpg)
Deep Learning in Python
Multiple hidden layersAge
5
32
4
4
-5
-1
1
2
2
-3
7
Calculate with ReLU Activation Function
5
![Page 32: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/32.jpg)
Deep Learning in Python
Multiple hidden layersAge
5
32
4
4
-5
-1
1
2
2
-3
7
Calculate with ReLU Activation Function
5
![Page 33: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/33.jpg)
Deep Learning in Python
Multiple hidden layersAge
5
32
4
4
-5
-1
1
2
2
-3
7
Calculate with ReLU Activation Function
5
![Page 34: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/34.jpg)
Deep Learning in Python
Multiple hidden layersAge
5
32
4
4
-5
-1
1
2
2
-3
7
Calculate with ReLU Activation Function
5
![Page 35: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/35.jpg)
Deep Learning in Python
Multiple hidden layersAge
5
32
4
Calculate with ReLU Activation Function
5
![Page 36: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/36.jpg)
Deep Learning in Python
Multiple hidden layersAge
5
32
4
Calculate with ReLU Activation Function
5
![Page 37: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/37.jpg)
Deep Learning in Python
Multiple hidden layersAge
5
32
4
Calculate with ReLU Activation Function
5
![Page 38: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/38.jpg)
Deep Learning in Python
Multiple hidden layersAge
5
32
4
Calculate with ReLU Activation Function
5
26
![Page 39: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/39.jpg)
Deep Learning in Python
Multiple hidden layersAge
5
3
Calculate with ReLU Activation Function
5
26
4
-5
![Page 40: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/40.jpg)
Deep Learning in Python
Multiple hidden layersAge
5
3
Calculate with ReLU Activation Function
5
26
4
-5
![Page 41: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/41.jpg)
Deep Learning in Python
Multiple hidden layersAge
5
3
Calculate with ReLU Activation Function
5
26
4
-5
![Page 42: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/42.jpg)
Deep Learning in Python
Multiple hidden layersAge
5
3
Calculate with ReLU Activation Function
5
26
4
-50
2
4
-1
1
2
2
7
-3
![Page 43: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/43.jpg)
Deep Learning in Python
Multiple hidden layersAge
5
3
Calculate with ReLU Activation Function
5
26
4
-50
2
4
-1
1
2
2
7
-30
52
364
![Page 44: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/44.jpg)
Deep Learning in Python
Representation learning● Deep networks internally build representations of pa"erns in
the data
● Partially replace the need for feature engineering
● Subsequent layers build increasingly sophisticated representations of raw data
![Page 45: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/45.jpg)
Deep Learning in Python
Representation learning
![Page 46: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/46.jpg)
Deep Learning in Python
Deep learning● Modeler doesn’t need to specify the interactions
● When you train the model, the neural network gets weights that find the relevant pa"erns to make be"er predictions
![Page 47: DEEP LEARNING IN PYTHON - Amazon S3 · Deep Learning in Python Example as seen by linear regression Predicted Transactions Bank Balance Retired Not Retired Model with no interactions](https://reader030.vdocument.in/reader030/viewer/2022012319/5b8914eb7f8b9a655f8b5f6f/html5/thumbnails/47.jpg)
DEEP LEARNING IN PYTHON
Let’s practice!