![Page 1: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/1.jpg)
ARTIFICIAL NEURAL NETWORKSAN EXAMPLE OF MACHINE LEARNING ALGORITHMS
![Page 2: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/2.jpg)
TABLE OF CONTENTS - ANN
• Introduction machine learning
• Basics of mathematics behind ANNs
• Things to take into account
digitaltrends.com
![Page 3: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/3.jpg)
TABLE OF CONTENTS - ANN
• Introduction machine learning
![Page 4: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/4.jpg)
MACHINE LEARNING
• Automated iterative algorithms to handle huge amounts of data
![Page 5: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/5.jpg)
MACHINE LEARNING
• Finding features and connections
![Page 6: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/6.jpg)
MACHINE LEARNING
• Finding features and connections
• Clustering
• Sorting
• Categorization
• Prediction
![Page 7: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/7.jpg)
BASIC IDEAS OF MACHINE LEARNING
![Page 8: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/8.jpg)
BASIC IDEAS OF MACHINE LEARNING
![Page 9: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/9.jpg)
WHAT ARE WE USUALLY DOING?
![Page 10: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/10.jpg)
BASIC IDEAS OF MACHINE LEARNING
• Unsupervised learning:
• Set algorithm gets input and find structures on its own
• By iterative update of some values
• building clusters, trees, connections
![Page 11: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/11.jpg)
BASIC IDEAS OF MACHINE LEARNING
• Supervised learning:
• Special cases:
• Semi-supervised learning – e.g. Netflix: you may also like
• Reinforcement learning – rewards and punishment feedback e.g. GO-Computer
![Page 12: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/12.jpg)
ARTIFICIAL NEURAL NETWORKS
• Famous example of supervised machine learning
• Popular subcategories:
![Page 13: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/13.jpg)
ARTIFICIAL NEURAL NETWORKS
• Basic ANNs
• Convolutional neural networks
• Recurrent neural networks
![Page 14: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/14.jpg)
ARTIFICIAL NEURAL NETWORKS
• Connect desired input with desired output !
• See if the algorithm can find a good internal representation
![Page 15: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/15.jpg)
PRINCIPLES OF ARTIFICIAL NEURAL NETWORKS
• Training phase Adjustment
• Test phase Evaluation
• Usage Prediction
![Page 16: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/16.jpg)
TABLE OF CONTENTS - ANN
• Basics of mathematics behind ANNs
![Page 17: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/17.jpg)
PRINCIPLES OF ARTIFICIAL NEURAL NETWORKS
• Choose desired input and output
• Classifiers or Regressors
• The algorithm tries to find its own way
to connect them
• This needs a good and large dataset !
![Page 18: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/18.jpg)
MATHEMATICAL REPRESENTATION OF THIS CONNECTION- TRAINING PHASE
• Initialization of weights
• Feed forward
• Compare difference between calculated
and real output with loss function
• Iteratively update of weights by a distinct
amount (learning rate) via
backpropagation to minimize loss function
• Repeat this process by a distinct amount of
iterations (Epochs)
![Page 19: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/19.jpg)
![Page 20: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/20.jpg)
IMPORTANT VOCABULARY FOR TRAINING PHASE
• Weights – factors to weight the values of different nodes
• All the learning lies in the adjustment of weights !!!
• Activation function – a function applied to the value of a node
• Loss function – e.g. mean squared error, mean absolute percentage error (regressors)
binary crossentropy (classifiers)
• Difference between real output and calculated output
• Should be minimized
• Backpropagation algorithm
• Stochastic gradient descent
• Mechanism to iteratively adjust the weights
![Page 21: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/21.jpg)
AND NOW ADD SOME HIDDEN LAYERS
The Artificial Neural Networks handbook: Part 1
www.datasciencecentral.com
![Page 22: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/22.jpg)
TABLE OF CONTENTS - ANN
• Things to take into account
![Page 23: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/23.jpg)
WHEN TO USE ARTIFICIAL NEURAL NETWORKS?
• It connects desired input with desired output !
![Page 24: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/24.jpg)
WHEN TO USE ARTIFICIAL NEURAL NETWORKS?
• It connects desired input with desired output !
• As long as you have enough training data…
![Page 25: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/25.jpg)
Change.org
Optimum.net
![Page 26: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/26.jpg)
WHEN TO USE ARTIFICIAL NEURAL NETWORKS?
• It connects desired input with desired output !
• As long as you have enough training data…
• Always !?
![Page 27: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/27.jpg)
WHEN TO USE ARTIFICIAL NEURAL NETWORKS?
• It connects desired input with desired output !
• As long as you have enough training data…
• Always !?
• Classification and regression prediction problems
![Page 28: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/28.jpg)
GARBAGE IN – GARBAGE OUT
• Limitations are quite clear,
if you understood the mathematics
of the feed forward algorithm
![Page 29: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/29.jpg)
GARBAGE IN – GARBAGE OUT
• Limitations are quite clear,
if you understood the mathematics
of the feed forward algorithm
What are these activation functions
doing in our tiny network?
![Page 30: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/30.jpg)
EXAMPLE OF A HOUSE
Bedrooms Area m2 Age Public
Transport lines
Urban Rural Price
3 150 130 2 0 1 200.000
2 100 5 5 1 0 350.000
4 200 60 1 0 1 100.000
1 70 10 1 1 0 ???
Rows: samples (each row is an input vector)
Columns: categories (each in a distinct input node)
![Page 31: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/31.jpg)
EXAMPLE OF A HOUSE
Bedrooms Area m2 Age Public
Transport lines
Urban Rural Price
3 150 130 2 0 1 200.000
2 100 5 5 1 0 350.000
4 200 60 1 0 1 100.000
1 70 10 1 1 0 ???
Dummy Variable Trap
Redundant Information can trap the backprob algorithm with unwanted gradients
For Regression Problems
![Page 32: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/32.jpg)
EXAMPLE OF A HOUSE
Bedrooms Area m2 Age Public Transport lines Urban Price Quickly
sellable?
Days to sell
3 150 130 2 0 200.000 0 274
2 100 5 5 1 350.000 1 12
4 200 60 1 0 100.000 1 23
1 70 10 1 1 300.000 ??? ???
Do you see a problem due to weight
adjustment?
Typical learning rate : 0.001
![Page 33: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/33.jpg)
INPUT
• Carefully think about your input
• Relative independency of columns/categories within a sample
• Dependency of rows/samples within a category: SCALING
• Provide different types of input: time courses are usually a bad idea
• Try recurrent neural networks if time steps are of equal length
Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable? Days to sell
3 150 130 2 0 200.000 0 274
2 100 5 5 1 350.000 1 12
4 200 60 1 0 100.000 1 23
1 70 10 1 1 300.000 ??? ???
![Page 34: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/34.jpg)
OUTPUT
• Carefully think about your output
• It’s harder for the network to predict different outputs at the same time
• Nearly all weights contribute to each output
• Try to avoid mixing output types
• Classifiers are much easier to evaluate
• And often easier to train
Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable? Days to sell
3 150 130 2 0 200.000 0 274
2 100 5 5 1 350.000 1 12
4 200 60 1 0 100.000 1 23
1 70 10 1 1 300.000 ??? ???
![Page 35: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/35.jpg)
WHAT’S THE EASIEST WAY TO BUILD AN ANN?
• Using python and the KERAS or PYTORCH library
• 13 – 30 lines of code for the framework and basic pipeline
from importing data till evaluation !!
![Page 36: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/36.jpg)
WHAT’S THE EASIEST WAY TO BUILD AN ANN?
• Using python and the KERAS or PYTORCH library
• 11 – 30 lines of code for the framework and basic pipeline
from importing data till evaluation !!
So where is the problem?
![Page 37: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/37.jpg)
Backpropagation is a very popular neural network learning
algorithm because it is conceptually simple, computationally
efficient, and because it often works. However, getting it to work
well, and sometimes to work at all, can seem more of an art than a
science. Designing and training a network using backprop requires
making many seemingly arbitrary choices such as the number and
types of nodes, layers, learning rates, training and test sets, and so
forth. These choices can be critical, yet there is no foolproof recipe
for deciding them because they are largely problem and data
dependent.
Yann LeCun – Efficient BackProp
![Page 38: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/38.jpg)
HYPERPARAMETERS – OUR PART IN THE PROCESS
• Number of Hidden layers their number of nodes
• Batch size and Epochs (related to frequency of weight updates)
• Learning rate and Optimizer (Stochastic gradient descent)
• Loss functions and activation functions
• Regularization, dropouts, noise layers and other methods to avoid overfitting
• Architecture of the network
![Page 39: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/39.jpg)
HYPERPARAMETERS – OUR PART IN THE PROCESS
• The big problem: ALL PARAMETERS HAVE A TRADEOFF
![Page 40: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/40.jpg)
EXAMPLE OF A MORE COMPLICATED ARCHITECTURERESIDUAL NETWORK LAYERS
• Show output of early layer to much deeper layers
• Refeeding predicted outputs
• Reshow the original input
• Focus region of weight adjustment
due to auxiliary outputs and auxiliary
loss functions
![Page 41: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/41.jpg)
INFORMATION IS ALL – GARBAGE IN - GARBAGE OUT
• Preprocessing and Scaling data is crucial
• Good and large datasets reduce overfitting dramatically
• Every additional type of information can boost your prediction ability dramatically
• Refeeding your initial guesses in the network can help – initial guesses are information
• Auxiliary inputs and outputs can give a good drive your network
• The random initialization of weights can have a bigger impact than parameter adjustments !
![Page 42: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/42.jpg)
CNN
• Basics
Labs.bawi.io
![Page 43: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/43.jpg)
CONVOLUTIONAL NETS ARE 3D STRUCTURES
• Image processing – 3D tensor , consisting of Image (2D) splitted into RGB channels
http://cs231n.github.io/convolutional-networks/
![Page 44: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/44.jpg)
CONVOLUTIONAL NETWORK + ANN
medium.freecodecamp.org/an-intuitive-guide-to-convolutional-neural-networks-260c2de0a050
![Page 45: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/45.jpg)
ZERO-PADDING
![Page 46: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/46.jpg)
CONVOLUTION
• Filter size (receptive field)
• Stride
• Padding
medium.freecodecamp.org/an-intuitive-guide-to-convolutional-neural-networks-260c2de0a050
![Page 47: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/47.jpg)
CONVOLUTION
• 3D input needs 3D Kernels
• Here are 2 Kernels/Filters
• number of Filters choosable
• number of features
http://cs231n.github.io/convolutional-networks/
![Page 48: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/48.jpg)
CONVOLUTION
• 3D input needs 3D Kernels
• Here are 2 Kernels/Filters
• number of Filters choosable
• number of features
• Stride ?
• Output volume?
http://cs231n.github.io/convolutional-networks/
![Page 49: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/49.jpg)
APPLYING AN ACTIVATION FUNCTION (E.G. RELU)
http://cs231n.github.io/convolutional-networks/
![Page 50: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/50.jpg)
DIMENSION OF THE FILTERS/KERNELS
Researchgate.net
![Page 51: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/51.jpg)
LET’S CHECK IF WE HAVE UNDERSTOOD THE CONVOLUTION
• Stride
• # Filters
• Filter size
• Padding
jeremyjordan.me/convnet-architectures/
![Page 52: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/52.jpg)
RECEPTIVE FIELD REDUCES NUMBER OF CONNECTIONS
• 5x5x3 Filter has 75 weights (+1 bias parameter)
![Page 53: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/53.jpg)
RECEPTIVE FIELD REDUCES NUMBER OF CONNECTIONS
• 5x5x3 Filter has 75 weights (+1 bias parameter)
• Multiplied by number of pixels in the CONV output layer
![Page 54: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/54.jpg)
RECEPTIVE FIELD REDUCES NUMBER OF CONNECTIONS
• 1) 5x5x3 Filter has 75 weights (+1 bias parameter)
• 2) Multiplied by number of pixels in the CONV output layer
• 3) Multiplied by number of filters
• Parameter sharing
kicks out 2)
http://cs231n.github.io/convolutional-networks/
![Page 55: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/55.jpg)
POOLING
Size choosable – 2x2 reduces size by 75% without significant loss of information
Speeds up processing
http://cs231n.github.io/convolutional-networks/
![Page 56: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/56.jpg)
LET’S CHECK IF WE HAVE UNDERSTOOD EVERYTHING
• Stride
• # Filters
• Filter size
• Padding
• Pooling size
• What’s the
blue stuff ?
jeremyjordan.me/convnet-architectures/
![Page 57: Artificial neural networks - Brain Circuits · •Try recurrent neural networks if time steps are of equal length Bedrooms Area m2 Age Public Transport lines Urban Price Quickly sellable?](https://reader033.vdocument.in/reader033/viewer/2022051512/602ca7d62512347ddc138a82/html5/thumbnails/57.jpg)
LET’S CHECK IF WE HAVE UNDERSTOOD EVERYTHING
https://medium.com/machine-learning-bites/deeplearning-series-convolutional-neural-networks-a9c2f2ee1524