introduction to convolutional neural nets
Post on 12-Apr-2017
619 Views
Preview:
TRANSCRIPT
Facebook!
Clarifai
Let’s reveal the mystery!
Neural Nets
What this can do!
Let’s start with a node
ArchitectureForward Pass•Calculating the loss
Backward Pass
•Backpropagation Algorithm
•Distributing Gradients
Optimizing
•Reducing the loss
•Updating the weight matrix
Updating weights
Loss FunctionUpdating weights
YOU CAN’T TRAINIF THERE ARE NO GRADIENTS
This went deeper!How ?With the help of two superheros!
Deep Neural Networks
What makes this special?
Hierarchical Feature Representation
Why this feature thing is so effective ?
Then what happened ?Deep Neural Nets became harder and harder to train!
DEEP NETS!
Y U NO EASY?
Number Of Parameters?
These nets are huge!
Millions of layers Millions of nodesBillions of parameters
BUT DON’T KNOW HOW MANY HIDDEN LAYERS / NODES TO
USE?
WHEN YOU WANT TO BUILD A NEURAL NET
Overfitting
Killing the training process
Neuron in a neural network
Sigmoid activation function
Too much connections ? What if we input some exact features!
Feature Engineering
Canny edge detection filter
How we see things ?
Important!
Not all pixels!
But patterns of pixels!
This is fast!
Right to left ?
Can you remember
NOT EVERYTHING
BUT IMPORTANT THINGS!
Can we design set of features for machines ??
No way!We may design some high level features!
But our machines deal with PIXELS!
What if we let the machine to extract it’s own features!
Wow that was nice :)
BRACE YOURSELVESCNN IS COMING
Imagenet Challenge
Step by step
ConvNets have one layer called CN layer after the input layer
This act as an automatic feature extractor
Additional Layers!Convolution layer
RELU layer (Nonlinearity)
Max pooling layer
Amazing local connectivity
Let’s go layerwise!
Convolutional Layer This is simply applying convolution to input pixels
Now! What is convolution?
Mathematical way
Easy way!
For an image
CONVOLUTION
EVERYWHERE
More generally...
Filter search for things in the image
When filter sees something similar to it’s orientation in an image it will fire up!
Image get transformed into something new !
We call this new image as a feature mapBecause it mapps some features from the original
image w.r.t to the filter!
Right - Feature map obtained by applying canny edge filter on an image
It’s not only a single filter!
Many filters searching for diffrent things!
How amazing!
So we will have different feature maps!
Simply each filter has it’s own task
Gabor filters are similar to those of the human visual system, and they have been found to be particularly appropriate for texture
representation and discrimination.
How filter weights looking at an image!
This is how it’s done!
Feature maps = Set of neurones
That’s convolution layer!But where’s the magic ?
All these filters are trainable!
Means ?
These are not man made filters! Look at the values of following filter kernels
What about the filter?
We can design size , channels of the filter
But not its values!
Each filter values is trainable weight
All in one!So every filter values is trainable
It’s like a set of weights !
But how ?
With backprop
Convolution is a perfectly differentiable function!
So we can learn it’s weight parameters
Reminder!
Then we apply some nonlinearity We apply rely on each point in the feature maps
Remember : These points in the feature maps are like neurons
This is like just forgetting the negative parts of the
So the orientation be like
Then the pooling layer
This is like a filter without parameters!
This is for subsample or to reduce the dimensions of the feature maps
Simply this will extract the most important features from the feature map
That’s basics! Lenet
So finally….
We can train these filter weights using backprop and some kind of optimization algorithms(Adam)
Here’s a such view of a trained filter
Motivation!
Q & A !
top related