![Page 1: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/1.jpg)
CPSC 340:Machine Learning and Data Mining
Convolutional Neural Networks
Fall 2016
![Page 2: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/2.jpg)
Admin
• Assignment 5:
– Due Friday, 1 late day to hand in Monday, etc.
• Assignment 6:
– Due next Friday (usual late day policy, assuming phantom “classes”).
• Final:
– December 12 (8:30am – HEBB 100)
– Covers Assignments 1-6.
– Final from last year and list of topics will be posted.
– Closed-book, cheat sheet: 4-pages each double-sided.
![Page 3: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/3.jpg)
Last Lectures: Deep Learning
• We’ve been discussing neural network / deep learning models:
• On Friday we discussed unprecedented vision/speech performance.
https://arxiv.org/pdf/1409.0575v3.pdf
![Page 4: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/4.jpg)
Last Lectures: Deep Learning
• We’ve been discussing neural network / deep learning models:
• On Monday we discussed heuristics to make it work:
– Parameter initialization and data transformations.
– Setting the step size(s) in stochastic gradient.
– Alternative non-linear functions like ReLU.
– Different forms of regularization:
• L2-regularization, early stopping, dropout.
• These are often still not enough to get deep models working.
![Page 5: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/5.jpg)
Motivation: Automatic Brain Tumor Segmentation
• Task: segmentation tumors and normal tissue in multi-modal MRI data.
• Applications:– Radiation therapy target planning, quantifying treatment responses.– Mining growth patterns, image-guided surgery.
• Challenges:– Variety of tumor appearances, similarity to normal tissue.– “You are never going to solve this problem.”
Input: Output:
![Page 6: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/6.jpg)
Naïve Voxel-Level Classifier
• We could treat classifying a voxel as supervised learning:
• We can formulate predicting yi given xi as supervised learning.
• But it doesn’t work at all with these features.
![Page 7: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/7.jpg)
Need for Context
• The individual voxel values are almost meaningless:
– This xi could lead to different yi.
• Intensities not standardized.
• Non-trivial overlap in signal for different tissue types.
• “Partial volume” effects at boundaries of tissue types.
![Page 8: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/8.jpg)
Need for Context
• We need to represent the spatial “context” of the voxel.
– Include all the values of neighbouring voxels?
• Using all voxels requires lots of data to find patterns.
– Measure summary statistics (mean, variance, etc.) of the neighbourhood?
• Loses spatial information present in voxels.
– Standard approach is uses convolutions to represent neighbourhood.
![Page 9: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/9.jpg)
1D Convolution
• 1D convolution input:
– Signal ‘x’ which is a vector length ‘n’.
• Indexed by i=1,2,…,n.
– Filter ‘w’ which is a vector of length ‘2m+1’:
• Indexed by i=-m,-m+1,…-2,0,1,2,…,m-1,m
• 1D convolution output:
– New vector ‘z’ of length ‘n’ with elements:
![Page 10: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/10.jpg)
1D Convolution Examples
• Element zi of 1D convolution is given by:
• Examples:
– “Identity”
– “Translation”
![Page 11: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/11.jpg)
1D Convolution Examples
• Element zi of 1D convolution is given by:
• Examples:
– “Identity”
– “Average”
![Page 12: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/12.jpg)
Boundary Issue
• What can we about the “?” at the edges?
• Can assign values past the boundaries:• “Zero”:
• “Replicate”:
• “Mirror”:
• Or just ignore the “?” values and return a shorter vector:
![Page 13: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/13.jpg)
1D Convolution in Matrix Notation
• Each element of a convolution is an inner product:
• So convolution is a matrix multiplication:
![Page 14: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/14.jpg)
Why is this useful?
• Consider a 1D dataset:– Want to classify each
time into yi in {1,2,3}.
– Example: sound data.
• Easy to distinguish class 2 from the other classes (xi are smaller).
• Harder to distinguish between class 1 and class 3 (similar xi range).– But convolutions can represent that class 3 is more “spiky”.
![Page 15: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/15.jpg)
1D Convolution Examples
• Translation convolution shift signal:
![Page 16: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/16.jpg)
1D Convolution Examples
• Averaging convolution computes local mean:
![Page 17: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/17.jpg)
1D Convolution Examples
• Averaging over bigger window gives coarser view of signal:
![Page 18: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/18.jpg)
1D Convolution Examples
• Gaussian convolution blurs signal:
– Compared to averaging it’s more smooth and maintains peaks better.
![Page 19: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/19.jpg)
1D Convolution Examples
• Sharpen convolution enhances peaks.
![Page 20: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/20.jpg)
1D Convolution Examples
• Laplacian convolution approximates derivative:
![Page 21: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/21.jpg)
1D Convolution Examples
• Laplacian convolution approximates derivative:
![Page 22: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/22.jpg)
1D Convolution Examples
• Laplacian of Gaussian approximates derivative after blurring:
![Page 23: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/23.jpg)
1D Convolution Examples
• We often use maximum over several convolutions as features:
– We could take maximum of Laplacian of Gaussian over xi and its 16 KNNs.
![Page 24: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/24.jpg)
Why is this useful?
• Easy to distinguish class 2 from with original signal.
• Easy to distinguish class 1 from 3 with max(Laplacian(Gaussian)).– Convolutions and max(convolutions) are very useful for sequence data.
• For sound data two related techniques are Fourier transforms and spectrograms.
![Page 25: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/25.jpg)
Images and Higher-Order Convolution
• 2D convolution:
– Signal ‘x’ is the pixel intensities in an ‘n’ by ‘n’ image.
– Filter ‘w’ is the pixel intensities in a ‘2m+1’ by ‘2m+1’ image.
• The 2D convolution is given by:
• 3D and higher-order convolutions are defined similarly.
![Page 26: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/26.jpg)
Image Convolution Examples
![Page 27: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/27.jpg)
Image Convolution Examples
![Page 28: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/28.jpg)
Image Convolution Examples
![Page 29: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/29.jpg)
Image Convolution Examples
![Page 30: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/30.jpg)
Image Convolution Examples
![Page 31: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/31.jpg)
Image Convolution Examples
![Page 32: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/32.jpg)
Image Convolution Examples
![Page 33: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/33.jpg)
Image Convolution Examples
![Page 34: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/34.jpg)
Image Convolution Examples
![Page 35: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/35.jpg)
Image Convolution Examples
![Page 36: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/36.jpg)
Image Convolution Examples
![Page 38: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/38.jpg)
Image Convolution Examples
![Page 39: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/39.jpg)
Image Convolution Examples
![Page 40: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/40.jpg)
Image Convolution Examples
![Page 41: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/41.jpg)
Image Convolution Examples
![Page 42: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/42.jpg)
Image Convolution Examples
![Page 43: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/43.jpg)
3D Convolution
![Page 44: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/44.jpg)
3D Convolution
![Page 45: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/45.jpg)
3D Convolution
![Page 46: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/46.jpg)
3D Convolution
![Page 47: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/47.jpg)
3D Convolution
![Page 48: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/48.jpg)
Motivation for Convolutional Neural Networks
• Consider training neural networks on 256 by 256 images.
• Each zi in first layer has 65536 parameters (and 3x this for colour).
– We want to avoid this huge number (due to storage and overfitting).
• Key idea: make Wxi act like convolutions (to make it smaller):
– Each row of W only applies to part of xi.
– Use the same parameters between rows.
![Page 49: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/49.jpg)
Convolutional Neural Networks
• Convolutional Neural Networks classically have 3 layer “types”:
– Fully connected layer: usual neural network layer with unrestricted W.
![Page 50: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/50.jpg)
Convolutional Neural Networks
• Convolutional Neural Networks classically have 3 layer “types”:
– Fully connected layer: usual neural network layer with unrestricted W.
– Convolutional layer: restrict W to results of several convolutions.
![Page 51: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/51.jpg)
Convolutional Neural Networks
• Convolutional Neural Networks classically have 3 layer “types”:
– Fully connected layer: usual neural network layer with unrestricted W.
– Convolutional layer: restrict W to results of several convolutions.
– Pooling layer: downsamples result of convolution.
• Can add invariances or just make the number of parameters smaller.
• Usual choice is ‘max pooling’:
![Page 52: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/52.jpg)
LeNet for Optical Character Recognition
http://blog.csdn.net/strint/article/details/44163869
![Page 53: CPSC 340: Data Mining Machine Learningschmidtm/Courses/340-F16/L31.pdfMachine Learning and Data Mining Convolutional Neural Networks Fall 2016 Admin •Assignment 5: –Due Friday,](https://reader034.vdocument.in/reader034/viewer/2022050101/5f4034f60b03273aa6112515/html5/thumbnails/53.jpg)
Summary
• Convolutions are flexible class of signal/image transformations.
• Max(convolutions) can yield features that make classification easy.
• Convolutional neural networks:
– Restrict W(m) matrices to represent sets of convolutions.
– Often combined with max (pooling).
• Next time: modern convolutional neural networks and applications.
– Image segmentation, depth estimation, image colorization, artistic style.