image analysis with deep learning -...
TRANSCRIPT
![Page 1: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/1.jpg)
Deep Learning Made Easy with GraphLab CreatePiotr Teterwak!Dato Team
1
![Page 2: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/2.jpg)
Who I am
Piotr Teterwak Software Engineer
2
![Page 3: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/3.jpg)
GraphLab Create• A platform for building predictive
applications, fast!• Data engineering on Big Data!• Interactive visualization!• Fast machine learning toolkits!• Easy deployment!!
• Python frontend, C++ backend
3
![Page 4: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/4.jpg)
The Dato Team
4
![Page 5: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/5.jpg)
Making Deep Learning Easy
5
![Page 6: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/6.jpg)
Deep Learning
6
![Page 7: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/7.jpg)
Deep Learning Made Easy!• Intuitive API!• Transfer Learning!• Integration with other tools in GraphLab
Create
7
![Page 8: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/8.jpg)
What is Deep Learning?
8
![Page 9: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/9.jpg)
Machine Learning• Algorithms that can learn from data without
being explicitly programmed. !• One example would be image
classification, i.e binning an image as one of a fixed number of categories.
9
![Page 10: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/10.jpg)
Machine Learning
“cat”
10
![Page 11: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/11.jpg)
Deep Learning
“cat”
f1(x) f2(x) f3(x)
11
![Page 12: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/12.jpg)
http://deeplearning.stanford.edu/wiki/images/4/40/Network3322.png
Deep Neural Networks
P(cat|x)
P(dog|x)
12
![Page 13: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/13.jpg)
Deep Neural Networks• Can model any function with enough
hidden units. !• This is tremendously powerful: given
enough units, it is possible to train a neural network to solve arbitrarily difficult problems. !
• But also very difficult to train, too many parameters means too much memory+computation time.
13
![Page 14: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/14.jpg)
Neural Nets and GPU’s• Many operations in Neural Net training can
happen in parallel!• Reduces to matrix operations, many of
which can be easily parallelized on a GPU.
14
![Page 15: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/15.jpg)
Convolutional Neural Nets• Strategic removal of edges
15
Input Layer
Hidden Layer
![Page 16: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/16.jpg)
Convolutional Neural Nets• Strategic removal of edges
16
Input Layer
Hidden Layer
![Page 17: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/17.jpg)
Convolutional Neural Nets• Strategic removal of edges
17
Input Layer
Hidden Layer
![Page 18: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/18.jpg)
Convolutional Neural Nets• Strategic removal of edges
18
Input Layer
Hidden Layer
![Page 19: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/19.jpg)
Convolutional Neural Nets• Strategic removal of edges
19
Input Layer
Hidden Layer
![Page 20: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/20.jpg)
Convolutional Neural Nets• Strategic removal of edges
20
Input Layer
Hidden Layer
![Page 21: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/21.jpg)
Convolutional Neural Nets
http://ufldl.stanford.edu/wiki/images/6/6c/Convolution_schematic.gif
21
![Page 22: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/22.jpg)
Pooling layer
Ranzato, LSVR tutorial @ CVPR, 2014. www.cs.toronto.edu/~ranzato
22
![Page 23: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/23.jpg)
Pooling layer
http://ufldl.stanford.edu/wiki/images/6/6c/Pooling_schematic.gif
23
![Page 24: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/24.jpg)
Overall architecture
A. Krizhevsky, I. Sutskever and G.E. Hinton. “ImageNet Classification with Deep Convolutional Neural Networks”. NIPS (2012)
24
![Page 25: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/25.jpg)
Hierarchichal Representation
Y. Bengio (2009)
25
Hands -‐ Face -‐ Ground
![Page 26: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/26.jpg)
Input
Learned hierarchy
Lee et al. ‘Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations’ ICML 2009
Deep learning features
Output26
![Page 27: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/27.jpg)
Where can we use Deep Learning?
27
![Page 28: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/28.jpg)
Image tagging
28
![Page 29: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/29.jpg)
A quick demo!
29
![Page 30: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/30.jpg)
!!!!!!!!!
• Notice the cycle…you can only break out of this with intuition, time, and lots of frustration.!
• But, when you do, magic happens!
Create Model
Labelled data
Train Set
Test Set
80%
20%
Validate?
Probably not good enough
Adjust hyper-‐parameters
Deep learning workflow
30
![Page 31: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/31.jpg)
Simplifying Deep Learning with Deep Features and Transfer Learning
31
![Page 32: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/32.jpg)
Transfer learning• Train a model on one task, use it for
another task!• Examples!
• Learn to walk, use that knowledge to run !• Train image tagger to recognize cars, use that
knowledge to recognize trucks.
32
![Page 33: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/33.jpg)
Input
Learned hierarchy
Lee et al. ‘Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations’ ICML 2009
Deep learning features
Output33
![Page 34: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/34.jpg)
Lee et al. ‘Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations’ ICML 2009
Mid-‐level features probably useful for other tasks which require detection of facial anatomy
Feature extractionInput
Learned hierarchy
34
![Page 35: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/35.jpg)
http://deeplearning.stanford.edu/wiki/images/4/40/Network3322.png
Extract activations from some deep layer of neural network
Feature extraction
35
![Page 36: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/36.jpg)
Create Simpler Model
Labelled data
Extract Features using Neural Net
trained on different task
Train Set
Test Set
80%
20%
Validate?
Probably worksDeploy$$$
Transfer learning using deep features
36
![Page 37: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/37.jpg)
Using ImageNet-trained network as extractor for general features• Using classic AlexNet architechture pioneered by
Alex Krizhevsky et. al in ImageNet Classification with Deep Convolutional Neural Networks !
• It turns out that a neural network trained on ~1 million images of about 1000 classes makes a surprisingly general feature extractor!
• First illustrated by Donahue et al in DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
37
![Page 38: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/38.jpg)
Demo
38
![Page 39: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/39.jpg)
Caltech-101
39
![Page 40: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/40.jpg)
Caltech-101
40
Extract features here
![Page 41: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/41.jpg)
Deep Features and Logistic Regression
41
![Page 42: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/42.jpg)
What else can we do with Deep Features?
42
![Page 43: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/43.jpg)
Finding similar images
43
![Page 44: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/44.jpg)
Clustering images
Goldberger et al.
Set of Images
44
![Page 45: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/45.jpg)
How general are these Deep Features?
45
![Page 46: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/46.jpg)
Deep Features are Generalizable
46
![Page 47: image analysis with deep learning - NVIDIAon-demand.gputechconf.com/gtc/2015/presentation/S5630...Deep learning workflow 30 Simplifying Deep Learning with Deep Features and Transfer](https://reader036.vdocument.in/reader036/viewer/2022081502/5ec714dbf17d3575a27ac09b/html5/thumbnails/47.jpg)
Thank you!• To learn more: http://www.dato.com/learn!!
• Play with the demos:!• Pathways: https://pathways-demo.herokuapp.com/!• Phototag: http://phototag.herokuapp.com/!
!• Contact: !
• [email protected] !!
• We are hiring! [email protected]!!• Thank you to Nvidia for designing and providing the hardware to make this
possible!!
• Please complete the Presenter Evaluation. Your feedback is important!
47