nando de freitas - department of computer science, university of … · 2015. 2. 19. · microsoft...
TRANSCRIPT
![Page 1: Nando de Freitas - Department of Computer Science, University of … · 2015. 2. 19. · Microsoft PowerPoint - lecture8.ppt [Compatibility Mode] Author: nandoadmin Created Date:](https://reader036.vdocument.in/reader036/viewer/2022071114/5feadfd5aa928a6b541d19f7/html5/thumbnails/1.jpg)
Neural networksNando de Freitas
![Page 2: Nando de Freitas - Department of Computer Science, University of … · 2015. 2. 19. · Microsoft PowerPoint - lecture8.ppt [Compatibility Mode] Author: nandoadmin Created Date:](https://reader036.vdocument.in/reader036/viewer/2022071114/5feadfd5aa928a6b541d19f7/html5/thumbnails/2.jpg)
Outline of the lecture
This lecture introduces you to the fascinating subject of classificationand regression with artificial neural networks. In particular, it
Introduces multi-layer perceptrons (MLPs) Teaches you how to combine probability with neural networksso that the nets can be applied to regression, binary classificationand multivariate classification. Discusses the modular approach to backpropagation and neuralnetwork construction in Torch, which was introduced in theprevious lecture.
![Page 3: Nando de Freitas - Department of Computer Science, University of … · 2015. 2. 19. · Microsoft PowerPoint - lecture8.ppt [Compatibility Mode] Author: nandoadmin Created Date:](https://reader036.vdocument.in/reader036/viewer/2022071114/5feadfd5aa928a6b541d19f7/html5/thumbnails/3.jpg)
MLP – 1 neuronu
1 q
q
1
2
yi = P(yi =1| xi ,q )xi1
^
q3xi2
![Page 4: Nando de Freitas - Department of Computer Science, University of … · 2015. 2. 19. · Microsoft PowerPoint - lecture8.ppt [Compatibility Mode] Author: nandoadmin Created Date:](https://reader036.vdocument.in/reader036/viewer/2022071114/5feadfd5aa928a6b541d19f7/html5/thumbnails/4.jpg)
MLP – 1 neuronu
1 q
q
1
2
yi = P(yi =1| xi ,q )xi1
^
q3xi2
![Page 5: Nando de Freitas - Department of Computer Science, University of … · 2015. 2. 19. · Microsoft PowerPoint - lecture8.ppt [Compatibility Mode] Author: nandoadmin Created Date:](https://reader036.vdocument.in/reader036/viewer/2022071114/5feadfd5aa928a6b541d19f7/html5/thumbnails/5.jpg)
MLP – 3 neurons, 2 layersu o
1 qq
1
2
u o
1
q
q
8
4
uq
q
6
7
15q11
12
21
12
11
yi = P(yi =1| xi ,q )^xi1
xi2
q3q9
![Page 6: Nando de Freitas - Department of Computer Science, University of … · 2015. 2. 19. · Microsoft PowerPoint - lecture8.ppt [Compatibility Mode] Author: nandoadmin Created Date:](https://reader036.vdocument.in/reader036/viewer/2022071114/5feadfd5aa928a6b541d19f7/html5/thumbnails/6.jpg)
MLP – 3 neurons, 2 layersu o
1 qq
1
2
u o
1
q
q
8
4
uq
q
6
7
15q11
12
21
12
11
yi = P(yi =1| xi ,q )^xi1
xi2
q3q9
![Page 7: Nando de Freitas - Department of Computer Science, University of … · 2015. 2. 19. · Microsoft PowerPoint - lecture8.ppt [Compatibility Mode] Author: nandoadmin Created Date:](https://reader036.vdocument.in/reader036/viewer/2022071114/5feadfd5aa928a6b541d19f7/html5/thumbnails/7.jpg)
MLP – 3 neurons, 2 layersu o
1 qq
1
2
u o
1
q
q
8
4
uq
q
6
7
15q11
12
21
12
11
yi = P(yi =1| xi ,q )^xi1
xi2
q3q9
![Page 8: Nando de Freitas - Department of Computer Science, University of … · 2015. 2. 19. · Microsoft PowerPoint - lecture8.ppt [Compatibility Mode] Author: nandoadmin Created Date:](https://reader036.vdocument.in/reader036/viewer/2022071114/5feadfd5aa928a6b541d19f7/html5/thumbnails/8.jpg)
![Page 9: Nando de Freitas - Department of Computer Science, University of … · 2015. 2. 19. · Microsoft PowerPoint - lecture8.ppt [Compatibility Mode] Author: nandoadmin Created Date:](https://reader036.vdocument.in/reader036/viewer/2022071114/5feadfd5aa928a6b541d19f7/html5/thumbnails/9.jpg)
MLP – Regressionu o
1 q
q
1
2
u o
1
q
q
3
4
uq
q
6
7
15q11
12
21
12
11
xiyi = E(yi | xi ,q )^
![Page 10: Nando de Freitas - Department of Computer Science, University of … · 2015. 2. 19. · Microsoft PowerPoint - lecture8.ppt [Compatibility Mode] Author: nandoadmin Created Date:](https://reader036.vdocument.in/reader036/viewer/2022071114/5feadfd5aa928a6b541d19f7/html5/thumbnails/10.jpg)
MLP – Regressionu o
1 q
q
1
2
u o
1
q
q
3
4
uq
q
6
7
15q11
12
21
12
11
xiyi = E(yi | xi ,q )^
![Page 11: Nando de Freitas - Department of Computer Science, University of … · 2015. 2. 19. · Microsoft PowerPoint - lecture8.ppt [Compatibility Mode] Author: nandoadmin Created Date:](https://reader036.vdocument.in/reader036/viewer/2022071114/5feadfd5aa928a6b541d19f7/html5/thumbnails/11.jpg)
MLP – Multiclassu o
u o
uq
q
6
7
1
5q11
12
21
12
11
uq
q
122
uq
q
21
q
yi1^
yi2^
yi3^
1 qq
1
2
1
q
q
8
4
xi1
xi2
q3q9
![Page 12: Nando de Freitas - Department of Computer Science, University of … · 2015. 2. 19. · Microsoft PowerPoint - lecture8.ppt [Compatibility Mode] Author: nandoadmin Created Date:](https://reader036.vdocument.in/reader036/viewer/2022071114/5feadfd5aa928a6b541d19f7/html5/thumbnails/12.jpg)
MLP – Multiclassu o
u o
uq
q
6
7
1
5q11
12
21
12
11
uq
q
122
uq
q
21
q
yi1^
yi2^
yi3^
1 qq
1
2
1
q
q
8
4
xi1
xi2
q3q9
![Page 13: Nando de Freitas - Department of Computer Science, University of … · 2015. 2. 19. · Microsoft PowerPoint - lecture8.ppt [Compatibility Mode] Author: nandoadmin Created Date:](https://reader036.vdocument.in/reader036/viewer/2022071114/5feadfd5aa928a6b541d19f7/html5/thumbnails/13.jpg)
MLP – Multiclass
![Page 14: Nando de Freitas - Department of Computer Science, University of … · 2015. 2. 19. · Microsoft PowerPoint - lecture8.ppt [Compatibility Mode] Author: nandoadmin Created Date:](https://reader036.vdocument.in/reader036/viewer/2022071114/5feadfd5aa928a6b541d19f7/html5/thumbnails/14.jpg)
Deep learning & backprop
![Page 15: Nando de Freitas - Department of Computer Science, University of … · 2015. 2. 19. · Microsoft PowerPoint - lecture8.ppt [Compatibility Mode] Author: nandoadmin Created Date:](https://reader036.vdocument.in/reader036/viewer/2022071114/5feadfd5aa928a6b541d19f7/html5/thumbnails/15.jpg)
Deep learning & backprop
![Page 16: Nando de Freitas - Department of Computer Science, University of … · 2015. 2. 19. · Microsoft PowerPoint - lecture8.ppt [Compatibility Mode] Author: nandoadmin Created Date:](https://reader036.vdocument.in/reader036/viewer/2022071114/5feadfd5aa928a6b541d19f7/html5/thumbnails/16.jpg)
Reverse auto-diff: why it works
![Page 17: Nando de Freitas - Department of Computer Science, University of … · 2015. 2. 19. · Microsoft PowerPoint - lecture8.ppt [Compatibility Mode] Author: nandoadmin Created Date:](https://reader036.vdocument.in/reader036/viewer/2022071114/5feadfd5aa928a6b541d19f7/html5/thumbnails/17.jpg)
Deep learning: linear layer
![Page 18: Nando de Freitas - Department of Computer Science, University of … · 2015. 2. 19. · Microsoft PowerPoint - lecture8.ppt [Compatibility Mode] Author: nandoadmin Created Date:](https://reader036.vdocument.in/reader036/viewer/2022071114/5feadfd5aa928a6b541d19f7/html5/thumbnails/18.jpg)
Deep learning: ReLU layer
![Page 19: Nando de Freitas - Department of Computer Science, University of … · 2015. 2. 19. · Microsoft PowerPoint - lecture8.ppt [Compatibility Mode] Author: nandoadmin Created Date:](https://reader036.vdocument.in/reader036/viewer/2022071114/5feadfd5aa928a6b541d19f7/html5/thumbnails/19.jpg)
Deep learning: extremely flexible!
![Page 20: Nando de Freitas - Department of Computer Science, University of … · 2015. 2. 19. · Microsoft PowerPoint - lecture8.ppt [Compatibility Mode] Author: nandoadmin Created Date:](https://reader036.vdocument.in/reader036/viewer/2022071114/5feadfd5aa928a6b541d19f7/html5/thumbnails/20.jpg)
Next lecture
In the next lecture, we will look at a successful type of neuralnetwork that is very popular in speech and object recognition,known as a convolutional neural network.