recurrent neural network - cvut.cz · motivation two households, both alike in dignity, in fair...
TRANSCRIPT
![Page 1: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/1.jpg)
Recurrent Neural NetworkMichal Bouška
![Page 2: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/2.jpg)
Recurrent Neural NetworkLSTMGRUPointer Network
2
![Page 3: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/3.jpg)
Problem
Two households, both alike in dignity,In fair Verona, where we lay our scene,From ancient grudge break to new mutiny,Where civil blood makes civil hands unclean.From forth the fatal loins of these two foesA pair of star-cross'd lovers take their life;Whose misadventured piteous overthrowsDo with their death bury their parents' strife.The fearful passage of their death-mark'd love,And the continuance of their parents' rage,Which, but their children's end, nought could remove,Is now the two hours' traffic of our stage;The which if you with patient ears attend,What here shall miss, our toil shall strive to mend.
3
![Page 4: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/4.jpg)
Recurrent Neural Network
4
![Page 5: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/5.jpg)
Architecture
5
![Page 6: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/6.jpg)
Architecture
6
![Page 7: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/7.jpg)
Problem of general RNN
7
![Page 8: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/8.jpg)
Problem of general RNN
8
![Page 9: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/9.jpg)
LSTM
9
![Page 10: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/10.jpg)
Sigmoid activation
10
![Page 11: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/11.jpg)
Tanh activations
11
![Page 12: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/12.jpg)
Forget Gate
12
![Page 13: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/13.jpg)
Input Gate
13
![Page 14: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/14.jpg)
Input Gate
14
Romeo Montague => rm - rf /
![Page 15: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/15.jpg)
Cell State
15
![Page 16: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/16.jpg)
Output Gate
16
![Page 17: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/17.jpg)
GRU
17
![Page 18: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/18.jpg)
Architecture
18
![Page 19: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/19.jpg)
Sequence
19
![Page 20: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/20.jpg)
Motivation
Two households, both alike in dignity,In fair Verona, where we lay our scene,From ancient grudge break to new mutiny,Where civil blood makes civil hands unclean.From forth the fatal loins of these two foesA pair of star-cross'd lovers take their life;Whose misadventured piteous overthrowsDo with their death bury their parents' strife.The fearful passage of their death-mark'd love,And the continuance of their parents' rage,Which, but their children's end, nought could remove,Is now the two hours' traffic of our stage;The which if you with patient ears attend,What here shall miss, our toil shall strive to mend.
20
![Page 21: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/21.jpg)
Problem
Two households, both alike in dignity,In fair Verona, where we lay our scene,From ancient grudge break to new mutiny,Where civil blood makes civil hands unclean.From forth the fatal loins of these two foesA pair of star-cross'd lovers take their life;Whose misadventured piteous overthrowsDo with their death bury their parents' strife.The fearful passage of their death-mark'd love,And the continuance of their parents' rage,Which, but their children's end, nought could remove,Is now the two hours' traffic of our stage;The which if you with patient ears attend,What here shall miss, our toil shall strive to mend.
21
![Page 22: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/22.jpg)
Pointer Network
22
![Page 23: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/23.jpg)
Architecture
● Attention pointing to input● In each step select one input element
23
![Page 24: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/24.jpg)
Empirical result
Convex Hull Delaunay Triangulation
24
![Page 25: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/25.jpg)
Empirical Result
25
![Page 26: Recurrent Neural Network - cvut.cz · Motivation Two households, both alike in dignity, In fair Verona, where we lay our scene, From ancient grudge break to new mutiny, Where civil](https://reader033.vdocument.in/reader033/viewer/2022042220/5ec677d1a60cb616bc75694f/html5/thumbnails/26.jpg)
Resources
● http://papers.nips.cc/paper/5866-pointer-networks.pdf● https://towardsdatascience.com/illustrated-guide-to-lstms-and-gru-s-a-ste
p-by-step-explanation-44e9eb85bf21● https://colah.github.io/posts/2015-08-Understanding-LSTMs/● http://shakespeare.mit.edu/romeo_juliet/full.html● https://optimization.mccormick.northwestern.edu/index.php/Traveling_sal
esman_problems● https://worldsmarathons.com/marathon/giulietta-romeo-half-marathon● https://www.wikiwand.com/en/William_Shakespeare
26