seq2 seq learning
TRANSCRIPT
![Page 1: Seq2 seq learning](https://reader030.vdocument.in/reader030/viewer/2022021420/58cfa6411a28ab6b088b54ed/html5/thumbnails/1.jpg)
Seq2Seq learningand recent advances
Vu Pham, 08.03.2017
![Page 2: Seq2 seq learning](https://reader030.vdocument.in/reader030/viewer/2022021420/58cfa6411a28ab6b088b54ed/html5/thumbnails/2.jpg)
Agenda
●
●
●
2
![Page 3: Seq2 seq learning](https://reader030.vdocument.in/reader030/viewer/2022021420/58cfa6411a28ab6b088b54ed/html5/thumbnails/3.jpg)
Recap: CNN for NLP
● →
●
●
●3
![Page 4: Seq2 seq learning](https://reader030.vdocument.in/reader030/viewer/2022021420/58cfa6411a28ab6b088b54ed/html5/thumbnails/4.jpg)
Recurrent cell
4
![Page 5: Seq2 seq learning](https://reader030.vdocument.in/reader030/viewer/2022021420/58cfa6411a28ab6b088b54ed/html5/thumbnails/5.jpg)
LSTM cell
5
![Page 6: Seq2 seq learning](https://reader030.vdocument.in/reader030/viewer/2022021420/58cfa6411a28ab6b088b54ed/html5/thumbnails/6.jpg)
Recurrent neural nets
6
C
●
●
![Page 7: Seq2 seq learning](https://reader030.vdocument.in/reader030/viewer/2022021420/58cfa6411a28ab6b088b54ed/html5/thumbnails/7.jpg)
Avoid explicit segmentation of the output sequence
●○○
●
7
![Page 8: Seq2 seq learning](https://reader030.vdocument.in/reader030/viewer/2022021420/58cfa6411a28ab6b088b54ed/html5/thumbnails/8.jpg)
Seq2Seq: intro
●
○ →
●
○ →
●
○
8
![Page 9: Seq2 seq learning](https://reader030.vdocument.in/reader030/viewer/2022021420/58cfa6411a28ab6b088b54ed/html5/thumbnails/9.jpg)
Seq2Seq: idea
9
I go to the zoo <EOS>
Je vais au zoo <EOS>
C
Encoder Decoder
Sequence to Sequence Learning with Neural Networks
![Page 10: Seq2 seq learning](https://reader030.vdocument.in/reader030/viewer/2022021420/58cfa6411a28ab6b088b54ed/html5/thumbnails/10.jpg)
Seq2Seq: Training
10
C
I go to the zoo <EOS>
Je vais au zoo <EOS>
Je vais au zoo
→ ∅ ∅ ∅ ∅ ∅
→ ∅ ∅ ∅ ∅ ∅
![Page 11: Seq2 seq learning](https://reader030.vdocument.in/reader030/viewer/2022021420/58cfa6411a28ab6b088b54ed/html5/thumbnails/11.jpg)
Seq2Seq: Inference
11
I go to the zoo <EOS>
Je vais
Je
au
vais
zoo
au
<EOS>
zoo
![Page 12: Seq2 seq learning](https://reader030.vdocument.in/reader030/viewer/2022021420/58cfa6411a28ab6b088b54ed/html5/thumbnails/12.jpg)
Seq2Seq: Inference with (pruned) beam search
12
Je
Elle
Ils
... <EOS>
allé
...
au
......
...
vais
suis
...
...
...
...
...
...
...
![Page 13: Seq2 seq learning](https://reader030.vdocument.in/reader030/viewer/2022021420/58cfa6411a28ab6b088b54ed/html5/thumbnails/13.jpg)
Attention mechanism
13
I go to the …... <EOS>
h0 h1 h2 h3 ... hn
c
MLPa0 a1 a2 a3
b0 b1 b2 b3
Je
Implemented in Tensorflow seq2seq()
Neural Machine Translation by Jointly Learning to Align and TranslateShow, Attend and Tell: Neural Image Caption Generation with Visual Attention
![Page 14: Seq2 seq learning](https://reader030.vdocument.in/reader030/viewer/2022021420/58cfa6411a28ab6b088b54ed/html5/thumbnails/14.jpg)
What we learned
●
●
●
14
![Page 15: Seq2 seq learning](https://reader030.vdocument.in/reader030/viewer/2022021420/58cfa6411a28ab6b088b54ed/html5/thumbnails/15.jpg)
Machine Translation
15
![Page 16: Seq2 seq learning](https://reader030.vdocument.in/reader030/viewer/2022021420/58cfa6411a28ab6b088b54ed/html5/thumbnails/16.jpg)
Image Captioning
16
VGG
14 x 14 x 512 196 x 512
Seq2Seq
196 sequences
![Page 17: Seq2 seq learning](https://reader030.vdocument.in/reader030/viewer/2022021420/58cfa6411a28ab6b088b54ed/html5/thumbnails/17.jpg)
Grammar parser
17
Grammar as a Foreign Language, NIPS 2014
![Page 18: Seq2 seq learning](https://reader030.vdocument.in/reader030/viewer/2022021420/58cfa6411a28ab6b088b54ed/html5/thumbnails/18.jpg)
Conversational bot
18
A Neural Conversational Model, ICML Workshop 2015
![Page 19: Seq2 seq learning](https://reader030.vdocument.in/reader030/viewer/2022021420/58cfa6411a28ab6b088b54ed/html5/thumbnails/19.jpg)
Skipthoughts
19
Skip-Thought Vectors, NIPS 2015
![Page 20: Seq2 seq learning](https://reader030.vdocument.in/reader030/viewer/2022021420/58cfa6411a28ab6b088b54ed/html5/thumbnails/20.jpg)
What else?●
○○○○○
●○○
●○○
20
![Page 21: Seq2 seq learning](https://reader030.vdocument.in/reader030/viewer/2022021420/58cfa6411a28ab6b088b54ed/html5/thumbnails/21.jpg)
What we learned
●
●
●○
○
○
○
21