a hierarchical encoder-decoder for paragraph...
TRANSCRIPT
A Hierarchical Encoder-Decoder for ParagraphSummarization
Farzaneh Mahdisoltani
Department of Computer ScienceUniversity of Toronto
CSC2541, Fall 2016
Farzaneh Mahdisoltani (University of Toronto)A Hierarchical Encoder-Decoder for Paragraph SummarizationCSC2541, Fall 2016 1 / 8
Paragraph Summarization
Summarization Phenomena:¯
Deletion
Paraphrase
Generalization
Farzaneh Mahdisoltani (University of Toronto)A Hierarchical Encoder-Decoder for Paragraph SummarizationCSC2541, Fall 2016 2 / 8
Related Work
Extractive Summarization: Stitching words and phrases from thesource
Dorr et al. (2003), Cohn et al. (2008), Woodsend et al. (2010)
Abstractive Summarization: Generating richer summaries based onunderstanding the input
Attention based: Rush et al. (2015)Statistical machine translation for headline generation: Banko et.al.(2000)RNN with attention: Chopra et al.(2016)Encoder-Decoder: Hu et al.(2015)
Farzaneh Mahdisoltani (University of Toronto)A Hierarchical Encoder-Decoder for Paragraph SummarizationCSC2541, Fall 2016 3 / 8
Hierarchichal Encoder-Decoder Model for Summarization
S1
S2
S3
S4
S5S6
Sentence to Vector enc(P)
Paragraph
enc(S1)
enc(S2)
enc(S3)
enc(S4)
enc(S5)enc(S6)
Paragraph to Vector
Sentence Level Decoder
enc(O1)
enc(O2)
enc(O3)
Word Level Decoder
O1
O2
O3
Summary Abstract Concepts
Farzaneh Mahdisoltani (University of Toronto)A Hierarchical Encoder-Decoder for Paragraph SummarizationCSC2541, Fall 2016 4 / 8
Hierarchichal Encoder-Decoder Model for Summarization
Objective Function:∑
t P(otw |o<tw , os , hp)
Encoder Decoder
WordLevel
SentenceLevel
h1i h2
i h3i h4
i
si
wi1 wi
2 wi3 wi
4
h1i h1
j h2j h3
j h4j h5
j
vj1 vj
2 vj3 vj
4 vj5
yj
h1s h2
s start
d1s
o1s
d1w d2w d3w
o1w o2w o3d
o1w o2w
h1i h1
j h2j h3
j h4j h5
j
wj1 wj
2 wj3 wj
4 wj5
sj
p
Farzaneh Mahdisoltani (University of Toronto)A Hierarchical Encoder-Decoder for Paragraph SummarizationCSC2541, Fall 2016 5 / 8
Selecting the Correct Summary from Candidates
S1S2S3S4S5S6
Sentence to Vector enc(P)
Paragraph
enc(S1)enc(S2)enc(S3)enc(S4)enc(S5)enc(S6)
Paragraph to Vector
enc(O1)enc(O2)enc(O3)
O1O2O3
Sentence to Vector enc(S1)
Paragraph to Vector
enc(O1)enc(O2)enc(O3)
O1O2O3
Sentence to Vector enc(S2)
Paragraph to Vector
enc(O1)enc(O2)enc(O3)
O1O2O3
Sentence to Vector enc(S3)
Paragraph to Vector
enc(O1)enc(O2)enc(O3)
O1O2O3
Sentence to Vector enc(S4)
Paragraph to Vector
Candidates
Farzaneh Mahdisoltani (University of Toronto)A Hierarchical Encoder-Decoder for Paragraph SummarizationCSC2541, Fall 2016 6 / 8
Preliminary Results
Paragraph Representation Method Accuracy
Random 25%Word2Vec 65%Skip-Thought Vectors 62%Bag of Words 67%RNN Encoder 57%Hierarchichal Encoder 54%
Table: Training the encoder for multiple choice question
Farzaneh Mahdisoltani (University of Toronto)A Hierarchical Encoder-Decoder for Paragraph SummarizationCSC2541, Fall 2016 7 / 8
Thank You
Farzaneh Mahdisoltani (University of Toronto)A Hierarchical Encoder-Decoder for Paragraph SummarizationCSC2541, Fall 2016 8 / 8