modern history for text summarization and beyondpfliu.com/talk/summarization.pdfsummarization and...
TRANSCRIPT
![Page 1: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/1.jpg)
Modern History for Text Summarization and Beyond
Pengfei LiuPostdoc at LTI of CMUpfliu.com
![Page 2: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/2.jpg)
Outline
Overview of Modern History
Some Highlighted Topics
Our Recent Work
![Page 3: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/3.jpg)
Outline
Overview of Modern History
Some Highlighted Topics
Our Recent Work
![Page 4: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/4.jpg)
Preparation: Research Papers
Summarization Papers:
• Year: 2013-Now
• Conference: ACL / EMNLP / NAACL / ICML / ICLR / AAAI / IJCAI / NeurIPS
![Page 5: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/5.jpg)
Preparation: Research Concepts
rich of task settings!
![Page 6: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/6.jpg)
Holistic Analysis
0
5
10
15
20
25
2013 2014 2015 2016 2017 2018 2019
# Pa
persACL EMNLP NAACL
2013-2015: increasing!
2015-2016: trough!
2016-2019: rapid increasing
![Page 7: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/7.jpg)
Fine-grained Analysis: Overview (2013-19)
![Page 8: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/8.jpg)
2013-2014 Task-setting Generation way
1) No NNs!2) First RL Paper3) Multimodal
![Page 9: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/9.jpg)
Top10 Cited Papers
![Page 10: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/10.jpg)
2015 Neural Arch.Task-setting Generation way
1) Finally, NNs arrived!
2) New dataset for NN!
![Page 11: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/11.jpg)
Top10 Cited Papers
![Page 12: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/12.jpg)
2016 ArchitectureTask-setting Generation way
1) Relatively more NNs2) Few papers
3) Important Techniques!
![Page 13: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/13.jpg)
2016 ArchitectureTask-setting Generation way
1) Relatively more NNs2) Few papers
3) Important Techniques!0
10
20
2013 2014 2015 2016 2017 2018 2019
# Pa
pers
ACL EMNLP NAACL
![Page 14: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/14.jpg)
Top10 Cited Papers
CNNDM
Copy
Coverage
Copy
Neural Ext
![Page 15: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/15.jpg)
Neural Arch.2017
2) VAE arrived!3) More NNs but not booming … lag behind …
1) GNNs arrived!
0
10
20
2013 2014 2015 2016 2017 2018 2019
# Pa
pers
ACL EMNLP NAACL
![Page 16: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/16.jpg)
Top10 Cited Papers
![Page 17: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/17.jpg)
2018: booming!
1) Booming of NNs2) Booming of RL!3) Booming of Abs!
![Page 18: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/18.jpg)
Top10 Cited Papers
![Page 19: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/19.jpg)
2019
1) Booming of Pre-training2) Booming of GNNs3) Booming of Evaluation
![Page 20: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/20.jpg)
Top10 Cited Papers
![Page 21: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/21.jpg)
Lessons from History
• The development of deep NNs lags behind other tasks
• Summarization tasks requires some customized techniques (e.g. copy)
• Only technique-ready is not enough … dataset also matters!
• A good match between “techniques” and “datasets”
![Page 22: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/22.jpg)
Outline
Overview of Modern History
Some Highlighted Topics
Our Recent Work
![Page 23: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/23.jpg)
Outline
Overview of Modern History
Some Highlighted Topics
Our Recent Work
FactualitySemantic equivalencePre-trained models
![Page 24: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/24.jpg)
Evaluation Metric: Factuality
• Event factuality prediction• the degree to which an event mentioned in a sentence has happened
• Social Media• fake news detection
• Dialog• consistency
• Machine Translation• semantic divergence
• Summarization
![Page 25: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/25.jpg)
Factuality in Text Summarization
• Motivation
Whether facts in the generated summaries can be covered
• GoalGiven a document , and generated summary: the purpose is to learn a function fact checker:
d 1 2g { , , , }
ms s s L
Fact(d,g)
Born in Honolulu, Hawaii, Obama is a US Citizen
Obama is American
Document Sentence?
![Page 26: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/26.jpg)
A Case Study on “FactCC” (Kryscinski et al.)
cnndm xsum arxiv pubmed bigpatent_b tifu_long0.413981 0.198641 0.438764 0.475539 0.637429 0.413868
Setting:
• Evaluating “FactCC” on (document, references) (Ideally, 100%)• Not good at predicting positive pairs.
![Page 27: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/27.jpg)
A Case Study on “FactCC” (Kryscinski et al.) Setting:
• Cross-dataset• Extractive model: higher factuality score (but not 100%)• Abstractive model: using a in-domain training set is not a
guarantee.
![Page 28: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/28.jpg)
Factuality in Text Summarization
• Challenges
• How to define a fact? (sentence? triple?)
• How to evaluate the effectiveness of your proposed
factuality checker?
• Source documents are too long!!!
• Negative predictive power
![Page 29: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/29.jpg)
More Recent Progress …
• Asking and Answering Questions to Evaluate the Factual Consistency of Summaries (Wang et al. 2020, ACL)
![Page 30: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/30.jpg)
More Recent Progress …
• Asking and Answering Questions to Evaluate the Factual Consistency of Summaries (Wang et al. 2020, ACL)
![Page 31: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/31.jpg)
Evaluation Metric: Semantic equivalence
• Growing Trends:
• BLEURT: Learning Robust Metrics for Text Generation (Sellam et al. 2020, ACL)
• BERTScore: Evaluating Text Generation with BERT(Zhang et al. 2019)
• MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance (Zhao et al. 2019)
• Pre-trained (+ fine-tuned) • Semantic Matching
• Learnable
• Examples:
• Fine-grained
![Page 32: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/32.jpg)
Growing Trend: Fine-grained Evaluation
• Fine-grained Meta-evaluation
• Evaluation metrics behave similarly (average-scoring range) strongly disagree in the higher-scoring range (Peyrard et al.2019)
• Calculating correlation in different top-K systems (Ma et al.2019)
![Page 33: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/33.jpg)
Unsupervised Pre-trained Models
Extractive
Abstractive
Summarization
NLU (BERT-family)
NLG (BART)
Pre-trained
![Page 34: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/34.jpg)
Unsupervised Pre-trained Models• Extractive
• Extractive -> Abstractive
• Exploring Task-specific Loss
![Page 35: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/35.jpg)
Outline
Overview of Modern History
Some Highlighted Topics
Our Recent Work
![Page 36: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/36.jpg)
Current Work
Interpretable Analysis
Model Refining
(1) Explicitly modelling inter-sentence interaction MATTERS!
(2) Summary-level optimization is rewarding
(1) Heterogeneous Graph Neural Networks for ExtractiveDocument Summarization (ACL 2020)
(2) Extractive Summarization as Text Matching (ACL 2020)
Searching for Effective Neural Extractive Summarization: What Works and What’s Next (ACL 2019)
![Page 37: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/37.jpg)
Interpretable Analysis: Why?
65
70
75
80
85
90
1 2 3 4 5 6 7 8 9 10 11
Change trend of scores on GLUE
38.539
39.540
40.541
41.542
Nal lapat i
et …
Narayan e
t…
Chen et…
Zhou et…
Xu et a
l .2019
Change trend of R-1 on a Text Summarization Dataset
The performances of many NLP tasks have begun to plateau
gradually flattening.
![Page 38: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/38.jpg)
Interpretable Analysis: Why?
• Superior performance but low interpretability
• If the pros and cons are unknown, how could we:
• make suitable choices under different scenarios
• design more powerful methods
![Page 39: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/39.jpg)
Interpretable Analysis: How?
• Explaining the prediction behavior
• Understanding the functionality of a neural component
• Revealing the darkness of pre-trained models
![Page 40: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/40.jpg)
Methodology for Understanding NLP-oriented Models
Training-Testing environment:
• Training environment: different models are first generated with different specifications
• Testing environment : a model should be evaluated with different observed testbeds
![Page 41: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/41.jpg)
Bring it into Text Summarization
• Training environment• Testing environment
Searching for Effective Neural Extractive Summarization: What Works and What’s Next (ACL 2019)
![Page 42: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/42.jpg)
Bring it into Text Summarization
![Page 43: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/43.jpg)
Bring it into Text Summarization
• Training environment
• Testing environment
![Page 44: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/44.jpg)
Bring it into Text Summarization
Besides the “Rouge”
• Other metrics:• Positional bias• Repetition• Sentence length
• Evaluation testbeds:• Cross-domain evaluation (eight domains)• Sentence shuffling
observe from different aspects
![Page 45: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/45.jpg)
Explicitly modelling inter-sentence interaction MATTERS!
The gap between datasets heavily influences the cross-datasets generalization
Summary-level optimization is rewarding
Takeaways
![Page 46: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/46.jpg)
Explicitly modelling inter-sentence interaction MATTERS!
The gap between datasets heavily influences the cross-datasets generalization
Summary-level optimization is rewarding
Takeaways
![Page 47: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/47.jpg)
Heterogeneous Graph Summarizer
Heterogeneous Graph Neural Networks for ExtractiveDocument Summarization (ACL 2020)
words
sentences
Sequential order
![Page 48: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/48.jpg)
Heterogeneous Graph Summarizer
Heterogeneous Graph Neural Networks for ExtractiveDocument Summarization (ACL 2020)
Graph-structure
![Page 49: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/49.jpg)
Heterogeneous Graph Summarizer
Heterogeneous Graph Neural Networks for ExtractiveDocument Summarization (ACL 2020)
Graph:• Node: word, sentence, document• Edge: tf-idf
![Page 50: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/50.jpg)
Advantages
1) Aware of overlapping information
2) Both words and sentences keep themselves updated
3) Flexibly extended• Relay node: entity• Satellite node: document
![Page 51: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/51.jpg)
Results
![Page 52: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/52.jpg)
Explicitly modelling inter-sentence interaction MATTERS!
The gap between datasets heavily influences the cross-datasets generalization
Summary-level optimization is rewarding
Takeaways
![Page 53: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/53.jpg)
Matching-based Summarization
Extractive Summarization as Text Matching (ACL 2020)
Extracting sentences as a sequence labeling problem
![Page 54: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/54.jpg)
Matching Summarization
Extractive Summarization as Text Matching (ACL 2020)
1) paradigm shift with regard to the way we build neural extractive summarization systems
3) bypasses the difficulty of summary-level optimization (e.g. RL) by contrastive learning
2) a good summary should be more semantically similar to the source document than the unqualified summaries
![Page 55: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/55.jpg)
Optimization Principles
![Page 56: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/56.jpg)
Experiment
![Page 57: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/57.jpg)
Beyond a SOTA result
Theoretical Analysis
• On what types of datasets, the expected gain of summary-level approach is large over sentence-level approach?
• And how to characterize the expected gain?
Outlook
• The power of matching framework has not been fully exploited
• Build the connections with learnable evaluation metrics:• BERTScore• MoverScore• BLEURT
![Page 58: Modern History for Text Summarization and Beyondpfliu.com/talk/summarization.pdfSummarization and Beyond Pengfei Liu Postdoc at LTI of CMU pfliu.com. Outline Overview of Modern History](https://reader034.vdocument.in/reader034/viewer/2022042307/5ed3fc508d46b66d226337ec/html5/thumbnails/58.jpg)
Thank you