zero-shot sequence labeling: transferring knowledge from ...zero-shot sequence labeling 3 + + + - +...
TRANSCRIPT
![Page 1: Zero-shot Sequence Labeling: Transferring Knowledge from ...Zero-shot sequence labeling 3 + + + - + + + + + - + I like to playing the guitar and sing very louder . It was so long time](https://reader034.vdocument.in/reader034/viewer/2022051905/5ff72d0f4cfd855de958e5de/html5/thumbnails/1.jpg)
1
Zero-shot Sequence Labeling:Transferring Knowledge
from Sentences to TokensMarek Rei Anders Søgaard
![Page 2: Zero-shot Sequence Labeling: Transferring Knowledge from ...Zero-shot sequence labeling 3 + + + - + + + + + - + I like to playing the guitar and sing very louder . It was so long time](https://reader034.vdocument.in/reader034/viewer/2022051905/5ff72d0f4cfd855de958e5de/html5/thumbnails/2.jpg)
2
Sequence labeling
_ _ C C _ _ _ _ _ _
Our data indicate that increased NF-kappa B DNA binding is ...
Hedge cue detection:
+ + + - + + + + + - +
I like to playing the guitar and sing very louder .
Error detection:
![Page 3: Zero-shot Sequence Labeling: Transferring Knowledge from ...Zero-shot sequence labeling 3 + + + - + + + + + - + I like to playing the guitar and sing very louder . It was so long time](https://reader034.vdocument.in/reader034/viewer/2022051905/5ff72d0f4cfd855de958e5de/html5/thumbnails/3.jpg)
3
Zero-shot sequence labeling
+ + + - + + + + + - +
I like to playing the guitar and sing very louder .
It was so long time to wait in the theatre .
I look forward to receiving reply to my enquiry .
This is a great opportunity to learn more about whales .
Therefore, houses will be built on high supports .
![Page 4: Zero-shot Sequence Labeling: Transferring Knowledge from ...Zero-shot sequence labeling 3 + + + - + + + + + - + I like to playing the guitar and sing very louder . It was so long time](https://reader034.vdocument.in/reader034/viewer/2022051905/5ff72d0f4cfd855de958e5de/html5/thumbnails/4.jpg)
4
Main idea
Neural sentence classification model01
02
03 Make attention weights behave like sequence labeling output
Based on self-attention
![Page 5: Zero-shot Sequence Labeling: Transferring Knowledge from ...Zero-shot sequence labeling 3 + + + - + + + + + - + I like to playing the guitar and sing very louder . It was so long time](https://reader034.vdocument.in/reader034/viewer/2022051905/5ff72d0f4cfd855de958e5de/html5/thumbnails/5.jpg)
5
Model architecture
![Page 6: Zero-shot Sequence Labeling: Transferring Knowledge from ...Zero-shot sequence labeling 3 + + + - + + + + + - + I like to playing the guitar and sing very louder . It was so long time](https://reader034.vdocument.in/reader034/viewer/2022051905/5ff72d0f4cfd855de958e5de/html5/thumbnails/6.jpg)
6
Based on softmax:
Based on sigmoid + normalisation:
Soft attention weights
![Page 7: Zero-shot Sequence Labeling: Transferring Knowledge from ...Zero-shot sequence labeling 3 + + + - + + + + + - + I like to playing the guitar and sing very louder . It was so long time](https://reader034.vdocument.in/reader034/viewer/2022051905/5ff72d0f4cfd855de958e5de/html5/thumbnails/7.jpg)
7
Optimising the attention
We can constrain the attention values based on the sentence-level label.
1. Only some, but not all, tokens in the sentence can have a positive label.
2. There are positive tokens in a sentence only if the overall sentence is positive.
![Page 8: Zero-shot Sequence Labeling: Transferring Knowledge from ...Zero-shot sequence labeling 3 + + + - + + + + + - + I like to playing the guitar and sing very louder . It was so long time](https://reader034.vdocument.in/reader034/viewer/2022051905/5ff72d0f4cfd855de958e5de/html5/thumbnails/8.jpg)
8
Alternative methods
1. Labeling through backpropagation
2. Relative frequency
3. Supervised sequence labeling
Selvaraju et al (2016)
![Page 9: Zero-shot Sequence Labeling: Transferring Knowledge from ...Zero-shot sequence labeling 3 + + + - + + + + + - + I like to playing the guitar and sing very louder . It was so long time](https://reader034.vdocument.in/reader034/viewer/2022051905/5ff72d0f4cfd855de958e5de/html5/thumbnails/9.jpg)
9
Evaluation: CoNLL 2010Detection of uncertain language in scientific articles
![Page 10: Zero-shot Sequence Labeling: Transferring Knowledge from ...Zero-shot sequence labeling 3 + + + - + + + + + - + I like to playing the guitar and sing very louder . It was so long time](https://reader034.vdocument.in/reader034/viewer/2022051905/5ff72d0f4cfd855de958e5de/html5/thumbnails/10.jpg)
10
Evaluation: FCEDetecting grammatical errors in essays written by language learners.
![Page 11: Zero-shot Sequence Labeling: Transferring Knowledge from ...Zero-shot sequence labeling 3 + + + - + + + + + - + I like to playing the guitar and sing very louder . It was so long time](https://reader034.vdocument.in/reader034/viewer/2022051905/5ff72d0f4cfd855de958e5de/html5/thumbnails/11.jpg)
11
Examples
![Page 12: Zero-shot Sequence Labeling: Transferring Knowledge from ...Zero-shot sequence labeling 3 + + + - + + + + + - + I like to playing the guitar and sing very louder . It was so long time](https://reader034.vdocument.in/reader034/viewer/2022051905/5ff72d0f4cfd855de958e5de/html5/thumbnails/12.jpg)
12
Applications
Sequence labeling without data01
Data exploration and feature analysis02
Model visualisation and interpretation03
![Page 13: Zero-shot Sequence Labeling: Transferring Knowledge from ...Zero-shot sequence labeling 3 + + + - + + + + + - + I like to playing the guitar and sing very louder . It was so long time](https://reader034.vdocument.in/reader034/viewer/2022051905/5ff72d0f4cfd855de958e5de/html5/thumbnails/13.jpg)
13
Thank you! Any questions?