style transfer through back-translationsprabhum/docs/style_transfer.pdf · style transfer the...
TRANSCRIPT
![Page 1: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/1.jpg)
Style Transfer Through Back-Translation
Shrimai Prabhumoye, Yulia Tsvetkov, Ruslan Salakhutdinov, Alan W Black
![Page 2: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/2.jpg)
What is Style Transfer ● Rephrasing the text to contain specific stylistic
properties without changing the intent or affect within the context.
![Page 3: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/3.jpg)
What is Style Transfer ● Rephrasing the text to contain specific stylistic
properties without changing the intent or affect within the context.
“Shut up! the video is starting!”
“Please be quiet, the video will begin shortly.”
![Page 4: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/4.jpg)
Motivation
I have an exam today.
May the Force be with you!
Best of Luck!SiriUser
![Page 5: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/5.jpg)
User Adaptation
I am frustrated with work. My models are
not working!
SiriUser
No! Try not! Do or do not, there
is no try!
![Page 6: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/6.jpg)
User Adaptation
I am frustrated with work. My models are
not working!
SiriUser
Have the courage to follow your heart and
intuition. They somehow know what
you truly want to become.
![Page 7: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/7.jpg)
Applications
![Page 8: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/8.jpg)
Anonymization● To preserve anonymity of users online, for personal security concerns
(Jardine, 2016), or to reduce stereotype threat (Spencer et al., 1999).
![Page 9: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/9.jpg)
Balanced Data● Demographically-balanced training data for downstream applications.
![Page 10: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/10.jpg)
To create a representation that is devoid of style but holds the meaning of the input sentence.
Our Goal
![Page 11: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/11.jpg)
Challenges
Content Style
![Page 12: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/12.jpg)
Challenges● No Parallel Data! ● “The movie was very long.” ● “I entered the theatre in the bloom of youth
and emerged with a family of field mice living in my long, white mustache.”
● Hard to detect style
![Page 13: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/13.jpg)
● Back-Translation
● Translating an English sentence to a pivot
language and then back to English.
● Reduces the stylistic properties
● Helps in grounding meaning
Our Solution
![Page 14: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/14.jpg)
Overview
How it works? How to train? Evaluation
![Page 15: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/15.jpg)
MT e fencoder decoder
Architecture
![Page 16: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/16.jpg)
MT e fencoder decoder
I thank you, Rep. Visclosky
je vous remercie, Rep. Visclosky
Architecture
![Page 17: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/17.jpg)
MT e fencoder decoder
I thank you, Rep. Visclosky
je vous remercie, Rep. Visclosky
MT f eencoder decoder
Architecture
![Page 18: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/18.jpg)
MT e fencoder decoder
MT f eencoder
I thank you, Rep. Visclosky
je vous remercie, Rep. Visclosky
Architecture
![Page 19: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/19.jpg)
MT e fencoder decoder
MT f eencoder
I thank you, Rep. Visclosky
je vous remercie, Rep. Visclosky
Style 1decoder
Style 2decoder
I thank you, senator Visclosky
I’m praying for you sir.
Architecture
![Page 20: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/20.jpg)
Overview
How it works? How to train? Evaluation
![Page 21: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/21.jpg)
Style 1decoder
Style 2decoder
Train Pipeline
![Page 22: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/22.jpg)
Style 1decoder
Style 2decoder
classifier
Train Pipeline
![Page 23: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/23.jpg)
Style 1decoder
Style 2decoder
classifier
Train Pipeline
![Page 24: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/24.jpg)
● Encoder-Decoders follow sequence-to- sequence framework (Sutskever et al., 2014; Bahdanau et al., 2015)
Experimental Settings
![Page 25: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/25.jpg)
Baseline
![Page 26: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/26.jpg)
● WMT 15 data ● News, Europarl and Common Crawl ● ~5M parallel English - French sentences
Neural Machine Translation
Model BLEU
English - French 32.52
French - English 31.11
![Page 27: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/27.jpg)
Style Tasks
Task Labels Corpus
Gender Male, Female Yelp
Political Slant Republican, Democratic Facebook Comments
Sentiment Modification Negative, Positive Yelp
![Page 28: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/28.jpg)
Overview
How it works? How to train? Evaluation
![Page 29: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/29.jpg)
● Style Transfer Accuracy
● Meaning Preservation
● Fluency
Evaluation
![Page 30: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/30.jpg)
● generated sentences are evaluated using a pre-trained style classifier ● Transfer the style of test sentences and test the classification
accuracy of the generated sentences for the desired label.
Classifier Model Accuracy
Gender 82%
Political Slant 92%
Sentiment Modification 93.23%
Style Transfer Accuracy
![Page 31: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/31.jpg)
Style Transfer Accuracy
![Page 32: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/32.jpg)
● Human Annotation: A/B Testing ● The annotators are given instructions. ● Annotators are presented with the original sentence.
Preservation of Meaning
A B =
![Page 33: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/33.jpg)
● “Which transferred sentence maintains the same sentiment of the source sentence in the same semantic context (i.e. you can ignore if food items are changed)”
● “Which transferred sentence maintains the same semantic intent of the source sentence while changing the political position”
● “Which transferred sentence is semantically equivalent to the source sentence with an opposite sentiment”
Instructions
![Page 34: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/34.jpg)
Preservation of Meaning
![Page 35: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/35.jpg)
● Human annotators were asked to annotate the generated sentences for fluency on a scale of 1-4.
● 1: Unreadable ● 4: Perfect
Fluency
![Page 36: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/36.jpg)
Fluency
![Page 37: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/37.jpg)
● The loss function of the generators includes two competing terms, one to improve meaning preservation and the other to improve the style transfer accuracy.
● Sentiment modification task is not well-suited for evaluating style transfer
● The style-transfer accuracy for gender is lower for BST model but the preservation of meaning is much better for the BST model, compared to CAE model and to “No preference” option.
Discussion
![Page 38: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/38.jpg)
● Male -- Female
my wife ordered country fried steak and eggs.
My husband ordered the chicken salad and the fries.
● Female -- Male
Save yourselves the huge headaches,
You are going to be disappointed.
Gender Examples
![Page 39: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/39.jpg)
● Republican -- Democratic
I will continue praying for you and the decisions made by our government!
I will continue to fight for you and the rest of our democracy!
● Democratic -- Republican
As a hoosier, I thank you, Rep. Vislosky.
As a hoosier, I’m praying for you sir.
Political Slant Examples
![Page 40: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/40.jpg)
● Negative -- Positive
This place is bad news!
This place is amazing!
● Positive -- Negative
The food is excellent and the service is exceptional!
The food is horrible and the service is terrible.
Sentiment Modification Examples
![Page 41: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/41.jpg)
● Enhance back-translation by pivot through several languages ○ to learn a better grounded latent meaning
representation. ● Use multiple target languages with single source
language as described in (Johnson et al., 2016) to see whether pivoting via multiple languages captures better semantic representations.
Future Directions
![Page 42: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/42.jpg)
● Deploy the system in a real world conversational agent to analyze the effect on user satisfaction
● Caring for more styles!
Future Directions
![Page 43: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/43.jpg)
Thank You
![Page 44: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/44.jpg)
● Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural net- works. In Proc. NIPS, pages 3104–3112.
● Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Ben- gio. 2015. Neural machine translation by jointly learning to align and translate. In Proc. ICLR.
● Eric Jardine. 2016. Tor, what is it good for? political repression and the use of online anonymity-granting technologies. New Media & Society.
● Steven J. Spencer, Claude M. Steele, and Diane M. Quinn. 1999. Stereotype Threat and Women’s Math Performance. Journal of Experimental Social Psy- chology, 35:4–28.
References
![Page 45: Style Transfer Through Back-Translationsprabhum/docs/Style_Transfer.pdf · style transfer The style-transfer accuracy for gender is lower for BST model but the preservation of meaning](https://reader036.vdocument.in/reader036/viewer/2022062508/604e9e29d638af6cb96f607b/html5/thumbnails/45.jpg)
● Melvin Johnson, Mike Schuster, Quoc V Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Vie ́gas, Martin Wattenberg, Greg Corrado, et al. 2016. Google’s multilingual neural machine translation system: enabling zero-shot translation. arXiv preprint arXiv:1611.04558.
● Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2017. Style transfer from non-parallel text by cross-alignment. In Proc. NIPS.
References