what is the use of machine translation? exploring the post-editing … · 2017-05-30 ·...
TRANSCRIPT
What Is the Use of Machine Translation?Exploring the Post-Editing Process
Maarit Koponen
University of Turku
Conference on Translation and TechnologyMay 18, 2017
M. Koponen (University of Turku) What is the use of MT? May 18, 2017 1 / 16
MT in the news
http://www.theverge.com/2016/9/27/13078138/google-translate-ai-machine-learning-gnmt
http://gizmodo.com/xerox-photocopiers-can-now-translate-whatever-youre-cop-1764028266
https://www.extremetech.com/extreme/
228831-new-in-ear-device-could-be-the-beginning-of-a-world-without-language-barriers
https://www.wsj.com/articles/the-language-barrier-is-about-to-fall-1454077968
M. Koponen (University of Turku) What is the use of MT? May 18, 2017 2 / 16
MT in the past
M. Koponen (University of Turku) What is the use of MT? May 18, 2017 3 / 16
What is the use of MT?
M. Koponen (University of Turku) What is the use of MT? May 18, 2017 4 / 16
What is the use of MT?
M. Koponen (University of Turku) What is the use of MT? May 18, 2017 4 / 16
What is the use of MT?
M. Koponen (University of Turku) What is the use of MT? May 18, 2017 4 / 16
What is the use of MT?
M. Koponen (University of Turku) What is the use of MT? May 18, 2017 4 / 16
Post-editing
Given that Fully Automatic High Quality Translation (FAHQT) of anygiven text remains a distant possibility for the future, post-editing,usually understood as a human being (normally a translator)comparing a source text with the machine translation and makingchanges to it to make it acceptable for its intended purpose, remainsa standard feature of any serious machine translation system that will beused for commercial purposes.(Geoffrey S. Koby 2001, 1)
M. Koponen (University of Turku) What is the use of MT? May 18, 2017 5 / 16
MT and PE in the Finnish context
MT and Finnish
MT quality for Finnish lower compared to most large Europeanlanguages.
Rich morphology, productive compound words and relatively free wordorder cause problems for MT (see e.g. Koskenniemi et al. 2012).
The translator perspective: MT@EC maturity test (2011) amongEuropean Commission translators, percentage of MT sentencesassessed useful with editing:
I Spanish 95%I Swedish 85%I Estonian 65%I Finnish 40%
M. Koponen (University of Turku) What is the use of MT? May 18, 2017 6 / 16
MT and PE in the Finnish context
MT and Finnish
MT quality for Finnish lower compared to most large Europeanlanguages.
Rich morphology, productive compound words and relatively free wordorder cause problems for MT (see e.g. Koskenniemi et al. 2012).
The translator perspective: MT@EC maturity test (2011) amongEuropean Commission translators, percentage of MT sentencesassessed useful with editing:
I Spanish 95%I Swedish 85%I Estonian 65%I Finnish 40%
M. Koponen (University of Turku) What is the use of MT? May 18, 2017 6 / 16
MT and PE in the Finnish context
Post-editing in Finland
Not very common until recently, poor MT quality cited as reason (seee.g. Mikhailov 2015).
Use is increasing as technologies are being developed: governmentorganizations, language service providers, freelance translators.
MT errors and post-editing
In addition to the number of errors, the type of error may affectpost-editing effort.
I Temporal effort - time spent in post-editingI Technical effort - keystrokes, mouse operationsI Cognitive effort - identifying errors, planning corrections
Studies have shown that word form errors involve less effort thanword order errors, for example.
But does this apply to languages like Finnish?(see e.g. Koponen 2016; Krings 2001)
M. Koponen (University of Turku) What is the use of MT? May 18, 2017 7 / 16
MT and PE in the Finnish context
Post-editing in Finland
Not very common until recently, poor MT quality cited as reason (seee.g. Mikhailov 2015).
Use is increasing as technologies are being developed: governmentorganizations, language service providers, freelance translators.
MT errors and post-editing
In addition to the number of errors, the type of error may affectpost-editing effort.
I Temporal effort - time spent in post-editingI Technical effort - keystrokes, mouse operationsI Cognitive effort - identifying errors, planning corrections
Studies have shown that word form errors involve less effort thanword order errors, for example.
But does this apply to languages like Finnish?(see e.g. Koponen 2016; Krings 2001)
M. Koponen (University of Turku) What is the use of MT? May 18, 2017 7 / 16
Problem: What is an error, anyway?
“any feature of the translation which causes the posteditorto put pen to paper”
(Green 1982, 101)
Do people agree what needs to be corrected?
A study investigating the PE processes of 5 editors (English-Finnish MT)compared how often the editors changed the same word
Number of editors 5 4 3 2 1 0
Percentage of words 12% 11% 10% 8% 13% 45%Koponen & Salmi (forthcoming)
M. Koponen (University of Turku) What is the use of MT? May 18, 2017 8 / 16
Problem: What is an error, anyway?
“any feature of the translation which causes the posteditorto put pen to paper”
(Green 1982, 101)
Do people agree what needs to be corrected?
A study investigating the PE processes of 5 editors (English-Finnish MT)compared how often the editors changed the same word
Number of editors 5 4 3 2 1 0
Percentage of words 12% 11% 10% 8% 13% 45%Koponen & Salmi (forthcoming)
M. Koponen (University of Turku) What is the use of MT? May 18, 2017 8 / 16
Comparing different PE versions
Example
ST We will publish your name as you provide it (unless you ask us not to)
MT Julkaisemme nimenne niin kuin annatte sen (paitsi jos pyydattemeita olemaan)
MT paitsi jos pyydatte meita olemaanA paitsi jos pyydatte ettemme kayta sitaB paitsi jos erikseen kiellat senC paitsi jos pyydat meita olemaan julkaisemattaD paitsi jos kiellatte senE paitsi jos pyydatte meita kayttamatta
Koponen & Salmi (forthcoming)
M. Koponen (University of Turku) What is the use of MT? May 18, 2017 9 / 16
Investigating the correctness and necessity of edits
0
50
100
150
200
250
deleted inserted word form order
cor+nec cor+unnec incor+nec incor+unnec
Figure: Correctness and necessity by type of edit
Koponen & Salmi (forthcoming)M. Koponen (University of Turku) What is the use of MT? May 18, 2017 10 / 16
Pauses and editing events in the keylogging data
Edit Extended Production Deletion TimeEditor rate pauses events events (min)
A 42 54 774 404 23:19B 50 21 1092 260 20:07C 42 28 663 190 13:34D 38 38 544 169 15:43E 26 36 440 74 17:42
Table: Comparison of number of events and pauses in keylogging data(Extended pause: ≥ 5 s)
All 5 editors have an extended pause before the passage discussed inprevious example.
M. Koponen (University of Turku) What is the use of MT? May 18, 2017 11 / 16
Pauses and editing events in the keylogging data
Edit Extended Production Deletion TimeEditor rate pauses events events (min)
A 42 54 774 404 23:19B 50 21 1092 260 20:07C 42 28 663 190 13:34D 38 38 544 169 15:43E 26 36 440 74 17:42
Table: Comparison of number of events and pauses in keylogging data(Extended pause: ≥ 5 s)
All 5 editors have an extended pause before the passage discussed inprevious example.
M. Koponen (University of Turku) What is the use of MT? May 18, 2017 11 / 16
Problem: What happens during pauses?
M. Koponen (University of Turku) What is the use of MT? May 18, 2017 12 / 16
Eyetracking
M. Koponen (University of Turku) What is the use of MT? May 18, 2017 13 / 16
In conclusion
The use of machine translation and post-editing is increasing astechnology develops.
The quality of MT and feasibility of PE varies in different languagepairs.
Different types of MT errors may affect the PE process in differentways.
Findings and assumptions related to MT errors, PE processes andeffort are largely based on languages with relatively sparse morphology– more research is needed involving languages like Finnish andEstonian.
M. Koponen (University of Turku) What is the use of MT? May 18, 2017 14 / 16
In conclusion
The use of machine translation and post-editing is increasing astechnology develops.
The quality of MT and feasibility of PE varies in different languagepairs.
Different types of MT errors may affect the PE process in differentways.
Findings and assumptions related to MT errors, PE processes andeffort are largely based on languages with relatively sparse morphology– more research is needed involving languages like Finnish andEstonian.
M. Koponen (University of Turku) What is the use of MT? May 18, 2017 14 / 16
Thank you!
Questions? Comments?
M. Koponen (University of Turku) What is the use of MT? May 18, 2017 15 / 16
Literature
R. Green. 1982. The MT errors which cause most trouble to posteditors. V. Lawson, ed.Practical experience of machine translation. Proceedings of a conference, London, 5-6November 1981, 101–104. North Holland Publishing Company, Amsterdam.
GS. Koby. 2001. Editor’s Introduction – Post-Editing of Machine Translation Output:Who, What, Why, and How (Much). In Krings (2001), p. 1-23.
M. Koponen. 2016. Machine Translation Post-editing and Effort: Empirical Studies onthe Post-editing Process. Doctoral dissertation, University of Helsinki.
M. Koponen & L. Salmi. (forthcoming). Post-editing quality: Analyzing the correctnessand necessity of post-editor corrections. Accepted for publication in LinguisticaAntverpiensia, New Series: Themes in Translation Studies.
K. Koskenniemi et al. 2012. Suomen kieli digitaalisella aikakaudella – The FinnishLanguage in the Digital Age. G. Rehm & H. Uszkoreit (series editors). META-NET WhitePaper Series. Springer.
HP. Krings. 2001. Repairing texts: Empirical investigations of machine translationpost-editing process. GS Koby (ed./trans.). The Kent State University Press, Kent, OH.
M. Mikhailov. 2015. Minor language, major challenges: the results of a survey into the ITcompetences of Finnish translators. Journal of Specialised Translation 24:89-111
M. Plitt, F. Masselot. 2010. A productivity test of statistical machine translationpost-editing in a typical localisation context. Prague Bulletin of Mathematical Linguistics93:7-16
M. Koponen (University of Turku) What is the use of MT? May 18, 2017 16 / 16