taus review#4 - the innovation issue - july 2015
DESCRIPTION
ÂTRANSCRIPT
1
Reviews of Language Business and Technology in Asia and Africa. Columns by Nicholas Ostler, Lane Greene and Luigi Muzii and guest
author John Moran.
The Innovation Issue
of language business and technologyTAUS REVIEW
July 2015 - No. IV
PLUS an interesting use case from Intel.
2
I will inform you after checking An artificial intelligence approach to translation will revolutionize your business.
our stock.
I will inform you after checking An artificial intelligence approach to translation will revolutionize your business.
our stock.
An artificial intelligence approach to translationwill revolutionize your business.
To realize a society in which everyone can interact freelyacross language barriers with the use of machine translation technology, and thereby contribute to invigoration and innovation in businesses.
https://miraitranslate.com/en/
3
How do we communicate in an ever more globalizing world? Will we all learn to speak the same language? A lingua franca, English, Chinese, Spanish? Or will we rely on translators to help us bridge the language divides?
Language business and technology are core to the world economy and to the prevailing trend of globalization of business and governance. And yet, the language sector, its actors and innovations do not get much visibility in the media. Since 2005 TAUS has published numerous articles on translation automation and language business innovation on its web site. Now we are bundling them in TAUS Review, an online quarterly magazine.
TAUS Review is a magazine with a mission. We believe that a vibrant language and translation industry helps the world communicate better, become more prosperous and more peaceful. Communicating across hundreds – if not thousands – of languages requires adoption of technology. In the age of the Internet of Things and the internet of you, translation – in every language – becomes embedded in every app, on every screen, on every web site, in every thing.
In TAUS Review reporters and columnists worldwide monitor how machines and humans work together to help the world communicate better. We tell the stories about the successes and the excitements, but also about the frustrations, the failures and shortcomings of technologies and innovative models. We are conscious of the pressure on the profession, but convinced that language and translation technologies lead to greater opportunities.
TAUS Review follows a simple and straightforward structure. In every issue we publish reports from four different continents – Africa, Americas, Asia and Europe – on new technologies, use cases and developments in language business and technology from these regions. In every issue we also publish perspectives from four different ‘personas’ – researcher, journalist, translator and language – by well-known writers from the language sector. This is complemented by features and conversations that are different in each issue.
The knowledge we share in TAUS Review is part of the ‘shared commons’ that TAUS develops as a foundation for the global language and translation market to lift itself to a high-tech sector. TAUS is a think tank and resource center for the global translation industry, offering access to best practices, shared translation data, metrics and tools for quality evaluation, training and research.
Colophon
TAUS Review is a free online magazine, published four times per year. TAUS members and non-members may distribute the magazine through their web sites and online media. Please write to [email protected] for the embed code. TAUS Review currently has about 7,000 readers globally.
Publisher & managing editor: Jaap van der Meer Editor & publication manager, distribution and advertisements: Anne-Maj van der Meer
Enquiries about distribution and advertisements: [email protected]
Editorial contributions and feedback can be sent to:General: [email protected] reviews of language business and technology: • Africa review: [email protected]• Americas review: [email protected]• Asia review: [email protected]• Europe review: [email protected]’s perspectives of language business and technology:• Translator: [email protected] • Research: [email protected] • Language: [email protected]• Journalist: [email protected]
Magazine with a MissionI will inform you after checking An artificial intelligence approach to translation will revolutionize your business.
our stock.
I will inform you after checking An artificial intelligence approach to translation will revolutionize your business.
our stock.
An artificial intelligence approach to translationwill revolutionize your business.
To realize a society in which everyone can interact freelyacross language barriers with the use of machine translation technology, and thereby contribute to invigoration and innovation in businesses.
https://miraitranslate.com/en/
4
Leader
5. Leader by Jaap van der Meer
Reviews of language business & technologies
8. In Asia by Mike Tian-Jian Jiang
12. In Africa by Amlaku Eshetie
Columns
16. The Journalist’s Perspective by Lane Greene
18. The Language Perspective by Nicholas Ostler
21. The Translator’s Perspective by John Moran
25. The Research Perspective by Luigi Muzii
Features
30. Unmoderated Remote Usability Testing of Machine Translated Content by Will Burgett
38. Contributors
40. Directory of Distributors
43. Industry Agenda
Content
5
Leaderby Jaap van der Meer
No Government, No Innovation
“With all respect to the companies innovating in language today”, says Lane Greene in his column in this issue of TAUS Review, “no-one has achieved anything like that roll-up…” He refers to Uber and “Uberization”, a neologism that has become synonym for disruption of traditional and fragmented industries.
Nothing of the same scale is taking place yet in the global translation industry. Mergers and
acquisitions in our sector aim at increasing market share, but up to now they have not lead to real innovation. Where then is the real big innovation in the
translation industry coming from, we asked our columnists. From insiders or invaders?
Luigi Muzii refers to the conclusion that Mariana Mazzucato arrived at in her book “The Entrepreneurial State”: no government, no innovation. We’d like to think perhaps that the companies we admire bring us great inventions, but in fact the research underlying them is often funded by governments. One example for all: the ideas and mechanics behind Google Translate go back to, among others, Verbmobil, a research project funded by the German Federal Ministry of Education, Science, Research and Technology until the year 2000.
The project was aimed at giving Germany a top international position in language technology
and its economical application in the next millennium. But, as I wrote in my recent blog article “The Brains but not the Guts”, very often, at the end of a project, researchers join the companies that have the guts to make it work.
What the global translation industry needs now is ambitious government-funded research programs that exceed national borders. In analogue to the Human Genome Project, in the past we coined the idea of a Human Language Project in various TAUS conferences and articles. The Human Genome Project – after thirteen years of research funded by European and US governments – delivered the first documented human DNA in 2003 and since then the project ignited a revolution in human science. The original funds amounted to 2.7 Billion US Dollars. According to some reports, the benefits of the Human Genome Project now already hover around 1 Trillion US Dollars, or $178 for every public dollar spent. The discoveries delivered by the Human Genome Project have incubated a company like Illumina. Illumina’s machines for sequencing human DNA potentially rewrite the way we manage healthcare in the world. Disruptive innovation in optima forma.
In the meantime, the translation industry is just spinning its wheels. Localization project managers are burnt out on repetitive jobs and highly educated translators are exploited to produce more for less. The sector prides itself as an enabler for globalization, but it is just scratching the surface. Ninety percent of the content and ninety percent of
Where is the real big innovation in the translation
industry coming from? Insiders or
invaders?
What the global
translation industry
needs now is ambitious government-
funded research programs
that exceed national borders.
The TranslaTion indusTry is quickly becoming a high-Tech indusTry. so we say… buT fear noT. TranslaTor’s jobs are noT going away. alThough google TranslaTe may be considered a big innovaTion, The company keeps hunTing for more and beTTer human TranslaTors who can produce The mosT readable and besT localizaTions of iTs producTs.
6
Leaderby Jaap van der Meer
the languages remain untouched, because…. Well, because of the lack of innovation and automation.
Although making its strides into the sector, machine translation is still lacking the confidence that it will deliver consistent quality. Breakthroughs that require more than the bravery of even the most daring commercial players are needed. It seems that governments everywhere now realize the crucial roles they have to play in overcoming the traditional trade barriers. Japan is injecting the industry
with funding for the collection of translation data. With the Olympic Games coming to Tokyo in 2020, Japan wants to surprise the world with a population happily conversing in the languages of the tourists and
business people visiting the country. (See TAUS articles on translation innovation in Japan)
In Europe, the focus is on the Digital Single Market: the EC’s Connecting European Facility (CEF) is a major infrastructural funding program covering all things – like broadband internet, roads and bridges as well as automatic translation – that will help to truly open the internal European market. Bringing down the barriers within Europe (language being one of these barriers) could contribute an additional EUR 415 Billion to the European economy. And the “One Belt – One Road” program in China is another example of big thinking. Like the CEF program, the “Silk Road Economic Belt” program in China is aimed at removing physical, financial, cultural and language barriers.
There is no doubt that these major government funding programs will drip down in innovations in translation. The most ambitious governments are likely to stimulate the biggest innovations. With the 40 Billion US Dollars that General
Secretary Xi Jinping announced in November 2014 to invest in the Silk Road Economic Belt it looks like China may soon be driving more innovation in the translation industry than other economies.
Our dream of the Human Language Project, the disruptive i n n o v a t i o n , that would f u n d a m e n t a l l y change the way we communicate on this planet, the type of innovation that opens trillion dollar plus economic opportunities and rewrite the rules in human evolution, requires that all g o v e r n m e n t s collaborate. Because this is not very likely to happen we keep our feet on the ground and invite both industry insiders and invaders to come and present their best examples of innovation at the TAUS Annual Conference in San Jose, CA in October. See the Call for the TAUS Innovation Excellence Awards.
Machine translation is
still lacking the confidence that it will deliver consistent
quality.
Send your comments or questions to [email protected]
The type of innovation that opens
trillion dollar plus economic opportunities
and rewrite the rules in human
evolution, requires that all
governments collaborate.
7
A thousand different workflows. One Solution.The Business Management System for the Translation Industry
www.plunet.com
8
Review of language business & technologies in Asiaby Mike Tian-Jian Jiang
Translation innovations that are leading the change and may help us surviving the change
By decorating an adjective “disruptive”, however, doesn’t really weigh clearer degrees on the spectrum of innovation, hence it is common to transform the question to the degree of inefficiency: how efficient is the target market1? Here we adapt this viewpoint for the market of translation, and then try to identify certain disruptive innovations that might be game-changers, in particular two factors of inefficiencies: human-resource and software ability.
Inefficient supply-demand match makingCrowd-sourcing translation services has been quite popular in the recent couple of years. On one hand, crowd translation indeed increased supply, and potentially decreased the price. On the other hand, the information asymmetry
between buyers and sellers remains almost the same. Customers may still find it difficult to locate the best translator who will yield the expected outcome, and vice versa.
While it is a general question to most businesses, translation can be even tougher when the work of conduct requires further “translation”. Typical tactics to tackle this problem include categorizing source texts, relying on translators’ reputations, assuring quality by post-editing and/or proofreading, etc.
1 https://medium.com/@moonstorming/why-inefficiency-is-needed-for-an-innovation-to-be-disruptive-8994845b09d9
What if a match making can be done systematically and semi-automatically? That is one of the questions DuoLingo is trying to answer.
Luis von Ahn, the creator of reCAPTCHA, started DuoLingo. According to the very first sentence of its Wikipedia page, DuoLingo is “a free language-learning and crowdsourced text translation platform.” To comprehend how language-learning and crowd translation align, it might be easier when we first understand how reCAPTCHA works. Like the original CAPTCHA, reCAPTCHA asks web surfers to type words shown in images, to prevent bots accessing restricted web pages.
Unlike CAPTCHA, r e C A P T C H A presents two sets of word images. One of which the answer is known in the system and another is not yet “translated”. Optical Character Recognition (OCR) doesn’t work on images like this. The idea is to utilize the wasted human effort on entering CAPTCHA, so the entered
innovaTion is uneasy when iT comes To differenTiaTion. There are many differenT kinds of innovaTions, ranging from improved cusTomer saTisfacTion To invenTed markeT demands, and in Terms of The recenT Trend, The “disrupTive” kind is The mosT welcomed innovaTion.
What if a match making can be done
systematically and semi-
automatically?
reCAPTCHA eases the pain
of normal validation that
is usually labor-intensive.
9
Review of language business & technologies in Asiaby Mike Tian-Jian Jiang
words can actually be the answer of failed OCR attempts. When it comes to the efficiency of digitalizing paper-based archives, reCAPTCHA eases the pain of normal validation that is usually labor-intensive. One may ask that if the hard-to-OCR text image didn’t come with an answer, how can reCAPTCHA be trustworthy? The trick is that if the known image got the correct result, it is usually true for the unknown one. Furthermore, the same unknown image can be presented to different users, so a voting system could help as well. By replacing every appearance of image, word/text, and OCR on the above description with source language, target language, and translation, it will start making sense to remodel reCAPTCHA into DuoLingo2.
One obvious problem with remodeling reCAPTCHA into a translation service is that a bilingual web surfer is as much an urban legend as a monolingual one. So DuoLingo began as a free language-learning platform with
gamification traits, with successfully significant amount of learners. The top learners were asked to translate some new texts and vote on other people’s translations. New texts could then be used as new learning materials.
As DuoLingo revealed, the demand of translation is currently from news sites like CNN and BuzzFeed. Despite the fact that this business model is still in its early stage, it will be not surprising that eventually texts in different genres and domains will join this learning-translation feedback.
Recently DuoLingo launched a language certification service and, in my opinion, this complimentary business model will strengthen the learning-translation feedback and build a
2 http://www.businessinsider.com/luis-von-ahn-creator-of-duolingo-recaptcha-2014-3
healthy cycle by inverting the supply-demand dependency every once in a while, so buyers and sellers will both know what they can get.
For example, BuzzFeed first earned some extra bucks by selling their translated news articles to DuoLingo as learning materials, and later DuoLingo may sell more translated news articles back to BuzzFeed with quality no worse than the learning materials, while learners may get certified and then motivated to join the translation industry.
Inefficient corpus annotation for machine translation model trainingIf we swap reCAPTCHA’s image, text, and OCR elements with “machine” translation terms, would it be as promising as DuoLingo? One major issue lies in the preparation of learning materials, or annotated corpus in terms of machine translation.
For DuoLingo, it is reasonable to leave the alignment problem (as in determining which source phrases correspond to which target
The demand of translation is currently from news sites like CNN and BuzzFeed.
One major issue lies in
the preparation of learning materials.
10
Review of language business & technologies in Asiaby Mike Tian-Jian Jiang
phrase) to learners. For common machine translation training, however, it is required to include alignment as one of the corpus curation tasks. We want machine translation models to remember minimal phrases, and their translations, to fluently generate full translations from them no matter whether the exact source sentence is seen or not, just like what qualified translators can do. One may notice how similar this task sounds as in reCAPTCHA-to-OCR model training scenario. The unfortunate difference here between OCR and machine translation is that phrase alignment is not always straightforward.
One component from a source phrase could be missing in its target phrase, not to mention that not every language in the world comes with whitespace delimited word boundaries. Typical machine translation modeling nowadays involves a module of automatic alignment, such
as GIZA, which requires carefully handcrafted training data. Something that needs careful handcrafting often implies inefficiency, and an even worse situation of alignment is that it has to be done by experts, and the market of this kind of experts is limited, so either reCAPTCHA or DuoLingo won’t help much.
What if machine t r a n s l a t i o n modeling can get rid of the burden of a l i g n m e n t ? Thanks to the advance of deep learning, there might be a solution.
A series of researches led by Yoshua Bengio has shown that recurrent neural network, also known as deep learning (at least one major type of it), may be a potentially good alternative.
Instead of aligning source-target phrases strictly, this so-called neural machine translation performs “soft” alignment3.
If we try our best to describe what a soft alignment is, we could say that neural machine translation attempts to imitate how human translators do their jobs: searching for the best fit translation for a phrase in their minds, without trying to align every phrase in a strict one-to-one fashion. As one may see in the figure of the demo site, while old-fashioned one-to-one alignment is in black lines, neural machine translation can also take grey-scaled lines into account.
It could be somewhat confusing that more alignment options seems to be more complex. At first glance, it is. However, neural machine translation’s soft alignment doesn’t need human experts to annotate those multiple
3 http://lisa.iro.umontreal.ca/mt-demo
Typical machine translation modeling nowadays
involves a module of automatic alignment.
Neural machine
translation attempts to imitate
how human translators
do their jobs.
11
Send your comments or questions to [email protected]
choices in grey-scaled lines. Neural machine translation doesn’t ask for a single standard answer of alignment. It just learns to weigh its options. If we rephrase this nice feature in a translation industry’s way, one may say that the only thing neural machine translation asks for is translation memory.
If they are so magical, how come they haven’t taken over the world?Both DuoLingo and neural machine translation have their weaknesses. DuoLingo doesn’t really care about every perspective of translation quality, it only cares about macro factors, such as consistency between learning material and translation demand. Acquiring learning materials of non-alphabetical languages to bootstrap the whole cycle is not a trivial task.
A similar obstacle can take place in neural
machine translation, too. All the fine-grained specifications one may encounter in translation business will be still there for quite a while.
That being said, disruptive innovation probably shouldn’t be seen as a savior of the industry. It is more like a shiny new armor or sword to help common mortals to survive in the constantly changing environment.
Boost the value of your TMs!
Powered by MorphoLogic Localisation
Machine Translation System
Lower costs
Increased productivity
Shorter delivery time
@globalesemt www.globalese-mt.com
12
Ethiopian languages and the need for translation within languages
Amharic, the descendant of Ge’ez, has been the language of everything for a long time – administration, trade, education, communication, etc. The entire nation was learning Amharic at school, work and in life. It is only recently, since the formulation of the current constitution - 1994/95, that it was limited to serving as a federal working language and a language for the whole nation. Since then, many of the other vernaculars got the status of a working language and/or language of education in their respective regions.
The constitution is now 20 years old, and in those 20 years (nearly one generation) many local languages have developed writing systems, produced literature, and accumulated knowledge, which keeps each language community self-contained and less interrelated to the others in many respects. Children are taught in their mother tongue to the end of their elementary education, and in some regions, to the end of junior secondary education. The administrative language and the language of communication in almost all regions is the respective regional vernacular.
In contrast to this, there are people living and working in regions where the local language might not be their mother tongue. Hence, they either do not speak the language or do not want to speak that language at their workplace, at court, or anywhere else the situation demands they speak that language.
In addition to this, people travel from region to region to visit friends and family, for work or they take residence in another region. So they need to learn the culture and everything else about the other language community; the regional governments and businesses need to exchange information, values, experiences and results or decisions.
Therefore, all these needs and evolving problems entail that the gap between and within language communities in Ethiopia is getting bigger and bigger. In order to bridge this gap, we may need a political and long-run decision or policy. However, I am arguing that translation is the solution at hand.
Tourists, investors, researchers, media, etc. use translators of various local languages at various occasions. For example, the local media translates news from other regions when they report it; tour operators hire a local translator
eThiopia is one of The many linguisTically diverse naTions in africa. iT houses over 85 vernaculars of which many do noT have a loT of speakers or have developed wriTing sysTems.
The mosT dominanT Three languages are amharic, oromo (afaan oromo), and Tigrigna. These languages correspond To The Three large and dominanT eThnic groups – amhara, oromo and Tigre, respecTively. This would show ThaT language hegemony ofTen reflecTs or depicTs socio-poliTical hegemony.
The gap between and
within language communities in Ethiopia is getting bigger
and bigger.
Review of language business & technologies in Africaby Amlaku Eshetie
13
Review of language business & technologies in Africaby Amlaku Eshetie
when they take their tourists to a tourist destination in another region; researchers use an amateur or professional translator to get their survey questionnaires or data collection tools translated into the required vernacular; investors likewise get their working documents, contracts, manuals, etc. translated into the language of the region they operate in.
Thus in a country such as Ethiopia where large and many regions exist with their own i n d e p e n d e n t and functional l a n g u a g e s , translation is vital to bridge the gap and to allow the various l a n g u a g e communities to stay connected.
Challenges to the remedyAs mentioned in my previous articles, there are several limitations to the translation industry in order for it to serve as a bridge for the different language communities. A lack of skilled and experienced translators, the society’s attitude towards translation as a profession, and lack
of technical and organizational support for the industry to emerge are the most significant limitations.
While the latter two are the same for almost all of the local languages, a lack of skilled and experienced translators is a major problem especially in the rare or less developed local languages. A good example is my experience with the Kunama language. A few of my clients (for Amharic, Afaan Oromo,Tigrigna) also asked me to provide them with Kunama translations.
I tried to use the available networks which clients use to find Kunama people, but none of them brought results. Finally, one of the clients was able to find a translator of Kunama who lives in the USA, but when I contacted her, she seemed to be overqualified or too busy to work for me.
The attitude of the society is also a hindrance for translation to emerge as a profession and business. This in turn contributes to the non-availability of translators: when it is not considered as a profession, clients do not pay well for translators and translators are not attracted to consider translation as a business or profession. As a result, the business or the profession is non-existent or not well developed in Ethiopia.
Remedy to the challengesI believe the solutions are not unreachable. The first thing to drive will be the market itself. As investment flow increases, business grows and the need for interaction or information exchange among cultures and language communities develops, demand for translation services starts to pull the translators. Another salvation for the translation industry in Ethiopia may be an intrusion of industry gurus. If an established
A lack of skilled and experienced translators is a major problem especially in
the rare or less developed local
languages.
Review of language business & technologies in Africaby Amlaku Eshetie
The attitude of the society
is also a hindrance for translation to emerge as a
profession and business.
14
company enters and sets the standard and the system for the market available in the country along with the global competition, then it may not take long to revive and become a booming business.
Both approaches worked together in a few African countries as far as I have seen. If anyone from South Africa, Egypt, Kenya or anywhere else reads this and
wants to back up on this by commenting, I would truly appreciate that.
It may not take long to revive and become a booming business.
Send your comments or questions to [email protected]
Review of language business & technologies in Africaby Amlaku Eshetie
15
www.capitatranslationinterpreting.comE: [email protected] | T: +44 (0)845 367 7000
• Secure integration• Secure translation management system• Capita IL3 data center hosting• Secure machine translation solutions• ISO 9001/27001 certified
Secure Localization Management
Your data has never been soimportant, so why risk its loss?
why risk itC
M
Y
CM
MY
CY
CMY
K
Capita-TI-TAUS-Review-Advert-June-2015.pdf 1 22/06/2015 11:14:10
16
Ford himself, then, might have been surprised that, a century later, the industry he built has delivered – what, exactly? Nothing more than faster cars with internal-combustion engines. Slow progress in electric and hydrogen-powered cars, and a century of dreaming of personal flight, hyperspace or teleportation (think The Jetsons, Star Wars or Star Trek) later, the majority of rich-world people wake up in the morning and turn the key in a petrol-fueled car to go to work.
What if the best technology has already arrived? People like their petrol cars. It is hardly the only technology that persists against apparently long odds. Despite the fact that everyone in the rich world has a smartphone with a big, sharp, contrasty screen in their pocket at all times, most people still prefer to read longer
things improbably printed on dried sheets of pulped tree, a technology little improved since the Han Dynasty.
So the question of where innovation in the language industry will come from
is an open one. Will innovation be gradual, or sudden? Will it come from a company that already exists today, or from one yet to be founded? Will it come from someone who dreams in Portuguese, Dutch and Persian, or someone who dreams in Perl, Ruby and C++? (Or, as the title of a recent New York Times Magazine article put it, “is translation an art problem or a math problem?”) Finally, will the
disruption come in translation itself—or in the business models of its providers?
This is the view of an outsider. In a recent article in The Economist, I tried to tackle the same subject, but by the time I had explained the shape of the language industry today, for the benefit of a nonspecialist audience, I hardly had space left to speculate about tomorrow. Herewith, some more detail.
Will innovation be gradual or sudden? On the language technologies themselves, especially machine translation, the improvement of late has been gradual—and looks as though it may even be plateauing. In my last column, I reported that MacDuff Hughes, the head of engineering at Google Translate, confessed that he did not expect more data to improve his engine’s performance. At this point, the next jump forward in quality looks likely to be more sudden
The Journalist’s Perspectiveby Lane Greene
Insiders or outsiders
“if i’d asked people whaT They wanTed, They’d have said ‘a fasTer horse’.” The quoTe from henry ford is a sTaple of Tech-Themed conference Talks, The kind where “disrupTive” is on every Tongue. iT’s inTended, wiTh more Than a biT of smugness, To show jusT how head-in-sand, inside-The-box and, well, noT “disrupTive” some oTher people are. a fasTer horse, indeed.
Will the disruption come in
translation itself—or in the business models of its providers?
What if the best
technology has already
arrived?
17
The Journalist’s Perspectiveby Lane Greene
than gradual, as the next leap in innovation appears—whatever form that will take (and whenever it will happen).
Is a currently existing company in a position to bring that big innovation? Or, in today’s required cliché, I must ask and answer: who could be the Uber of translation? That company does not yet seem to exist. The reason Uber has become the tech-journalism cliché that it has become is the speed with which it rolled up its space. With all respect to the companies innovating in language today, no one has achieved anything like that roll-up—in a fragmented industry that could sorely use some consolidation. Companies like Microsoft and Google have huge resources, of course, but it can be hard to be groundbreaking inside big companies with legacy products to think about. Perhaps the next big leap will come from an outsider, perhaps another clever graduate student in the mold of Sergey and Larry, building server racks out of Legos and building the world’s next great company in their spare time.
Will the breakthrough come from more traditional language experts or the math-heads? Statistics-driven engines currently dominate machine translation. At a recent MT hackathon, Gideon Lewis-Kraus reports in the Times Magazine article mentioned above, “nobody betrayed any interest in language as such… they were there for the math.” One participant told him “It’s surprising how little it helps to know another language.” If, as MacDuff Hughes said, more data cannot help much, then the math-types are left trying to crunch it better. They may yet succeed gradually. But some marriage of math and language is more likely to produce a jump in quality. Having divorced the linguists in the past, the math-heads may now find themselves making up with them, and having a go at a second marriage, for the sake of the kids—their products.
Where the technological breakthroughs will come from is not clear. But another thing seems much clearer: the language-services industry is
ripe for disruption in its business model. As an outsider, I have the luxury of saying what many friends in the industry have told me privately away from the conference stage: the people in this industry are brilliant (not to mention fun), but much of the work is dull. The core personnel of the traditional language-services provider, the project managers, burn out and turn over quickly. (And no need to take my anonymous bar sources’ word for it; the industry c o n s u l t a n t Common Sense Advisory says the same.) Where the rubber meets the road, with translators themselves, rates are squeezed and times are tight for the mass of non-elite translators.
The business that could automate and streamline the most boring parts of the job will make everyone in the chain happier. Translators love French, not file formats. Clients want speed, flexibility and transparency, not the bad news that a job will take months and that this simply can’t be helped. And the good news here is that unlike fully automated, high-quality machine translation, no huge intellectual breakthrough is required. The companies of today can certainly survive by innovating their own business models—a faster horse, a faster car—and make a lot of money if they delight customers by doing so. If they don’t, a disruption will eventually come from outside. It’s only a matter of time.
Will the breakthrough come from
more traditional language
experts or the math-heads?
Send your comments or questions to [email protected]
18
offering To make sense of someone else’s language is an old profession. a word for interpreter, sTill currenT in Turkish (tercüman = archaic english dragoman), is firsT aTTesTed in assyrian of 1800 bc (targumannum); while iTs synonym in sumerian emebal goes back a furTher 500 years. (The firsT known assyrian inTerpreTer worked wiTh hurrian, in whaT is now souTh-easTern Turkey, while The sumerian one dealT in The language of meluhha, To The souTh, possibly The indus valley1. so To innovaTe in This business will be a TesTing ambiTion.
But success is most likely to be achieved if the focus is not on means, but on ends.
Machine translation, as developed and applied, has mainly been for decoding use: that is to say, the translator is trying to gain knowledge from a foreign source. This is one reason why poor-quality text is sometimes acceptable as an output: at least it will be a much less foreign text than the input, and possibly
accessible to the user’s common sense. Encoding uses – which attempt to make the knowledge in a familiar text accessible to foreigners – are much less
tractable: why should the target audience, who did not choose to have the text translated, bother with an imperfect product? As a result, the audience best served is the one that uses the regnant lingua franca, in our own era overwhelmingly English, though the same argument would also favour translation into other big-beast language communities, who can plausibly see the world accessible in their language as world enough. Above all, the availability of such translation spares them the massive effort of conscious language learning.
1 I.J. Gelb, The word for Dragoman in the Ancient New East, Glotta 2:1 (1968)
Language learning is another field fit for technical support (from conventional grammars, dictionaries and tutors through to CALL). But the effort is only repaid if the student foresees massive use of the new skill: this means that once again the regnant lingua franca will be the preferred goal, providing lots of hitherto inaccesible texts (and people!). Speakers of minority languages will aim to widen their scope by joining the second-language learners of this (already vast) language community.
So both these inter-lingual technologies – translation and learning – tend to favour the regnant lingua franca as their target language. The incidence and level of costs is different but in both cases, the implicit aim is to make the “foreign language problem” go away: wider communication is to be achieved, largely by adding to the effective reach of the language that is already dominant. To the languages that have speakers and writers worth attention, more
The Language Perspectiveby Nicholas Ostler
The audience best served is the one that uses
the regnant lingua franca.
New Lamps for Old
Machine translation, as developed and applied, has
mainly been for decoding use.
19
shall be given. “Foreign” – i.e. minority, less dominant – language communities gain little or nothing from use of these technologies.
But suppose the aim were not to by-pass the foreign languages, seen as obstacles to communication, but to empower them, as distinctive approaches to knowledge, hallowed by traditions that are open-ended, and in no way yet exhausted. Is it possible to give
them aspects of the power or investment that are already being devoted to the lingua franca, and (to a lesser extent) to the other big beasts in the linguistic jungle?
This is not pure idealism. One area of language technology where this is happening is the IDN (International Domain Names) project of ICANN (the Internet Corporation for Assignment of Names and Numbers) which regulates internet addresses globally2. More generally, the Unicode project has long laboured to define a system of compatible and distinct codes for all the characters of all the writing systems of the world (even past and present). The result is a rich and evolving system which transcends the narrow space of codes once laid out for ASCII, and then arbitrarily – and incompatibly – extended almost at random for other character-sets of the world’s languages.
The results bid fair to widen the effective use on digital platforms of the major non-English languages of the world. ICANN – in setting out the subset of Unicode scripts fit for use in top-level domain labels – is explicitly committed to include all characters in use for sufficiently vital languages as measured by the Ethnologue’s EGIDS scale. The criterion chosen is grade 4: “in vigorous use, with standardization and literature being sustained through a widespread 2 <https://www.icann.org/resources/pages/idn-2012-02-25-en>
system of institutionally supported education”. As a result, the writing will be included for 572 (8.2%) of the world’s languages, but as written by 82.6% of the world’s population3.
This is one formal way in which investment is being made in new language directions. Less formally, but with immediate human content, is the range of new social media projects (blogs, Facebook, Tweets, corpora) in indigenous languages which are indexed and accessible at Kevin Scannell’s site4. This empowering process can be promoted by application of computer-aided language-learning (CALL), but only if it is applied imaginatively, in combination with direct human content, which might involve a mix of conferences, festivals, summer schools, as well as conventional language course teaching. Machine Translation, and other software aimed at directly converting a stream in one language to another, seems less promising, just because it is focused on converting the cognitive content of a message or text, and so replaces, rather than builds, the cognitive skills that underlie command of a language. Marginally, the product of such encoding might be useful after the event, if the original text 3 <https://www.ethnologue.com/statistics/status>4 <http://borel.slu.edu/nlp.html>
Suppose the aim were not to by-pass the foreign languages, seen as obstacles to communication, but to empower
them.
The Language Perspectiveby Nicholas Ostler
20
The Language Perspectiveby Nicholas Ostler
were a familiar one: it could then be used for rapid reading, much as one might improve (to an extent) one’s familiarity in reading Latin by scanning a volume of Harrius Potter or Asterix in Latin translation.
The key is to facilitate, not replace, the development of skill – e.g., to provide a quick check on reading a passage in strange script, to show the structure of words or sentences, to look up new words.
The key is to facilitate, not replace, the development
of skill.
Send your comments or questions to [email protected]
The result would be to give a more penetrating sense of the language to which the learner aspires. As Ken Hale exclaimed, on first understanding the perversities of Tjiliwiri, an Australian language of initiates, and the joy in mastering a ritual system: “You certainly have something here!”1
1 A note on a Walbiri tradition of antonymy, in Semantics, ed. Steinberg & Jakobovits, Cambridge UP, 1975.
21
Using translator activity data in CAT tools to better understand translation automation
The Translator’s Perspectiveby John Moran
In other words, for most unassisted translation, word price and quality are traditionally the only two metrics that count. However, linguistic technology is slowly changing this view. Agency owners who were previously sceptical of the claims of MT vendors and organisations like TAUS are now trying to figure out how to manage post-editing projects to reduce translation costs. Some - mostly corporate - end buyers have been asking for discounts for
MT post-editing for some time. Un f o r t una t e l y, these discounts are often granted without proper regard for a t r a n s l a t o r ’ s hourly earnings – disgruntled former
post-editors are not hard to find at translator conferences. It is no exaggeration to say that the translation industry is still struggling to find its feet when it comes to establishing fair compensation models that take the volatility of MT into account.
However, as Jost has pointed out in this column in the past, post-editing of fully machine translated sentences is far from the only game in town. In Transpiral (my own agency), a number of the freelance translators we work with use various versions of Dragon (an Automatic Speech Recognition application
by Nuance Communications) to dictate their translations into English and German. We find the technology has no noticeable impact on price, as the translator normally buys Dragon himself. However, we do support its use as it leads to faster and more terminologically consistent translation. This is because fewer linguists need to work on large documents with short turnaround times. Also, translators who translate fast and well using dictation tend to be experienced to the point where they do not need to do much terminology research. To put it another way – they know their stuff.
These quality upsides are great, but they have to be balanced against the cost of careful bilingual review. Constant vigilance is required to prevent sometimes hard to stop errors we call “dictos”. For instance in the previous sentence, “stop” should have been “spot”. The two words sound like each other and they look similar. This (faked) dicto changed the meaning of the sentence, but it is still
mosT TranslaTion agencies do noT care how fasT a TranslaTor works, as long as They meeT The deadline and The qualiTy of The TranslaTed TexT is saTisfacTory. mosT TranslaTors who are paid by The word do care. cerTainly, iT can Take hours To chase down a single Term, buT a freelancer will soon sTarve if This happens Too frequenTly. mosT TranslaTors and projecT managers plan projecTs on The basis of beTween Two and Three Thousand words per day. The number of hours required To TranslaTe Those words varies more Than mosT people Think.
Word price and quality are
traditionally the only two metrics that
count.
The translation industry is still struggling to find its feet
when it comes to establishing fair compensation
models.
22
The Translator’s Perspectiveby John Moran
plausible, so a monolingual review of the target sentence does not suffice. This means that when dictation is involved – even for trusted translators – a reviewer needs to check both source and target sentences for an unintended change in meaning. Another category for TAUS’s Dynamic Quality Framework, perhaps?
Finally, interesting approaches to interactive or sub-segment level MT are starting to appear. Translators who manually add terms to an autosuggest dictionary or termbase over the years have long reported very real productivity gains. Our success here has been mixed. Terminology extraction is manually intensive
and doesn’t yield enough proposals. This means terms are hard to bootstrap for freelancers at the LSP level to the point where any negotiation regarding a discount would be reasonable. We still do it on large projects as a consistency
measure, but not on smaller ones. MT-based approaches like the MT Autosuggest plugin in Trados Studio, and statistically generated frequent terms like the Muse function in MemoQ, seem popular with some translators on some content types but not with others. It is the first thing some translators switch off. On the whole, agencies like my own that mainly employ freelance translators with in-house review don’t really know very much about what goes on in most offline or desktop-based CAT tools. This knowledge gap has been the focus of my research in the ADAPT Centre for the past five years.
The analysis of User Activity Data to get a better understanding of the process of translation in CAT tools is a young field. Projects like MateCAT and CasmaCAT and our own collaboration with Welocalize on iOmegaT have resulted in (we hope) some interesting publications where relatively large-scale translation productivity
tests could be carried out. Our software is an analytics platform to test a range of measures that can impact on translator productivity. Though it is based on OmegaT, a free open-source CAT tool we could adapt to our needs, we hope to collaborate with other proprietary CAT tools on sophisticated time reports.
Currently we can measure the impact of an MT, automatic speech recognition (using Dragon) and soon, we hope, a promising new approach to interactive MT where proposals are provided automatically at a sub-sentential level. Linguistic technology aside, time data can also help with vendor management by helping PMs to match translators with content that suits their skills and experience. The package, which can be licensed through the ADAPT Centre, comes with an interoperability suite to support older versions of Trados and a range of TMS systems, including SDL TMS and SDL WorldServer. Unsurprisingly, it also works particularly well with GlobalSight, the open-source system published by Welocalize, our development partner on the iOmegaT Translator Productivity testbench.
However, we are not alone in terms of translation speed reports. MemoQ has one and Wordfast Pro will soon have one too. For translators who keep their CAT tool versions up-to-date, Trados Studio 2014 will soon have a plugin called Studio Time Tracker that measures translation speed. These reports are
Finally, interesting
approaches to interactive or sub-segment level MT are starting to
appear.
23
private to the translator and can only be shared with the translator’s consent.
This may be a good thing for translators as tracking translation speed in words per hour can be dangerous to their interests. Some LSPs already use translation speed data to negotiate discounts, even when no MT is in play. One agency from New Zealand, with its own web-based CAT tool, even claims this as a selling point. Though time reports are not common across most systems with web-based CAT tools, economically the publishers behind them are less beholden to the needs of freelance translators as they are not their paying clients. This means they cannot vote against unpopular features with their wallets (though they can refuse to work in them). For example, a free web-based CAT tool developed under a European FP7 project called MateCAT has a time report with (as yet) no opt-out mechanism for translators.
The legalities of tracking working speed vary from country to country. In Germany during the late 90s I worked for a consulting company called Cap Gemini Ernst and Young for a number of years. It was never easy to measure employee productivity in large companies, as the permission of works councils was required. I understand the same is true in France, but not in my home country of Ireland. For freelancers it seems it is perfectly legal to record working
time.
It is in fact quite easy to measure the impact of MT on a translator’s working speed. You simply remove the MT from random segments, as we do in iOmegaT. A similar approach is taken in TAUS’s DQF platform. We call this SLAB testing (short for Segment Level A/B). Case A is translation from scratch and case B is translation using the MT. The approach is similar to one used in IBM’s internal TM/2 CAT tool. Once it has been established that translators are faster with MT, SLAB testing is not required and word price discount can be safely negotiated. It is important to keep an eye out for a drop in MT quality using at least intermittent speed reports. This can happen when a client introduces a new product feature or a technical writer retires.
Is time reporting intrusive? I don’t feel it is – but of course, I have skin in the game. In my defence, I would say that working with hundreds of translators over nearly 20 years I have noticed that very few translators take more than a passing interest in their productivity. The data I see from my own agency and in our
published productivity tests with iOmegaT suggest that much can be done to improve it. My hope is that translators will begin to realise this and take a more empirical approach to analysing how they translate, as well as a more open-minded approach to technologies like dictation software and variations on the theme of MT.
Send your comments or questions to [email protected]
The Translator’s Perspectiveby John Moran
It is in fact quite easy to measure the impact of MT on a translator’s
working speed.
John Moran
John Moran graduated with a degree
in computer science, linguistics and
German in 1997. He is a former
translator, lecturer in translation,
software engineer and is a co-owner of
Transpiral, an LSP in Dublin specialized
in German technical translation and Irish Gaelic. He is a
translation technology consultant and research fellow in
ADAPT - a large language technology research centre in
Trinity College Dublin - where he is working on a Ph.D. in
computer science. His research interest is the analysis of
user activity data in CAT tools.
24
Join us on July 30 for conversation, a potluck dinner and pool competition
RSVP Here
25
The lack of innovation
in the translation business
comes from pulverization.
Waiting for Landmarks
The Research Perspectiveby Luigi Muzii
An Amelioration OrgyThis could help explain why true innovations most often come from outsiders and why Evgeny Morozov’s target in To Save Everything Click Here is an “amelioration orgy” that he associates with Silicon Valley, whose favorite slogan, in the past few years, has quietly changed — he says — from ‘Innovate or Die’ to ‘Ameliorate or Die.’
Then again, in The Entrepreneurial State, Mariana Mazzucato outlines a number of case studies in different fields to show that most of today’s technologies (from the Internet to GPS, from touchscreens to NLP applications ) are heirs of earlier publicly funded projects. Many other of the most significant innovations of the last 50+ years (from photocopying to laser printing, from the personal computer to the WIMP paradigm and the GUI, from Ethernet
to ubiquitous computing) were publicly funded too and came from a single place in a fairly short time lapse.
Innovation in Translation: Navel-gazingGoogle launched Google Translate, an SMT engine, in October 2007 after years of using SYSTRAN’s RbMT engine developed
with massive i n v e s t m e n t s from the U S D O D and the EC. More or less overtly, GT was born to support the company’s core business, online advertising, and unintentionally disrupted one of the last craftsman businesses. Today, like it or not, GT has become the benchmark for machine translation and translation tout court, but Google remains a major translation buyer.
The translation community has been thriving for decades now on technology and basks in the belief of operating in a high-tech business, but most players are actually oldfashioned. The lack of innovation in the translation business
in The ten Faces oF innovation, To supporT his view of The inconsisTency of user-driven innovaTion, auThor Tom kelley quoTed henry ford who said, “if i had asked my cusTomers whaT They wanTed, They’d have said a fasTer horse.”
Go and face the future Come with me and introduce me
26
comes from pulverization, a disinclination to collaboration, and a highly conservative nature of players. Even the regular mergers and acquisitions have never reduced pulverization nor produced any real innovation. They aim at complementing customers, offerings, and extend market penetration, but very seldom at acquiring greater financial strength to fund innovation.
The last real innovation in translation were translation memories, a quarter of a century ago, as a development of the machine-aided translation effort following the (in)famous ALPAC report.
Even TMS are a very peculiar abridged application of project management software, afar from workflow management systems, which remain extraneous to the translation business, although they could be a leap forward.
More than the invisible hand, it is realistically predictable that the longa manus of some technological outsider will bring unintended benefits resulting from any innovations.
Quality is a perfect example of navel-gazing in translation, especially in the academic field. It is a most debated topic and yet the translation
community is still at the I-know-it-when-I-see-it stage. However, it is almost ludicrous to figure a judge making a landmark case by sentencing about the quality of translation.
Today, the value of an industry depends more on its technology, financials, and the image projected than on the value of its products. Despite technology, automatic evaluation of translation quality is yet to come, while there is no innovation in yet another and another quality standard. Should these new efforts see
the light, they would already be obsolete, and not raise the overall quality level, as the other standards. Unmistakably, they are meant to be inclusive, comprehensive, exhaustive, and universal, and are presented as simple, easy and quick to apply.
Technology and LifeTe c h n o l o g y has always scared people, even more so now that also white-collar jobs (that seemed away from any technological threat) could be lost to machines.
In Jeremy Rifkin’s view, big data, advanced analytics, artificial intelligence and robotics will increasingly replace human workforces across all sectors of the market economy as an IoT infrastructure is developed and the marginal cost of labor drops to near zero.
A 2013 study of Oxford Martin School estimated that 47% of US jobs are vulnerable to computerization. Two years later, another study envisioned that the digital economy might cause secular stagnation.
There is nothing more slippery than forecasts, and yet this seems to be a mere anticipation of reality.
In 2004, Frank Levy and Richard Murnane excluded that a driverless car could ever be possible. A decade later, Google cars have logged nearly 700,000 autonomous miles (1.1 million km), driver assistance systems have gone to mass production, and automatic parking, collision avoidance and cruise control systems are no longer a frill.
In The Glass Cage, Nicholas Carr envisions
The last real innovation in
translation were translation
memories, a quarter of a century ago
Big data, advanced
analytics, artificial intelligence and
robotics will increasingly
replace human workforces
The Research Perspectiveby Luigi Muzii
27
a future in which the human factor declines and echoes Tesla Motors’ Elon Musk warning, “With artificial intelligence, we are summoning the demon.” By the way, Tesla Model S is the first mass-production vehicle equipped with autopilot.
Carr also recalls Katherine Hayles observing in her 2012 book How We Think that when her computer goes down or her Internet connection fails, she feels lost, disoriented, unable to
work; in fact, she feels as if her hands have been amputated. In Hybrid Reality, Ayesha and Parag Khanna wrote that our r e l a t i o n s h i p w i t h technology is already beyond the purely inst rumenta l
level to enter the realm of life. In other words, we are the technology we use, and
the increasing, unknowledgeable intimacy with technology dictates and possibly biases expectations.
Datafication and Deep LearningThe brain’s computing speed is the equivalent of 1 kHz. My laptop computer runs at 1.90 GHz. Latest processors run at 3.5 GHz (1 GHz = 1 million kHz). Nonetheless, to accomplish the same number of a brain’s binary operations per second, a computer should be as large as a room, consuming the same power as a small town, while a human brain requires only 40 watts out of the 1,000 the whole body consumes.
Innervation makes the difference, enabling parallel rather than serial brute-force computing. In addition, thousands of years of (continuing) evolution have enabled the development of heuristics, while computers are still stuck to algorithms.
That’s why tech giants are pouring big money into deep learning to allow computers to operate following a model from example inputs and make predictions, with heavy-duty statistical analysis of huge amount of data.
So while IBM’s Watson was already set up to analyze health records, looking for medical insights, last year, Narrative Science launched Quill Engage, a free Google Analytics application to deliver plain-English, narrative-style reports, which Forbes is already using to cover basic financial stories.
Epistemic AsymmetryThe translation business is affected by a classic information asymmetry, while translation business players are affected by epistemic — and possibly non-cooperative — asymmetry. Epistemic asymmetry makes many in the translation business feel threatened by
Tech giants are pouring big money into deep learning.
We are the technology we use, and the increasing,
unknowledgeable intimacy with technology dictates and
possibly biases expectations.
The Research Perspectiveby Luigi Muzii
28
machine translation, as if it were about to wipe them out.
Some of these people claim for re-imagining technology, while translation technology disrupted the translation business precisely when business players started using it. Not surprisingly, according to a One Hour Translation survey of May 2015, more than 80% of consumers can tell the difference between a professional human translation and a machine translation output.
Another silent and yet fast-pacing disruption is on its way, coming from web-based platforms, many of which are, again, from technology companies rather than translation service
companies. The most advanced platforms are designed as virtual market places.
However, due to the many facets of translation, a real disintermediation is far from being achieved. From workflow customization to
staffing, we have a long road ahead.
The translation community is lagging behind and a foreseeable future is not in machine translation, but in aggregators, similarly to what happened in tourism. Again, these will come from translation customers who are eager to find a way to reduce information asymmetry, increase speed, and easily set pre-requisites to anticipate quality.
Maybe automation enhancements will eventually involve workflow and quality, with customers being able to buy or lease
platforms including full workflow customization and automatic profiling, sampling, and quality evaluation.
Automatic quality evaluation could already be at hand, through detection of bad samples for reworking and r a n k i n g - b a s e d assessment with a combination of content profiling
via previously collected requirements and inspection, and with linguistic evaluation revolving around a combination of correlation and dependence, precision and recall and edit distance.
Send your comments or questions to [email protected]
The translation community is lagging
behind and a foreseeable future is not in machine
translation, but in aggregators.
The Research Perspectiveby Luigi Muzii
29
memoQ.com
“memoQ is a very intuitive, user-friendly, easy to learn,
and easy to integrate translation tool. In addition,
the level of support provided by their team is excellent.”
Birgitte Bohnstedt
FLSmidth A/S
30
The ProblemDecember 2014 was the seventh anniversary of the launching of Intel’s first machine translation (MT) system. The language was Latin-American Spanish and it was integrated into the platform that published content to the Intel Customer Support website. Intel IT designed the workflow to automatically detect any additions and changes to the English content every twenty-four hours, queue them for translation, and publish the unedited output to the Spanish website. As a result of this deployment, ninety-five percent of the Spanish
was translated by machine and five percent by humans (new products, safety, and legal, content). Since the launch of Latin-American Spanish, Intel has developed and integrated many more machine translation languages into the Intel Customer Support website: Brazilian Portuguese, Simplified Chinese, Traditional Chinese, Korean, Russian, French, and Italian. Eventually many of these MT language engines were also integrated into the Intel Support Community Forum to give customers the ability to translate, in real time, forum postings from other customers or Intel support agents.
Intel and other organizations both inside and outside of the TAUS community have fueled and driven tremendous growth in unedited machine translated content. These investments have been made based on a financial principle that MT is the only cost-effective solution to bring massive amounts of content to a global audience. Yet very little usability testing and customer experience studies had been conducted and published. A few well-done usability studies on machine translated content using eye-tracking techniques published by CNGL (for example, Doherty and O’Brien, 2013) exist, but considering the growing dependency on machine translation to deliver all sorts of content, Intel decided it is now time to better understand the MT and user experience. If we could demonstrate acceptance of MT content, we could build trust in previously r e s i s t a n t regions and parts of the industry, which could allow us to quicken the pace of MT deployment. But how to conduct statistically rich studies in many languages, with globally dispersed users? With funding for translation
ABSTRACTwhile we conTend machine TranslaTion (mT) is noT appropriaTe for all of Today’s TranslaTion soluTions, inTel has promoTed using mT To TranslaTe knowledgebase conTenT since 2007. inTel iT recenTly compleTed a projecT To assess The users’ accepTance of The mT conTenT published on The supporT websiTe. inTel iT along wiTh our Taus colleagues have driven and enabled a sweeping adopTion of mT as a reasonable sTraTegy for companies, governmenTs, and organizaTions To reach Their global users. amidsT This rapid growTh of mT soluTions, very liTTle user experience research has been done on unediTed mT conTenT.
The barriers of cosT and complex global logisTics have kepT inTernaTionally scaled usabiliTy TesTs for mulTi-language mT comparisons ouT of reach, buT inTel iT recenTly compleTed a sTudy using unmoderaTed remoTe usabiliTy TesTing (uruT) sofTware, and found uruT noT only overcomes The barriers of cosT and global resourcing buT iT supporTed The hypoThesis ThaT There is a high-degree of accepTance of mT conTenT among users. These sTudies have shown ThaT There is an accepTance among Technical and non-Technical cusTomers alike for The un-ediTed mT conTenT ThaT inTel’s soluTion provides. This Tolerance was sTriking in ThaT iT shows accepTance of un-ediTed mT sourced conTenT even from wiThin some TradiTionally resisTanT geographies. The uruT sofTware offers organizaTions a cosT-effecTive soluTion To conducT focused, in-depTh, sTaTisTically significanT, mulTilingual sTudies of Their machine-TranslaTed websiTes wiThouT The expense of moderaTors, recruiTers, and complex usabiliTy labs. If we could
demonstrate acceptance of
MT content, we could build trust
in previously resistant regions and parts of the
industry.
Unmoderated Remote Usability Testing of Machine Translated Contentby Will Burgett
31
already in short supply, how to secure the budget for resources, facilities, and equipment to do traditional lab usability studies on a global scale? To solve the problem the Intel team sought out some innovation to address the needs.
The History Of Machine Translation at Intel CorporationIntel’s impetus for adopting machine translation was based on a number of factors:1. Volume of Content. Intel Customer
Support had over 8,000 English source files in its content management system. These documents were published to the website and then periodically updated as information changed.
2. Languages. To support its major global markets, Intel had to deliver translated
content in at least ten key market languages.3. Time. Typically it took about ten business
days for a complete localization cycle (1200-1500 files per cycle), even with the use of workflow automation technology. That cycle time meant that the translated content was at least ten days out of sync with the English content.
4. Budget. Because of the huge volume of content, we had depleted the budgets for human translation. For even the most important languages, there were only enough funds to translate about 15% of the English into any given target language.
5. User experience: With only 15-20% of the English content translated, users would have a poor experience using a patchwork of English and translated content.
Unmoderated Remote Usability Testing of Machine Translated Contentby Will Burgett
Over time using machine translation had a huge impact on the depth and breadth of translation, cost savings and avoidance, and the speed of translation. These improvements were easily measured. For example, as a result of translating the entire site into Spanish, visits to that website quadrupled in less than six months (from about 22,000 per month to 90,000 per month).
An interesting challenge was how to measure the customer experience of unedited machine translated content. First, a disclaimer in the
target language was placed on every page to inform the user that this is automatic translation:
“This information is a combination of a translation by human and machine translation by computer of the original content for your convenience. Updates and changes are translated and made available in 24 hours. This content is provided as general information only and should not be considered as complete or accurate.”
32
Unmoderated Remote Usability Testing of Machine Translated Contentby Will Burgett
After more than six years of
collecting data, there were very few comments
about the quality of
translation.
To empower the user, translated documents include a link to the corresponding English page if the user is not satisfied with the translation. Also from the very beginning we developed and deployed a survey widget that appears on every page. The survey asks one question:
“Did you find this information helpful? Yes or No.”
The survey widget also asks the user if they have any additional feedback and provides an area for them to write their comments.
Over the next few years, the data from our survey widget showed that the level of customer satisfaction from our machine-translated content was getting very close to English levels. After more than six years of collecting data, there were very few comments about the quality of translation. However, a short survey with comments was too one-dimensional to be useful in drawing conclusions about the usability of our machine translated content. We needed much more dialog with customers and better usability data to clearly see user experience in more than one dimension.
The problem is that a typical moderated usabi l i ty-testing approach requires the use of labs, resources to develop and conduct the tests, and expensive recruiting of typical users to take several hours
out of their day to come to the lab to do the usability studies. On top of that, the expense and physical location of the usability lab limited the number of participants and the languages.
So how could we overcome all of these limitations to gain a deeper and more dimensional picture of the user’s experience of unedited machine translated content?
The SolutionUnmoderated Remote Usability Testing (URUT) is an ideal solution for global websites. URUT technology automatically presents tasks to participants and tracks their interaction, including navigation path, page scrolling, and click location. The data collected measure the important usability dimensions of:• Effectiveness• Efficiency• Subjective satisfaction
URUT deals with the following issues that online user researchers typically must solve:1. Collecting large samples of data to quantify
usability and user experience. URUT generates larger user samples than in-lab moderated usability testing, which are limited by cost and space.
2. Users can participate in their natural context, at home or at work, or on the road using their own computers or mobile devices.
3. Organizations with a global target audience can test in many countries at the same time without a large budget.
4. URUT allows you to conduct benchmarks among different competing sites and see who is performing well and offers the best usability and user experience.
5. We want to know users’ real goals and why they do what they do. Web analytics tools give us lots of data about what happened on a site at an aggregate level, but we still don’t know why it happened or how it relates to different user goals. The flexibility and depth of many of the mature URUT tools in the market can analyze the user experience at a much deeper level.
URUT software can conduct both usability and survey- based studies, including:• Web Voice of the Customer (VOC)
• Intercept surveys• Using feedback tab• Mobile apps
• Web remote usability testing• Mobile usability testing• Card sorting
33
Unmoderated Remote Usability Testing of Machine Translated Contentby Will Burgett
The actual study or test contents that we created for each study had to be translated into the target
languages.
• Tree testing• Screenshot click test• Screenshot timeout testing• Online surveys
The types of data that can be collected using some types of URUT software include:• Usability• Survey responses• Information architecture• Behavioral data• Video replay• Video feedback for mobile
There are a number of companies that offer different URUT software and services and Intel selected one of the better known providers. The selected URUT application supported localization in many languages and included
all the study types that Intel was interested in. However, the actual study or test contents that we created for each study (survey questions, usability tasks, etc.) had to be translated into the target languages.
The StudyThe project team defined a number of goals and objectives they wanted to achieve with their initial usability test of unedited machine translated content on the customer support website:1. True intent of users2. Evaluate and quantify the user experience3. Establish a benchmark4. Collect data that help improve machine
translation and the user experience
The first set of user experience studies included a task-based study of procedures that the user had to perform on the customer-support website. The first tests were conducted using
English as the control language, Latin-American Spanish and Simplified Chinese as the target languages.
This was followed by a usability survey that was deployed in five languages including L a t i n - A m e r i c a n Spanish, Brazilian Portuguese, Russian, Simplified Chinese and Korean. The study used an intercept method to invite users to take the study (or decline). A goal of 200 responses was set for each language, and a window of one month was given to collect the responses.
The focus area of the test:• Were users able to accomplish their site-
visit goal• Level of user satisfaction • What errors or actors prevented users from
solving their problem• Quality and usability of content • How machine translated impacted user
experience
The ResultsThe following graphs show the results of the true intent study with the five target languages using English as the control.
A goal of 200 responses was set for each
language.
34
Unmoderated Remote Usability Testing of Machine Translated Contentby Will Burgett
1. What is your primary language?
The primary language for 25% of the users visiting the English site was not English. It would have been interesting to add more questions for these users that investigated how their experience may have been different than the native English speaker.
2. The language I used most on this site was
A significant number of respondents said that they used both English and their primary language (p+eng) while on the site. 3. What best describes the purpose of
your visit?
The majority of the users were on the site to download software or drivers. There were also significant numbers of users who were using the site to research product specifications or troubleshoot issues.
4. What was your overall satisfaction with the site?
For Russian and Brazilian-Portuguese, more than 60% rated their satisfaction level as 4 or 5; there was a similar result for Latin-American Spanish. Korean scored the lowest level of 4 and 5, but 50% of respondents did score it a 3.
5. At any point during your visit did you follow instructions for completing a task and did you successfully complete your task?
Only the people who answered “Yes” for the first question are shown in the second graph. Korean had the highest number of users who couldn’t complete the task successfully. That was not a surprising result since Korean tends
35
Unmoderated Remote Usability Testing of Machine Translated Contentby Will Burgett
to be a very difficult language for statistical machine translation.
6. What best describes the reason you were not able to complete the task?
There wasn’t any data or feedback that gave us any ideas on why the “content wasn’t accurate or incorrect.” In future studies it will be interesting to investigate if the information is missing or incomplete, and determine whether or not that was due to the quality of the machine translation.
7. What best describes your opinion of the language on this site?
Russian, Latin-American Spanish, and Brazilian scored the highest, Korean the lowest. Users of the English survey were not asked this question.
8. You answered that you used more than one language during your visit. Can you provide more details?
The sample for this question was much smaller. Although Russian had the highest number of “quality of language is not good”, it still had the highest overall score. Russian, because of its complex morphology and syntax, is also one of the more difficult languages for statistical-based MT.
9. How well do you understand English?
Spanish and Portuguese had the highest number of users responding that “I am fluent” in English.
10. What statement best describes your opinion of machine translation?
36
Unmoderated Remote Usability Testing of Machine Translated Contentby Will Burgett
About 13% of the respondents said they were not familiar with the term “machine translation.”
If English is excluded, this ratio drops to 10%. The responses to this question supported our hypothesis that users with a primary language other than English are much more comfortable with MT. These users were twice as likely to say that they thought MT
was “useful sometimes” or that they use it “all the time”. Similarly, users with English as a primary language were much more likely to say that they had never heard of MT.
Perhaps a follow-up question may have shown that they are more familiar with service names like “Google Translate” or “Microsoft Translator.” This fact well illustrates the benefit of moderation to enable follow up discussion.
11. English survey: Primary languages listed as “Other”
One of the very interesting findings of this true-intent study was that 51 out of the 200 participants taking the English survey stated their primary language was not English.
ConclusionsCustomers come to www.Intel.com and www.
support.intel.com for a multitude of reasons. This study focused on technical and non-technical users who primarily came to the Intel Customer Support site to accomplish the following tasks:• Download drivers or software• Search for product specifications• Troubleshoot problems
Overall we found the use of unedited machine-translated content did not impact the user experience in Latin-American Spanish, Brazilian-Portuguese, Russian, and Simplified Chinese. Only the quality of machine translated Korean showed an impact on the usability and user experience of the Korean site. Looking at how well m a c h i n e t r a n s l a t e d sites scored compared to the English reference site demonstrated that MT content a c t u a l l y increased the user’s level of satisfaction. This true intent study showed that the majority of the users visiting the five target language sites found machine translated content to be useful.
These initial studies also demonstrated the potential power and versatility of URUT technology to:• Enable both large and small organizations
to do focused, in-depth tests and studies of websites and web applications on a global scale.
• Simultaneously conduct statistically significant tests and studies across many languages.
• Conduct usability testing at a much lower cost without the need for labs or moderators.
Caveat: While Intel IT does conduct unmoderated remote usability testing on a variety of topics, one in-house user-experience
Users with English as a primary
language were much more likely to say
that they had never heard of
MT.
The majority of the users visiting
the five target language sites found machine
translated content to be useful.
37
Ryan Martin
Ryan Martin is an application
developer at Intel Corporation
(2001-present). Over the past
7 years, Ryan has designed
and deployed multiple
enterprise MT solutions across Intel. His interests
include translation automation, natural language
processing, computational linguistics, and data
analysis. Ryan lives in Portland, Oregon with his
wife and two kids.
Unmoderated Remote Usability Testing of Machine Translated Contentby Will Burgett
expert stated a preference for moderation to enable more in-depth follow up.
The results of this study do corroborate that moderation would have enabled follow up with users who said they didn’t understand the term “machine translation” to see if they were instead familiar with “Google Translate” or “Microsoft Translator”. And with a moderator we could have elicited feedback to better understand users’ ideas on why the content was viewed as incorrect.
However, for this first study on acceptance of MT content, the expense of moderation was not needed to help us learn what we really needed to know: Intel’s investments in deploying machine translated content for support.intel.com are not at the expense of the international users’ experience.
Will Burgett
Will Burgett has over 30
years of experience with
Electro Scientific Industries
(ESI), The Writing Company,
and Intel Corporation. Will
has managed publications,
localization, human factors engineering, product
development, innovation programs, and is a PMI
certified Project Management Professional. He was
Chairman of the Board for TAUS Data Association
and a member of the TAUS Advisory Board. Will
retired from Intel Corporation in January 2015
and is currently the Communications Manager for
the nonprofit Translators without Borders, and he
also operates his own rare and out-of-print book
business in Oregon.
Julie Chang
Julie Chang has over 15 years of
experience as a Localization Program
Manager at Intel Corporation. She
leads and develops translation
automation strategy for Intel
Customer Support group. She has
also managed and executed multiple Machine Translation
implementations and integrated Machine Translation
technology into various platforms across Intel organizations.
Julie lives in Portland, Oregon with her husband, daughter,
and two dogs.
Send your comments or questions to [email protected]
38
Amlaku Eshetie
Amlaku earned a BA degree in
Foreign Languages & Literature
(English & French) in 1997, and
an MA in Teaching English as a
Foreign Language (TEFL) in 2005,
both at Addis Ababa University, Ethiopia. He had been a
teacher of English at various levels until he switched to
translation and localisation in 2009. Currently, Amlaku
is the founder and manager of KHAABBA International
Training and Language Services at which he has been
able to create a big base of clients for services, such
as localisation, translation, editing & proofreading,
interpretation, voiceovers, copy writing.
Andrew Joscelyne
Andrew Joscelyne has been reporting
on language technology in Europe for
well over 20 years now. He also been a market watcher
for European Commission support programs devoted
to mapping language technology progress and needs.
Andrew has been especially interested in the changing
translation industry, and began working with TAUS from
its beginnings as a part of the communication team.
Today he sees language technologies (and languages
themselves) as a collection of silos – translation, spoken
interaction, text analytics, semantics, NLP and so on.
Tomorrow, these will converge and interpenetrate,
releasing new energies and possibilities for human
communication.
Mike Tian-Jian Jiang
Mike was the core developer of GOING
(Natural Input Method, http://iasl.iis.
sinica.edu.tw/goingime.htm), one of
the most famous intelligent Chinese
phonetic input method products.
He was also one of the core committers of OpenVanilla,
one of the most active text input method and processing
platform. He has over 12, 10, and 8 years experiences
on C++, Java, and C#, respectively. Also familiar with
Lucene and Lemur/Indri. His most important skill set is
natural language processing, especially for Chinese word
segmentation based on pattern generation/matching,
n-gram statistical language modeling with SRILM, and
conditional random fields with CRF++ or Wapiti.
Specialties: Natural Language Processing, especially for
pattern analysis and statistical language modeling.
Information Retrieval, especially for tuning Lucene and
Lemur/Indri. Text Entry (Input Method).
Brian McConnell
Brian McConnell is the Head of
Localization for Insightly, the leading
small business CRM service for
Google Apps. He is also the publisher
of Translation Reports, a buyers guide
for translation and localization technology and services,
as well as a frequent contributor to TAUS Review.
Specialties: Telecommunications system and software
design with emphasis on IVR, wireless and multi-modal
communications. Translation and localization technology.
Reviews
Contributors
39
Nicholas Ostler
Nicholas Ostler is author of three
books on language history, Empires
of the Word (2005), Ad Infinitum (on
Latin - 2007), and The Last Lingua
Franca (2010). He is also Chairman
of the Foundation for Endangered Languages, a global
charitable organization registered in England and Wales.
A research associate at the School of Oriental and African
Studies, University of London, he has also been a visiting
professor at Hitotsubashi University in Tokyo, and L.N.
Gumilev University in Astana, Kazakhstan. He holds an
M.A. from Oxford University in Latin, Greek, philosophy
and economics, and a 1979 Ph.D. in linguistics from
M.I.T. He is an academician in the Russian Academy of
Linguistics.
Lane Greene
Lane Greene is a business and
finance correspondent for The
Economist based in Berlin, and
he also writes frequently about
language for the newspaper and
online. His book on the politics of language around
the world, You Are What You Speak, was published by
Random House in Spring 2011. He contributed a chapter
on culture to the Economist book “Megachange”, and his
writing has also appeared in many other publications. He
is an outside advisor to Freedom House, and from 2005
to 2009 was an adjunct assistant professor in the Center
for Global Affairs at New York University.
Jost Zetzsche
Jost Zetzsche is a certified English-
to-German technical translator, a
translation technology consultant,
and a widely published author on
various aspects of translation. Originally from Hamburg,
Germany, he earned a Ph.D. in the field of Chinese
translation history and linguistics. His computer guide
for translators, A Translator’s Tool Box for the 21st
Century, is now in its eleventh edition and his technical
newsletter for translators goes out to more than 10,000
translation professionals. In 2012, Penguin published
his co-authored Found in Translation, a book about
translation and interpretation for the general public. His
Twitter handle is @jeromobot.
Perspectives
Luigi Muzii
Luigi Muzii has been working in the
language industry for more than
30 years as a translator, localizer,
technical writer, author, trainer,
university teacher of terminology and
localization, and consultant. He has
authored books on technical writing
and translation quality systems, and
is a regular speaker at conferences.
Contributors
40
KantanMT.comKantanMT.com is a leading SaaS based statistical machine translation platform that enables users to develop and manage customized MT engines in the cloud.
BrauerTrainingTraining a new generation of translators & interpreters for the Digital Age using a web-based platform + cafeteria-style modular workshops.
Lingo24Lingo24 delivers a range of professional language services, using technologies to help our clients & linguists work more effectively.
AppenAppen is an award-winning, global leader in language, search and social technology. Appen helps leading technology companies expand into new global markets.
LionbridgeLionbridge is the largest translation company and #1 localization provider in marketing services in the world, ensuring global success for over 800 leading brands
CloudwordsCloudwords accelerates content globalization at scale, dramatically reducing the cost, complexity and turnaround time required for localization.
Directory of Distributors
ConcordeConcorde is the largest LSP in the Netherlands. We be-lieve in the empowering benefits of technology in multilin-gual services.
Kawamura InternationalBased in Tokyo, KI provides language services to compa-nies around the world including MT and PE solutions to accelerate global business growth.
KHAABBA International Training and Language ServicesKHAABBA is an LSP company for African languages based in Ethiopia.
Global TextwareExpertise in many disciplines. From small quick turn-around jobs to complex translation. All you need to com-municate about in any language.
Crestec Europe B.V.We provide complete technical documentation services in any language and format in a wide range of subjects. Whatever your needs are, we have the solution for you!
Hunnect Ltd.Hunnect Ltd. is an MLV with innovative thinking and a clear approach to translation automation and training post-editors. www.hunnect.hu
Capita TICapita TI offers translation and interpreting services in more than 150 languages to ensure that your marketing messages are heard - in any language.
Iconic Translation MachinesMachine Translation with Subject Matter Expertise. We help companies adopt MT technology.
iDiscEstablished in 1987, iDISC is an ISO-9001 and EN-15038 certified language and software company based in Spain, Argentina, Mexico and Brazil.
Larsen Globalization Ltd Larsen Globalization Ltd is a recruitment company dedi-cated to the localization industry since 2000 with offices in Europe, the US and Japan.
Jensen LocalisationLocalization services for the IT, Health Care, Tourism and Automotive industries in European languages (mostly Nordic, Dutch and Spanish).
HCRHCR works in conjunction with language partners to deploy software products and linguistic services globally in core industries such as IT, Automotive and more.
InterTranslationsIntertranslations LLC is based in Athens, London and Nicosia offering translation and localization services in all languages.
Linguistic SystemsLSI provides foreign language translation services in over 115 languages and unlimited subject matter. Contact us at 877-654-5006 or www.linguist.com
MateCatMateCat is a free web CAT tool for LSPs and translators. Use it to translate your projects or to outsource to over 120,000 professional translators in one click.
Memsource CloudAn API-enabled translation platform that includes vendor management, translatio memory, integrated machine translation, and a translator’s workbench.
IOLARFounded in 1998, Iolar employs 40 highly-skilled linguists and engineers specialised in translation of highly demanding documentation and software localisation.
CPSLMultilingual language provider for global strategies: translation, localization, interpreting, transcription, voice over & subtitling.
41
Directory of Distributors
MoraviaFlexible thinking. Reliable delivery. Under this motto, Moravia delivers multilingual language services for the world’s brand leaders.
Rockant Consulting & TrainingWe provide consulting, training and managed services that transform your career from “localization guy/girl,” to a strategic adviser to management.
MorphoLogic LocalisationMorphoLogic Localisation is the developer of Globalese, an SMT system that helps increase translation productivity, decrease costs and shorten delivery times.
Morningside TranslationWe’re a leading translation services company partnering with the Am Law 100 and Fortune 500 companies around the globe.
PacteraPactera is a leading Globalization Services provider, partnering with our clients to offer localization, in-market solutions and speech recognition services.
Safaba Translation Solutions, Inc.A technology leader providing automated translation solutions that deliver superior quality and simplify the path to global presence unlike any other solution.
WelocalizeWelocalize offers innovative translation & localization solutions helping global brands grow & reach audiences around the world.
SYSTRANSYSTRAN is the market historic provider of language translation softwaresolutions for global corporations, public agencies and LSPs
tauyou language technologyMachine translation and natural language processing solu-tions for the translation industry
SoveeSovee is a premier provider of translation and video solutions. The Sovee Smart Engine “learns” translation preferences in 6800 languages.
Directory of Distributors
STP Nordic TranslationSTP is a technology-focused Regional Language Vendor specialising in English, French, German and the Nordic languages. See www.stptrans.com.
TraductaNETTraductanet is a linguistic service company specialising in translation, software and website localisation, terminology management and interpreting.
sQuidsQuid help companies integrate and exploit translation technologies in their workflows and maximize their use of their language data.
TildeTilde develops custom MT systems and online terminology services, with special expertise in the Nordic, Baltic, Russian, and CEE languages.
PlunetPlunet GmbH develops and markets the business and translation management solution Plunet BusinessManager for professional LSPs and translation departments.
SeproTecSeproTec is a 25 years experience Multilingual Service Provider ranked among the Top 40 Language Service Companies in the world.
text&formtext&form is an LSP with expertise in software & multimedia localization, technical translation, terminology management and SAP consulting.
UTH InternationalUTH International is an innovative professional provider of globalization solutions and industry information, serving customers with advanced technologies.
Mirai TranslateMirai Translate will custom-build a translation A.I.which make innovation happen for your businessand create an exciting “MIRAI (future)”.
XTRFXTRF is a platform for project management, quoting, invoicing, sales and quality management, integrated with CAT, accounting and CRM tools.
Trusted TranslationsInternationally recognized leader in multilingual translation & interpretation services. Committed to providing clients with the highest quality service.
Win & WinnowProvider of translation, multimedia and desktop publishing services founded in 2004. We are one of the top ten language services providers in Latin America.
42
Get your insights, tools, metrics, data, benchmarking, contacts and knowledge from aneutral and independent industry organization.
Join TAUS!
TAUS is a think tank and resource center for the global translation industry.
Open to all translation buyers and providers, from individual translators, language service providers & buyers to
governments and NGOs.
taus.net
43
Upcoming TAUS Events
TAUS Roundtable6 October, 2015Washington, DC (USA)
TAUS Annual Conference12-13 October, 2015San Jose, CA (USA)
TAUS QE Summit San Jose14 October, 2015San Jose, CA (USA) hosted by eBay
Upcoming TAUS Webinars
TAUS Translation Technology Showcase QuickTerm & Terminotix2 September, 2015, 5 PM CET
TAUS Translation Quality WebinarCrowdshaping Translation Quality16 September, 2015, 5 PM CET
Ephemeral translations18 November, 2015, 5 PM CET
Translation Automation Users CallTranslation as a Utility25 September, 2015, 5 PM CET
TAUS HAUS
TAUS Office AmsterdamKeizersgracht 7430 July, 2015, 6 PM CET
Industry Events
Brand2Global30 Sept - 1 Oct, 2015London (UK)
LocWorld Silicon Valley14-16 October, 2015Santa Clara, CA (USA)
MT Summit XV30 Oct - 3 Nov, 2015Miami, FL (USA)
Do you want to have your event listed here? Write to [email protected] for information.
Industry Agenda
44
• A leader in translation and interpretation services and your guarantee for quality at the most competitive rates• Flexible solutions to accommodate your needs• A committed and unparalleled customer service
Innovation and continuous improvement is embedded in our corporate culture
Toll Free (US) 1-877-255-0717www.trustedtranslations.comsales@trustedtranslations.com