roi and the value of machine translation
TRANSCRIPT
Lionbridge Technology Overview:Technology Best Practice
Presented by Nicholas McMahonModerated by Paula ShannonJuly 15, 2009
Three Types Of Machine Translation
Rules BasediTranslatoriTranslator
StatisticalGoogleGoogle
HybridSystran Enterprise
Company confidential – distribution prohibited without permission
Server 7
Top Three Considerations for MT
MT performance in one l i NOT MTlanguage is NOT MT
performance in another language or even in another
language direction
Company confidential – distribution prohibited without permission
Top Three Considerations for MT
MT is not an
“every language solution” for any of the MT Engines
Company confidential – distribution prohibited without permission
Top Three Considerations for MT
MT at an Enterprise Level
is NOT free
Company confidential – distribution prohibited without permission
Top Three Applications of MT:Chat
MT Chat ApproachesHuman to human instant dialogueHuman to semi auto response based onresponse based on key word findings
EvolutionEarly adoption – Value focus, very little distinct ‘ROI’ modeling
Company confidential – distribution prohibited without permission
modeling
Top Three Applications of MT:Support Content
Company confidential – distribution prohibited without permission
Top Three Applications of MT:Support Content
FAQ MT ApproachesRaw MT output for un‐ptranslated contentTiered Quality approach y pp
EvolutionVolume Success and reasonable adoptionVolume Success and reasonable adoption“Self serve” & “New content” increases user patience
Company confidential – distribution prohibited without permission
Top Three Applications of MT:User Guides
Company confidential – distribution prohibited without permission
Top 3 Applications of MT
Approaches
TM, MT + Post Editing of content
Considerations
Early Adoption – strong indicators of success
Goal is to MT to be ‘invisible’ inGoal is to MT to be invisible in process
Company confidential – distribution prohibited without permission
POLLING QUESTION
Are you currently considering or using MT in any of these situations?
Considering MT for Chat
C id i MT f FAQConsidering MT for FAQs
Considering MT for Volume User Content
Using MT for ChatUsing MT for Chat
Using MT for FAQs
Using MT for Volume User Contentg
Company confidential – distribution prohibited without permission
Enterprise Cost Considerations
“Start up Costs” Costs
Data Identification/ Corpus creation $0k ‐ $100k
Customization or training of content $5k ‐ $250k$ $
Licensing fees $0k ‐ $200k
Integration fees $10k ‐ $200k
Company confidential – distribution prohibited without permission
Enterprise Cost considerations
“Post Editing” Costs
Training of resources for post editing $5kTraining of resources for post editing $5k ‐ $10k
P t Editi t 65% 85% fPost Editing rates – 65% ‐ 85% of new word rates
Ongoing customization/ corpus g g / pmanagement annually $5k ‐ $200k
Company confidential – distribution prohibited without permission
ROI modeling for MT within Instant communication
User 1 User 2
My Second figure is an animal My role of a young handsome y gwith a tail and four feet
y f y gbeau is a boy with a tail and 4 feet
h ki d f i d i h d ? h dWhat kind of meaning and nice boy?
What do you mean? A handsome boy?
My second figures looks like an My second figure is like anMy second figures looks like an animal
My second figure is like an animal
It has four feet and a tail It has four feet and a tail
Oh, I understand and am the 8th I got it, it is my 8th figure
Company confidential – distribution prohibited without permission
Source: NTT Labs and Kyoto University,Effects of Machine Translation on Collaborative Work, Dec 06
ROI modeling for MT within Instant communication
Establishing the value of instant communications
Abilit t l t tAbility to resolve support requests Cost per multilingual call handling
Ability to communicate betterAbility to communicate betterValue of increased time to market
Company confidential – distribution prohibited without permission
ROI Modeling for MT within Instant Communication
MT in Chat Benefits VariablesStrong defined reference sets produce better resultsStrong defined language use (specific to a domainuse (specific to a domain or industry)Patience and Flexibility of user (not for complaints!)Favors Value of ‘currency’ of content versus qualityof content versus quality
Company confidential – distribution prohibited without permission
ROI modeling for MT within Instant communication
Results Defined Data Sets
5 ‐ 25% average success rate (resolution of issue)
User Satisfaction generally 30 –50%50%
Company confidential – distribution prohibited without permission
ROI modeling for MT within Instant communication
ROI Model – Statistical MT considered best match
10 Languages, 100,000 chat support requests
$Alignment of data for Corpus Creation: $30kProduct Licenses $100kIntegration of tools $20kIntegration of tools $20kHuman system/ process training $10kSystem tweaking : $20ky g $
Total Cost $180k
Company confidential – distribution prohibited without permission
ROI modeling for MT within Instant communication
Averaged Results for Calculation
10% support cases resolved
Average cost of alternate solution: $15 (FAQ routing/ human Call)
Benefit
100 000 * 10% * $15 $150 000100,000 10% $15 $150,000
Less Costs $180,000,
Basic Financial ROI Year 1 ‐$30,000
Company confidential – distribution prohibited without permission
POLLING QUESTION
Based on the considered MT chat model do you think the suggested ROI is:
Very Low
LLow
About Right
HighHigh
Very High
Company confidential – distribution prohibited without permission
ROI modeling for MT applied to FAQs
Establishing the value of FAQs?
Ability of users to autonomously resolve support requests
Increased Customer Satisfaction
Company confidential – distribution prohibited without permission
ROI modeling for MT applied to FAQs
Benefits VariablesStrong defined reference sets produce better resultsproduce better resultsQuality of source content constructionExpectations of user –disclaimers on qualityFavors un‐translated contentFavors un translated contentRules based or Hybrid best solutionsProvision of English source linksProvision of English source links provide additional contextCultural expectations
Company confidential – distribution prohibited without permission
Volume of source data available
ROI modeling for MT applied to FAQs
Results Defined Data Sets20% ‐ 35% average success rate (resolution of issue)
On average about 8%On average about 8% less than English
User Satisfaction ll 50 65%generally 50 – 65%
Company confidential – distribution prohibited without permission
ROI modeling for MT applied to FAQs
ROI Model – Hybrid MT Engine considered
10 Languages, 100,000 FAQs docs 450 words per doc.
$Alignment of data for Corpus Creation: $50kProduct Licenses $250kIntegration of tools/ solution $60kIntegration of tools/ solution $60kRating, feedback reporting $20kHuman system/ process training $10ky / p g $System tweaking : $50k
Company confidential – distribution prohibited without permission
Total Cost $390k
ROI modeling for MT applied to FAQs
Averaged Results for Calculation :
25% support cases resolved
Average cost of alternate solution: $35(Human Call Response)
Benefit
100,000* 25% * $35 = $875,000
Less Costs $390000
Basic Financial ROI Year 1 $485,000
Company confidential – distribution prohibited without permission
POLLING QUESTION
Based on the considered MT FAQ model do you think the suggested ROI is:
Very Low
LLow
About Right
HighHigh
Very High
Company confidential – distribution prohibited without permission
ROI Modeling for MT applied to User Guides
Establishing the value of MT within User DocumentationImprovement in output volumeReduction in total attracted cost of translationIncreased user content available
Company confidential – distribution prohibited without permission
ROI Modeling for MT applied to User Guides
Benefits VariablesStrong defined reference sets produce better resultsQuality of source content constructionconstructionKnowledge and skills of human resourcesSystems and tools integration of MT and TMTarget Language complexityTarget Language complexity
Company confidential – distribution prohibited without permission
ROI Modeling for MT applied to User Guides
Results Defined Data Sets‐10% ‐ +25% productivity improvement
Reduction in total attracted cost per wordp
Company confidential – distribution prohibited without permission
ROI Modeling for MT applied to User Guides
ROI Model – Rules Based MT considered
4 Languages, 2,000,000 words total
$Alignment of data for Corpus Creation: $5kProduct Licenses $20kHuman system/ process training $20kHuman system/ process training $20kSystem tweaking : $5k
Total Cost $50k
Company confidential – distribution prohibited without permission
ROI Modeling for MT applied to User Guides
Averaged Results
15 % increased in productivity
Average fully loaded per word cost $0.25
(accounting for fuzzy matches)
Benefit
2 000 000 * 15%* 0 25 $75 7502,000,000 * 15%* 0.25 = $75,750
Less Costs $50,000Less Costs $50,000
Basic Financial ROI Year one $25,750
Company confidential – distribution prohibited without permission
(about 10% of expected costs)
POLLING QUESTION
Based on the considered MT user guide model do you think the suggested ROI is:
Very Low
LLow
About Right
HighHigh
Very High
Company confidential – distribution prohibited without permission
MT ROI Take Away
E l Ad ti b t MT b d j t tiEarly Adoption, but MT based projects are reporting
MT solutions are dramatically improving
MT source data is growing exponentiallyMT source data is growing exponentially
Company confidential – distribution prohibited without permission
Knowledge Center
www.lionbridge.comwww.lionbridge.comDownload White Papers
• Building Stronger Brands Around the World: A Guide to Effective Global Marketing
• Strengthening Global Brands: Key Steps for
Download White Papers
Meaningful Communications around the World
• Building a Global Web Strategy: Best Practices for Developing your International Online Brand
• Machine Translation: Saving Time Lowering Costs
View Webinars
Machine Translation: Saving Time, Lowering Costs, and Improving Services
• Localization 2.0
• The Art and Science of Global Navigation
• The Best Global Web Sites (and Why)
• Localization 2.0
• The Art and Science of Global Navigation
• The Best Global Web Sites (and Why)
Company confidential – distribution prohibited without permission