eamt workshop 2015 - kantanmt

@KantanMT [email protected]

KantanMT Workshop

• Building your 1st KantanMT Engine

5 Minute Challenge


Data Models

E=mc - What’s it all about?2

The fundamental equation of statistical translation

Computational Challenge of SMT

Language Model

Translation Model

Optimal Search

• Bi-Lingual Data

• Glossary• NTAs• Stock Engines

• Mono-lingual Data

Types of Training

Bi-Lingual Data

Terminology

Mono-lingual Data

Training Data

* Translation Memories

* PDFs, DOCX, TXT

* TBX, XLSX

Stock Training Data

* 5 Billion Words available

Factors to Consider

• Training Data - Three main factors:Quality

• The linguistic quality of the training material is crucially important

Relevance to domain• A high quality MT system has good domain

knowledge• Similar to the way you’ve always worked with

Translation Memories and CAT tools

Quantity• The more training data you use to build your

engine the better its capacity to generate translations that mimic your translation style and terminology

• Getting the balance right!

Factors to Consider

Qu

alit

y


Another 5 min Challenge

• Building your 2nd KantanMT Engine

5 Min Challenge

• Building your 2nd KantanMT Engine

5 Minute Challenge


You’re not finished yet!

• Translating your 1st Document

2 Min Challenge


Measurements I

• F-Measure - Recall & Precision

Automated Measurements

Reference Translation

MT Output

Precision

correct

MT-Len

66%

Recall

correct

Ref-Len

80%

F-Measure

Precision * Recall

(Precision + Recall) /2

73%

• BLEU Score• Put simply – measures how many words overlap, giving

higher scores to sequential words

• High correlation between BLEU and human judgement of translation quality



MT Output


• TER (Translation Error Rate)• Min number of edits to transform output to match reference

• Levenshtein distance measure• General indicator of Post-Editing Effort


MT Output

TER

Substitutions + insertions + deletions

Reference-length

Comparative Measurements

• F-Measure Score• Recall & Precision calculation

• Closely linked to the relevancy of word selection for MT systems

Kantan BuildAnalytics™


• TER Score• A method to help predict the post-editing effort

• TER is quick to use and correlates highly with actual post-editing effort



• BLEU Score• Improvement upon F-Measure

• Takes word-order into consideration

• Linked to a sense of translation ‘fluency’




• Useful for • Engine Development

• Baseline measurements

• Determination of ‘possible’ engine quality and relevancy

• Reference set of comparative translations required

• Does not work on unseen translations

• Of limited use in determining • PE effort

• Resources

• Costs


KantanAnalytics


User Interface

KantanMT UX

eamt workshop 2015 - kantanmt

Data & Analytics

challenge measurements

considertraining data

translation memories

xlsx stock training

translation style

training material

postediting effort ter

linguistic quality