TAUS QE Summit – Dublin Jun 8Productivity Metrics – the proverbial minefield…

What data do we want to capture, and do we even care?

Translation Productivity | What metrics do we need?

• Hardly anybody buys a car without looking at the labels

• Plenty of confusion –

– mpg, km/l, l/100km

– fuel types & carbon emissions

– Urban, City, Highway

– But mostly agreed units and measurement practices, although…

• Are Translation Productivity Metrics like Fuel Consumption Data?

Productivity Metrics | Areas where they provide value

• Cost / Effort / Value – continuous validation of pricing models– Source Complexity– Fuzzy grids– Edit Distance Data– Ratios between per hour / per word

pricing, both for Selling and Buying

• Scheduling & Planning – accurate prediction of throughputs

• Resourcing – “Improvement Potential” of new

resources– Optimized Task assignments

Productivity Metrics | More questions than answers

• Bare Productivity (throughput) numbers don’t mean much without reference to things like– Complexity of the input– Quality of the output– The actual activity (T/E/P)– Identity (experience level of a resource) – Location– MT engine in use– Etc., etc.

• Large data sets will take care of outliers but we need to be able to further dissect

• Case Study done on a distinct program –– Fuzzy grid in use for a particular program

was inadequate– however, net new words throughput was

much higher at nearly 400 words / hour

Productivity Metrics | Challenges for an LSP

• Variability in ‘work types’, customer quality standards, resource qualifications, instructions, toolsets, MT engines, and more

• “Linguistic” productivities are only part of the total TAT calculation

• Most translation tasks are paid ‘per unit’

– Productivity (speed) is rarely a goal in itself

– We don’t instruct translators that they will be ‘measured’ for productivity

Agreed definitions and metadata on how to represent productivity metrics