XLIFF in a large-scale international OECD study

Case studyBritta Upsing, Steve Dept,

Andrea Ferrari, Heiko Rölke

Overview

I. Background Information: The PIAAC Study

II. Test creation and Translation Processa. Overview of processb. Translation and Adaptation Tools

III. User experience: Challenges and lessons learnt

Very brief introduction to PIAAC

Programme for

International

Assessment of

Adult

Competencies

• Target Population: non-institutionalized adults aged 16 to 64

• 27 participating countries

• 35 national versions• 26 languages

Goals and Scope

• Goal: “(PIAAC) will assess the level and distribution of adult skills across countries, focusing on the cognitive and workplace skills needed for successful participation in today's work environment.“ (OECD)

• Means: Computer or paper-based tests and a computer-based background questionnaire

• Scope: at least 5000 completed cases (test takers) per country

• Project Timeline: 2008 – 2013

The cognitive tests

- computer- or paper-based tests to measure literacy and numeracy skills

- computer-based test to measure how well participants solve problems using computers.

Example: Computer-based item

Overview

I. Background Information: The PIAAC Study

II. Test creation and Translation Processa. Overview of processb. Translation and Adaptation Tools

III. User experience: Challenges and lessons learnt

Test creation and translation process

Tests to be comparable across language versions-> “Master versions” developed centrally, translation in countries•Translators should not change the layout•Layout should be separated from translation-> XLIFF generated from item descriptions

Item production and translation process

Web-basedTranslation

portal

XLIFF

Translation/Reconciliation

tool (OLT)

XLIFF

Offline test authoring tool(xliff 1.1 generated

from item descriptions)

Separation of content and text

How

much…?

3/5

◄ ? ►

Combien de …?

3/5

◄ ? ►

Hoeveel …?

3/5

◄ ? ►

Wie viele …?

3/5

◄ ? ►

Translation Process• Web-based Portal for Translation

Country1_Language1Country1_Language1 Country1_Language2Country1_Language2

Translation & ReconciliationTranslation & Reconciliation Translation & ReconciliationTranslation & Reconciliation

Scoring AdaptationScoring Adaptation Scoring AdaptationScoring Adaptation

VerificationVerification VerificationVerification

Layout AdaptationLayout Adaptation Layout AdaptationLayout Adaptation

Final Sign OffFinal Sign Off

Final Check…..Final Check….. Final Check……Final Check……

More than 30,000 archived XLIFF files

Several hundreds of thousands temporary XLIFF files

60 translators, 30 reconcilers, 30 verifiers….

Translation and Adaptation Tools

• Item Management Portal to handle the workflow, uploading of xliff and preview of different translation versions

• Open Language Tool to translate xliff locally and offline

• Tag editors to translate scoring information and to adapt the layout (done centrally)

Web-based Translation Portal

Open Language Tool

Overview

I. Background Information: The PIAAC Study

II. Test creation and Translation Processa. Overview of processb. Translation and Adaptation Tools

III. User experience: Challenges and lessons learnt

User experience• For the PIAAC project:– Most problems were related to the translation and

adaptation tools, not to the xliff standard– The challenge remains that tools were – Either costly–Or insufficiently tested–Or not adapted for a wide audience–Or did not make the most of what the standard offers

Carrying information• Management of errata and successive source

versions is a tough nut to crack• Maintenance of TMs posed a variety of

unexpected challenges• Segment status was used to “track changes”

-> are there better ways to use xliff as a vehicle for this information?

Handling of tags

Handling of tags

• In tests with scoring blocks, the proportion of tags versus text was terrifying

• inline formatting that involves tag edition seemed difficult to handle for translators and led to corrupt xliff

• corrupt XLIFF files sometimes detected late in the process, so that a complex set of motions had to be repeated

Handling of layout

Strict separation of layout and text content• implies a lot of work on pre-press but is a • better alternative than translation in MS Word,

where translators introduce layout problems– Safe export of xliff from MS Word or InDesign

+ seamlessly re-import the translation? – Investigate extracting from e.g. an ENG document

and re-importing as RTL language

Dynamic Text• Definition of Dynamic Text Rules e.g. to

generate two identical text segments in the XLIFF each time gender-specific entries need to replace gender-neutral entries

Some lessons learnt, next steps

For a new study…..• Use of alt-trans to store translation and

adaptation history• Customized User guides developed on a need-

to-know basis (for different roles)• Extension of notes– Tool support– Read-only notes

Thank you!

Questions?