language resources and their commercial applications kara warburton [email protected]

Language resources and theircommercial applications

Kara [email protected]

ISO/TC 37 Terminology and other language and content resources

My aim

Demonstrate the value of language resources for commercial applications

Discuss why standards for language resources are important

Present TC37 as a standards-developing organization

Warning – slight terminology bias!


Managing language resources

A language resource is Information expressed in a natural language Information that supports the interpretation of

natural language Language resources can enhance business

processes If properly deployed Requires interoperability, which in turn

requires standards.


Why me?

Implemented terminological resources, lexical resources, and standards for content interoperability in business environments - Terminologist for IBM, LISA contributor, business consultant

Developed standards and best practices for language resources: ISO TC37, LISA

Practical experience as a technical writer and translator – using language resources in increasingly technical environments


The cold reality

The computer age has generated exponential growth of information and knowledge.

Even with the aid of computers, we can’t manage this volume of information. Why? Computers can’t understand “natural” language. They only understand “1” and “0”.

Natural language is largely unstructured; even many structured language resources are “unpredictably” structured.

This environment demands increasing volumes of structured language resources to enable next-generation computing


Business scenarios for managing language resources

Translation memories Terminologies and lexical resources for enhancing

NLP applications Content management and retrieval

Content repurposing Content classification Normalized language Keyword management

Example: term extraction tool – use of “layered” lexical resources; grammatical rules; ranking algorithms


Managing terminology supports both social and commercial interests

Economic/commercial: Control terminology to ensure quality and

minimize production costs. Build terminological resources that are

repurposable across the content management chain.

Increase competitiveness in local and global markets.

Social/geopolitical: Strengthen and protect minority languages. Support cultural diversity. Increase global presence and visibility.


Is “managed” terminology really important for a business?

In the automotive industry, almost 50% of translation errors are “wrong term” (Woyde)

40% of time required for text production is terminology work (Stellbrink)

Between 30% and 70% of errors in technical documentation are terminology errors (Schutz, and MULTIDOC)

Terminology work is necessary for between 4% and 6% of all words in a text (Champagne)

Return on investment: 10% ($100 investment yields $110 return) (Champagne)

Outsourced translations may be 50% more expensive if source terminology is inconsistent (Kjeldgaard)


Need more proof?

Terminology tools increase productivity by approx. 20% (Champagne)

Without a central reference, each needless search can take 20 to 30 minutes (Champagne)

It costs 10 times more to fix a term at the end of the production cycle than at the beginning (Xerox, JDEdwards)

Inconsistent or inaccurate terminology raises service costs

Terminology mistakes can lead to lawsuits for copyright or trademark infringement, or for damages due to defective products or incorrect user documentation.


IBM scenario…

“Terminology work is necessary for between 4% and 6% of all words in a text (Champagne)”

429 million words are translated per year in IBM. Thus over 21 million words require attention.

In 2009, IBM “processed” over 160,000 terms as part of the “content conveyor belt”, in nearly 3,000 specialized “dictionaries”.

Very small staff

High degree of automation


What “measures” need to be taken?

Deploy a terminology database that serves multiple purposes

Integrate the database into all content environments to ensure a “push” mechanism

Respect data management principles, such as data granularity, elementarity, etc.

Adopt best practices for terminology, such as term autonomy and concept orientation

Allow for extensibility for features such as morphology as needed for future applications


Basic example – repurpose information

<h1>CI revision conflicts</h1><p>When revising your <term keyref="ci">CI</term>, to avoid conflicts...</p>

<glossentry id="ci"> <glossterm>configuration item</glossterm> <glossdef>An entity in a configuration that satisfies an end-use function and can be uniquely identified.</glossdef>

<glossBody> <glossAlt> <glossAcronym>CI</glossAcronym> </glossAlt> </glossBody></glossentry>


Controlled authoring


Controlled translation


Search – Query expansion


Source data…


Search – Query correction


Synonyms/inconsistencies multiply in the target language – this is bad for business

automatic memory reclamation

remise en état automatique du mémoire

récupération automatique de mémoire

automatic storage reclamation

remise en état automatique de l’archivage

remise en état automatique du stockage

récupération automatique de l’archivage

récupération automique du stockage

garbage collection récupération de place

vidage de la corbeille

récupération de place en mémoire

récupération de positions inutilisées

récupération de l’espace mémoire


“Cosmetic” differences can become more than cosmetic in the target language

1. pupitre d’administration

2. console d’administration

3. pupitre admin

4. console admin

5. pupitre administratif

6. console administrative

1. administration console

2. admin console3. administrative

console


Explosion of affected compounds…

administrative console application / administration console application

administrative console button / administration console button

administrative console login page / administration console login page

core administrative console / core administration console

....


Fixing the problem isn’t easy…

Change “pupitre” to “console”….. Le pupitre administratif est ouvert. Vous devez

le fermer. La console administrative est ouverte. Vous

devez la fermer.


Development of terminology resources is also key for language planning

Prescriptive terminology approach – just like in enterprise environments

The Canadian experience: Termium, the BTQ Other examples: Danterm, Korterm,

Eurotermbank Termbases feed into widely-distributed bulletins

and other distribution media to support adoption and language reinforcement As an educational resource For social and political policies As an aid to commerce


Effective management of language resources requires

adherence to standards and best practices


Interoperability requires adherence to standards

Interoperability between tools and applications: CAT tools vs controlled authoring, Web interfaces, GMS, ECM, search engines…

Interoperability between users – writers, translators, publicists

For delivering derivative products – glossaries, Web sites, etc.

For different purposes – learning, commercialization, government, social services, language planning, tourism, etc.

For different media – online vs paper, hand-helds, transport interfaces, broadcasting media, marketing collateral, etc.


Standards at various levels

Data transfer File format File structure (data model) Encoding Markup Syntax Semantics


ISO TC37 – Terminology and other language and content resources

Standardization of principles, methods and applications relating to terminology and other language and content resources in the contexts of multilingual communication and cultural diversity.

Web site…


TC37 Current focus areas

Word segmentation Language annotations to facilitate machine

processing Terminology policies Translation quality Simultaneous interpretation Data categories - www.isocat.org XML representation and exchange formats Persistent identifiers in multilingual

environments


Key standards and best practices – ISO TC37

ISO 30042 – TBX ISO 16642 – TMF ISO 12620 new ISO TC37 Data Category Registry ISO Concept Database ISO 704 – Terminology work: Principles and

methods ISO 12616 – Translation-oriented terminography ISO 26162 – Design, implementation and

maintenance of terminology management systems Annotation schemes and frameworks (SC4)


Training professionals in language resource management – an opportunity!

Lack of university training programs Lack of competency in existing fragmented

university courses Increasing demand for qualified professionals For example,

LISA offered 6 workshops (there was a demand for more) - 73 companies attended.

TermNet summer school – attendance grows each year


Thank you!

language resources and their commercial applications kara warburton [email protected]

Documents

value of language resources

terminological resources

natural language information

control terminology

managed terminology

content interoperability

terminology errors schutz

multidoc terminology