elra/elda eu enlargement and integration workshop arona, september 2005 victoria arranz 1 european...

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 1

European Language Resources Association

ELRA/ELDA: The Importance of Sharing

Linguistic Resources

Victoria Arranz

ELRA/ELDA55-57 Rue Brillat-Savarin, 75013 Paris, FranceTel. +33 1 43 13 33 33 -- Fax. +33 1 43 13 33 30

Email: arranz@elda.org

http://www.elda.org/ or http://www.elra.info/

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 2

Presentation Outline

• ELRA’s Foundation and Mission• ELRA’s Strategic Technical Activities:

– Issues Concerning Archiving and Providing of Language Resources for the HLT Community

– Administrative and Legal Issues Regarding Language Resources

– Involvement in the Evaluation of Human Language Technologies

– Promotion of the Language Technology Field

• Concluding Remarks

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 3

Centralized Non-profit organisation for the:• Collection, distribution and validation of LRs and

tools,

• Production or commissioning of production of LRs

• Evaluation of Human Language Technologies

European Language Resource Association An Improved Infrastructure for Data Sharing and HLT Evaluation

Operational agency ELDA:

Evaluation & Language Resources

Distribution Agency

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 4

Europe and the Multilingual Issue

More languages...Even within the same country

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 5

European Language Resource Association An Improved Infrastructure for Data Sharing

A Repository Center:Technical & Logistic issuesCommercial issues (prices, fees, royalties)Legal issues (Licensing, IPR)Information Dissemination

Infrastructure for the evaluation of Human Language Technologies

providing resources, tools, methodologies, logistics

Exit strategies / Capitalization on evaluation packages

An Association of users of Language Resources

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 6

LRs & Human Language Technologiesas seen by ELRA

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 7

Contrastive AnalysisPublic “role”- Streamlining public funds

AFTER CE Established ELRA(LRs CURRENTLY DISTRIBUTED See ELRA Catalogue)

Return on InvestmentRational Capitalization"Public" Assets

ACQUILEXARSGENELEXONOMASTICAPLUSPOLYGLOTREWARDSUNDIALSUNSTAR

BEFORE ELRA: No resources were made availableDuplication of efforts and funding

<== LRs NEVER DISTRIBUTED

ACCORCOST-232CRATERMULTEXTPAROLESPEECHDAT FamilyTSNLP(Euro)WordNet

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 8

Language Resources Activities

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 9

• Issues concerning Archiving and Providing of Language Resources for the HLT community:

– Maintenance of ELRA’s Catalogue– Development of the Universal Catalogue and Identification

of Useful Resources– Development of the Basic LAnguage Resource Kit

(BLARK)– Development of the Extended LAnguage Resource Kit

(ELARK)– Validation and Quality Assessment– Production or Commissioning of the Production of LRs

Language Resources Activities

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 10

Language Resources Activities

• Administrative and Legal Issues regarding Language Resources:

– Handling of Legal Issues related to the Availability of LRs

– Distribution of LRs and Pricing Policy

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 11

Language Resources Activities

• Involvement in the Evaluation of Human Language Technologies:

– Evaluation projects within ELDA

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 12

Language Resources Activities

• Promotion of the Language Technology Field:

– Information Dissemination, Promotion and Awareness, Market Watch and Analysis

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 13

Issues Concerning Archiving and Providing of Language Resources

for the HLT Community

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 14

The Universal Catalogue

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 15

Basic LAnguage Resource Kit (BLARK)

• Compilation of necessary LRs that should be available for all languages & regarding different Language Technologies and Modules

• Defined by Steven Krauwer and first launched through ELSNET with Dutch Initiative “Dutch Human Language Technologies Platform”

• In the framework of ENABLER thematic network (European National Activities for Basic Language Resources): ELDA elaborated report

• Sample: work done for Arabic resources within NEMLAR project: http://www.nemlar.org

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 16

ELRA’s BLARK

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 17

Validation and Quality Assessment

• Users need info on LRs: technical specifications, assurance of quality,...

• Validation is used by ELRA in reference to: checking the adherence to standards, QC of LRs, suitability for market

• Definition of validation methodology required:– Projects for the production of guidelines, standards and

specifications: EAGLES, PAROLE, SPEECHDAT, INTERVAL, ...

– Establishment of a Validation Committee (VCom)• SPEX (for speech resources): http://www.spex.nl/validationcentre/• CST (for written resources): http://cst.dk/validation/index.html

– Cooperation with key players with relevant expertise to act as validation units

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 18

ELRA’s Production of LRs

• Spoken Language Resources:

– CHIL

– C-ORAL-ROM

– OrienTel

– Neologos

– Speecon

– TC-STAR

– ESTER

– MEDIA

• Written LRs:

– Technolangue projects:

• Lexitec

• Euradic

– Technolangue/Evalda platform projects:

• Equer

• EASy

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 19

Administrative and Legal Issues Regarding Language Resources

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 20

Handling of Legal Issues Regarding LRs

• Support of lawyers for basic principles on LRs licensing• One of ELRA’s priorities: simplify rel. between providers & users• Drafting of generic contracts (with responsibilities & obligations) to

encourage producers/providers to distribute• Establish what usage is allowed for LRs: R&D or Research only?• Protect providers and their LRs• Are available on ELDA’s web site: http://www.elda.org/article69.html

• ELRA:– not owner of LRs & sets fair price with owner, based on:

• Production costs• Expected revenues• ELRA’s distribution policy: try to offer discounted price for members• ELRA’s effort to obtain low prices for R&D

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 21

Involvement in the Evaluation of Human Language Technologies

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 22

Levels of Evaluation (ELSE)

-Basic Research Evaluation (validate research direction)

-Technology Evaluation (assessment of solution for well defined problem)

-Usage Evaluation (end-users in the field)

-Impact Evaluation (socio-economic consequences)

-Programme Evaluation (funding agencies)

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 23

Reusability of Resources

Producer/Consumer BNT CLIR TTS SUMZ LM

Broadcast News Reuse Reuse   Reuse Reuse

Cross-Lingual Information Retrieval / Extraction

Reuse      

Text To Speech Synthesis     Reuse    

Text Summarisation   Reuse Reuse Reuse Reuse

Language Model Evaluation   Reuse Reuse Reuse Reuse

ELSE: Reusability of resources from one campaign to another

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 24

ELRA’s Activity on Evaluation

• Carrying out evaluation campaigns: further involvement at:– French (Technolangue/Evalda) and – European (CLEF, TC-STAR, CHIL, etc.) levels.

• Distributing evaluation packages

• Producing or commissioning the production of needed LRs for evaluation

• Conducted with the support of an Evaluation Committee: ECom

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 25

Evaluation Projects within ELDA

• CLEF 2004 campaign: http://clef.isti.cnr.it/

• TC-STAR: http://www.tc-star.org/

• CHIL: http://chil.server.de/servlet/is/101/

• Technolangue/Evalda: ARCADE II, CESART, CESTA, EASY, ESTER, EQUER, EVASY, MEDIA

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 26

Promotion of the Language Technology Field

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 27

Information Dissemination, Promotion and Awareness, Market Watch and Analysis

• Considerable effort on the promotion and dissemination of the LT field & its activities:– Partner/coordinator in several conferences,

meetings, workshops:• LREC Conference series (~900 participants)

• SCALLA 2004

• International Conference on Arabic Language Resources and Tools 2004

• Speech and Language System for Human Communication Conference 2004

• LangTech Conferences

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 28

Information Dissemination, Promotion and Awareness, Market Watch and Analysis

• Elaboration and distribution of the ELRA Newsletter, quarterly publication dedicated to inform readers about HLT events, projects, new LRs, etc.

• Maintenance and/or setting up of a number of web sites:– ELRA: www.elra.info– ELDA: www.elda.org– LREC: www.lrec-conf.org– COCOSDA: www.cocosda.org– NEMLAR conference: www.nemlar-conf.org– LangTech conferences: http://www.lang-tech.org/– Technolangue: www.technolangue.net

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 29

Information Dissemination, Promotion and Awareness, Market Watch and Analysis

• Public announcements posted to mailing lists

• Sponsorship activities

• Carrying out of market analysis and survey

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 30

Concluding Remarks

• Overview on major activities at ELRA and ELDA• Aim of ELRA/ELDA continue active role in

support of HLT community:– Supporting researchers and developers in their needs of

LRs

– Evaluation guidelines

– Validation guidelines

– Development of standards

– Working on the legal, distribution or pricing issues

– We believe: we’re moving towards the future needs of LT in an organised and productive manner

ELRA/ELDAELRA/ELDAEU Enlargement and Integration WorkshopArona, September 2005

Victoria Arranz 31

LREC-2006…

www.lrec-conf.org