keynote presentation at mtsr07

Gauri Salokhe

http://www.fao.org/aims/

Work on Agricultural Information Management at FAO (2000 – to date)

Gauri Salokhe

[email protected]

Gauri Salokhe


FAO’s Work

• Goal of FAO is to reduce hunger in the world and to improve living conditions

• FAO itself produces huge amounts of content in its subject areas

• The First Article of the FAO Constitution:“The Organization shall collect, analyse, interpret and disseminate information relating to nutrition, food and agriculture.”

Gauri Salokhe


Starting Points in 2000• AGRIS (FAO Documentation)

• AGROVOC– Is there any need to invest in metadata and

vocabularies in the age of electronic information?

First Steps:• Brussels meeting (on Standards for electronic

publishing) and AGstandards initiative• First participation in Dublin Core initiative

Gauri Salokhe


Our Work

Metadata Standards

Gauri Salokhe


Resources we are dealing with.. • Document – Like Information Objects

(DLIOs) • Learning resources • Organizations• Projects• News and Events

Gauri Salokhe


Document – Like Information Objects - About

• Starting point was the AGRIS Database, which gathers bibliographic information from over 200+ centres from across the world

• It currently contains 2.5 million bibliographic records.

• Before the year 2000, participating centers were forced to use a proprietary software or complicated format conversions were necessary at the central database

Gauri Salokhe


center3

center1

center2

NEW

service2

common exchange layer

serviceetc.

center2

center1 center3

NEW

services

The Idea

Gauri Salokhe


Document – Like Information Objects – Our Proposal

• The AGRIS Application Profile (AGRIS AP) a standard created specifically to enhance exchange of metadata

• It is a metadata schema which draws elements from well known Metadata standards such as – Dublin Core (DC)– Australian Government Locator Service Metadata (AGLS)– Agricultural Metadata Element Set (AgMES) namespaces

• It allows sharing of information across dispersed bibliographic systems and provides guidelines on recommended best practices for cataloguing and subject indexing.

Gauri Salokhe


AGRIS AP, extends the DC

Gauri Salokhe


Why not simple DC?CitationFor example, the citation information using AGRIS AP is displayed as: <ags:citation> <ags:citationTitle>Journal of Agricultural Research and Extension

(Thailand)</ags:citationTitle> <ags:citationTitle>Warasan Wichai Lae Songsoem Wichakan Kaset</ags:citationTitle> <ags:citationIdentifier scheme="ags:ISSN">0125-8850</ags:citationIdentifier> <ags:citationNumber>18(2) p.1-12</ags:citationNumber> <ags:citationChronology>Apr-Sep 2001</ags:citationChronology></ags:citation>

The dumbing down process would result in the information being merged and presented as:

<dc:relation> Journal of Agricultural Research and Extension (Thailand); Warasan Wichai Lae Songsoem Wichakan Kaset; 18(2) p.1-12; ISSN: 0125-8850; Apr-Sep 2001</dc:relation>

Gauri Salokhe


Why not simple DC?RelationFor example, the relation information using AGRIS AP is displayed as: <dc:relation> <dcterms:isVersionOf>http://www.fao.org/agris/agmes/DC1-FAO1.doc

</dcterms:isVersionOf > <ags:relationHasTranslation>ftp://ftp.fao.org/fao/W8270c.pdf

</ags:relationHasTranslation></dc:relation>

The dumbing down process would result in the information being merged and presented as:

<dc:relation> http://www.fao.org/agris/agmes/DC1-FAO1.doc</dc:relation> <dc:relation> ftp://ftp.fao.org/fao/W8270c.pdf </dc:relation>

Gauri Salokhe


Current Status

• FAO has converted the central database to XML

• Now all active AGRIS Centres are sending data in AGRIS AP XML format.

• The standard has been adopted by: – Global Forestry Information System (GFIS)– Global Forum on Agricultural Research (GFAR)

Gauri Salokhe

http://www.fao.org/aims/http://oai.bibsys.no/repository?

verb=listRecords&metadataPrefix=agris&set=agris

Gauri Salokhe


Data Management

Data Management

ArchivalStorage

ArchivalStorage

AccessAccess

DescriptionDescription

CataloguingCataloguing

MetadataExport

MetadataExport

Dublin Core Unqualified

Dublin Core Unqualified

MARC21MARC21

AGRIS APAGRIS AP

Self-archivingSelf-archiving

The AGRIS-AP & the new AGRIS network

Gauri Salokhe


Figure 2. Agris Network: Content Management. The Figure 2 shows the main elements in the workflow for AGRIS Network and their connections following the new architecture. Conceptually, the workflow is divided in two principal actions: content management and exposing metadata.

InformationManagement

Specialist

InformationManagement

Specialist

Metadata+

Full text

Metadata+

Full text

SoftwareSoftware

AGRISWebsite

AGRISWebsite

AgrisService

Providers

AgrisService

Providers Standards

Standards

UsersUsers

ResearcherOpen Access

ResearcherOpen Access

ExternalService

Providers

ExternalService

Providers AgrisData

Providers

AgrisData

Providers

ContentManagement

ContentManagement

Self ArchivingSelf Archiving

KnowledgeExchange

KnowledgeExchange

DirectoriesDirectories

MethodsMethods

Internet

The AGRIS-AP & the new AGRIS network

Gauri Salokhe


DLIO - Tools

ME

TAMAKE

R

Gauri Salokhe


DLIO – Services (http://www.fao.org/agris/search)

Gauri Salokhe


Document – Like Information Objects – Next Steps

»AGRIS Ontology Project

Gauri Salokhe


Services – Navigation of objects

• Cross-searching of objects using AGROVOC Keywords

Gauri Salokhe


Services – One click re-query of Data

• Traversing the URIs

Gauri Salokhe


Learning resources - About• The United Nations General Assembly

recognizes the crucial role of capacity building for achieving the Millennium Development Goals

• It has called upon the United Nations organizations to increase their support to developing countries’ own efforts.

• Capacity and institution building is a core function of FAO

Gauri Salokhe


Learning resources – Our Proposal

• In order to provide structured access to FAO’s agricultural learning resources and capacity and institution building services, the “Capacity and Institution Building Portal” project was started in 2006.

• A metadata standard was needed to assure that each resource will be described appropriately.

Gauri Salokhe


Learning resources - the metadata• (DC) Title

– (AGS) Supplement Title• (DC) Creator• (DC) Subject

– (AGS) subjectCategories– (AGS) subjectThesaurus

• (DC) Description– (DCTERMS) Abstract– (AGS) Notes

• (DC) Publisher • (DCTERMS) Date• (DC) Type• (DC) Format-Type• (LOM) Aggregation Level

• (LOM) Size• (DC) Identifier• (DC) Language• (DC) Relation• (DCTERMS) Coverage • (DC) Rights• (LOM) Cost• (LOM) Intended End User Role• (LOM) Context• (LOM) Interactivity Level• (LOM) Typical Learning Time

Gauri Salokhe


Learning resources – Next Steps• The “Capacity and Institution Building

Portal” will be made available to FAO staff and member countries towards the end of the year

• Approximately, 300+ records have been identified in FAO as learning resources. Metadata for these resources is being created according to the Learning Resources AP

• Strong collaboration with the UNESCO (and CGIAR) initiatives

Gauri Salokhe


Gauri Salokhe


FAO and Dublin Core• Dublin Core Education Metadata Working Group

– Review the Dublin Core Education Profile – FAO is contributing Use Cases

Gauri Salokhe


Learning Resources Task Force• To set up a network of organizations to

facilitate sharing and reusing of learning resources

• To provide guidance, standards, technologies, tools, recommendations, and best practices.

Gauri Salokhe


Goals of the Task Force• Set up a community• Create an inventory • E-conference• Producing an initial set of best practice

recommendations• Deploying a pilot demonstrator of federating

learning repositories metadata

Gauri Salokhe


Our aims is to support sharing!• Data Provider: exposes resources

– Development Agencies and NGOs– Research Institutions– Industry Information Centres

• Service Provider: provides services based on these resources– Aggregators– Thematic or Regional views

Gauri Salokhe


Organizations - About• Information on agricultural organizations is available

from many different information services with independent databases.

• Very few of these systems share data among themselves and in no case do they coherently integrate data.

• This translates into three major difficulties: 1. maintenance is costly; 2. similar (but incomprehensive) information is

available from several sources; 3. visibility in all the existing databases requires

submitting the same data to multiple databases.

Gauri Salokhe


Organizations – Our Proposal

1. Provide a web-based tool that allows everyone to create metadata about their organization.

2. The tool will come with guidelines on "how-to" create the metadata file.

3. The XML file will be stored on institutes’ webserver and the URL will be registered in a central server

• The central server can be exploited to create other value-added services as desired.

• Periodic update requests will be sent out to all the organizations.

Gauri Salokhe


Organizations – the metadata• (AGS) organizationName

– (AGS) fullOrganizationName

– (AGS) organizationAcronym

• (AGS) address– (AGS) streetAddress– (AGS) country

• (AGS) telephone• (AGS) fax• (AGS) telex• (AGS) email• (DC) identifier (scheme

“dcterms:URI”)

• (DC) description• (DC) subject

– (AGS) subjectThesaurus• (AGS) organizationType• (DC) relation

– (DCTERMS) isPartOf (sugg: scheme “dcterms:URI”)

– (DCTERMS) replaces (sugg: scheme “dcterms:URI”)

• (DC) date– (DCTERMS) created– (DCTERMS) modified

Gauri Salokhe


Organizations – Next Steps• The “metamaker”

• The registry is being developed by our partner organization the Global Forum on Agricultural Research (GFAR)

• Data will be collected by targeting the partner organizations (GFAR, GFIS etc.)

Gauri Salokhe


Projects - About• Repositories of Project metadata exist but...

1. maintenance is costly; 2. similar (but incomprehensive) information is

available from several sources; 3. visibility in all the existing databases requires

submitting the same data to multiple databases.

• Promote data sharing

Gauri Salokhe


Projects – Our Proposal

• To develop core data standard • Provide user friendly documentation and best practices • Provide machine-readable encoding (such as eXtensible

Markup Language) to allow automatic validation of shared information.

• Partner organization (WIS International), with FAO and GFAR, is developing a proposal for the metadata standard.

Gauri Salokhe


Projects – the metadata• Project Name• Project Code : What is on the

document• Project Description• Project Objectives• Project Subject

– subject categories (ASC)– keywords (AGROVOC)

• Project Budget• Project Duration

– start date– end date– extension date

• Project identifier (URL)

• Project status • Project coverage.spatial • Donor(s)• Participating organizations :

Organizations that have received funding and are carrying out the project.– name– acronym– address (City, Country)

• Coordinator– Contact name– Contact organization– Contact email– Contact telephone

Gauri Salokhe


Projects – Next Steps

• Get an agreement from partners on the elements

• Provide a XML DTD for validation of the metadata files

• Promote the standard as a possible “means” for sharing project information without going into too many details

• Create a registry

Gauri Salokhe


News and Events - About• An event is defined here as “something that

happens at a given place and time.” An event can be broken into different ‘subsets’, for example by day or session. In this document, it will be the larger entity that will be addressed.

• For example, some of the events in FAO are: – Eighth AOS Workshop -- 7 years of AOS:

Achievements and Next Steps, Rome, Italy, 21-22 September 2007

– Special Session on Agricultural Metadata & Semantics : 2nd International Conference on Metadata and Semantics Research (MTSR'07), Greece, October 11-12, 2007

Gauri Salokhe


News and Events - About• At present, the sharing is done in a ad-hoc

manner, leading to specialized formats and incompatible exchange models.

• the exports are done without using a standard set of elements.

• This limits creation of value-added services such as calendars.

Gauri Salokhe


News and Events – Our Proposal

• In the case of News, we use standard RSS news format. The basic requirements are elements provided by the RSS standard: <link>, <title>, and <description>

• In the case of Events, an expanded metadata set was proposed.

• RSS feeds that comply with the proposal can be

registered at www.agrifeeds.org

Gauri Salokhe


News and Events – the metadata• (AGS) dateStart

• (AGS) ags:dateEnd

• (AGS) location

– (AGS) locationCity

– (AGS) locationCountry

• (DC) type

• (AGS) organizer<ags:dateStart schema="dcterms:W3CDTF">2007-09-21</ags:dateStart> <ags:dateEnd schema="dcterms:W3CDTF">2007-09-22</ags:dateEnd> <ags:location> <ags:locationCity>Rome</ags:locationCity> <ags:locationCountry schema="dcterms:ISO3166">ITA</ags:locationCountry> </ags:location><dc:type>Workshop</dc:type> <ags:organizer>Food and Agriculture Organization</ags:organizer>

Gauri Salokhe


News and Events – Next Steps

• Agrifeeds launched in September 2007

• The “metamaker” is under construction (GFIS)

• Data will be collected by targeting webmasters (GFAR, GFIS etc.)

Gauri Salokhe


News and Events – Tools

Gauri Salokhe


Our Work II

Vocabularies

Gauri Salokhe


Agricultural Domain

AGROVOC

NAL Thesaurus

CABI Thesaurus

Dedicated KOSs

Non-dedicated KOSs

e.g., ASFA thesaurus

e.g., the Multilingual Forestry Thesaurus

e.g., the Sustainable Development

website classification

e.g., biological taxonomies such as NCBI and ITIS

GEMET

Other thematic thesauri

Existing Thesauri and Knowledge Organization Systems (KOSs)

Common concepts are not declared

No or very limited interoperability

Insufficient subject + language coverage

Severe maintenance problems

Very limited machine readability

Only very simple encoding of semantic relations

Gauri Salokhe


AGROVOC Development (1982–2007)

Edition, Year no. of descriptors1st ed., 1982 8 660

2nd ed., 1992 14 714

3rd ed., 1995 16 107

4th ed., 1999/2000 16 607

Electronic ed., 2007http://www.fao.org/aims/ag_intro.htm

28 435

Gauri Salokhe


Development (1982–2007)

Descriptor growth in AGROVOC (En)

28435

14714

8660

16107 16607

0

5000

10000

15000

20000

25000

30000

1st edition (1982) 2nd edition (1992) 3rd edition (1995) 4th edition (2000) E-Edition (2007)

Year

Des

crip

tors

Gauri Salokhe


Multilinguality

0

5000

10000

15000

20000

25000

30000

35000

40000

45000

AR

EN

ES

FR

ZH

CS

DE

HU

JA

PT

SK

TH

Languages

Tota

l Num

ber

of C

once

pts

Non-Descriptors

Descriptors

Total number of concepts

Gauri Salokhe


Formats• AGROVOC is available in following formats for free

downloading (non-commercial use):• MySQL • MS Access• SKOS• Postgres • OWL• TagText• ISO2709• others

• Up to now 500 users have downloaded AGROVOC (in 14 months ~ 35 downloads each month)

• AGROVOC Web services allows users to connect directly to the AGROVOC Thesaurus

Gauri Salokhe


i

Gauri Salokhe


AGROVOC Concept Server• Concept-based (as opposed to term-based)

• Enriched semantics in RT relationships

• Develop specialized sub-domain ontologies, such as − Fisheries ontology− Thai plant ontology − Food safety ontology − Crop-wild ontology− Geopolitical ontology for FAO information systems− Organization ontology, etc.

Gauri Salokhe


Concept Terms Strings

Concept

Relationshipsbetweenconcepts

Lexicalization/Term

String

Relationshipsbetweenstrings

Relationshipsbetweenterms

designated by

manifested asOther information:language/culture

subvocabulary/scopeaudiencetype, etc.

Note

annotation relationship

Relationship

RelationshipsbetweenRelationships

Gauri Salokhe


USE/UFConceptc_167term

i_en_buffaloes_syncerus

Buffaloes (syncerus)

termi_en_african_buffaloes

termi_es_búfalo_africano

Búfalo africanoAfrican buffaloes

has_lexicalization

Conceptc_1040

subclassofConceptc_3182

RTs....

termi_zh_ 非洲水牛

非洲水牛

rdfs:label:

termi_ja_ アフリカ水牛

アフリカ水牛

termi_es_búfalo_syncerus

Búfalo (syncerus)

has_synonymhas_translation

Gauri Salokhe


AGROVOC Concept server Workbench

.doc, .pdf, .xml, etc.

Concept Hierarchy

ipun

t

AOS/CS Workbenchconcordance pattern-matchingmultilingual

text corpus

input

Gauri Salokhe


Geo-Political Ontology

• Names– official and short

• names:– Arabic,– Chinese,– English,– French,– Spanish,– other....

• Codes– UN code – M49

– ISO-3166 Alpha-2

– and Alpha-3

– UNDP code

– GAUL code

– FAOSTAT

– AGROVOC

• Coordinates– Max, Min Latitude

– Max, Min Longitude

Gauri Salokhe


Requirements – dynamics• To track historical changes

West Germany

East Germany

Germany

From 1990 to present

Czech Republic

Slovakia

Czechoslovakia

From 1993 to present

Gauri Salokhe


Requirements - neighboring territories• To compare data on certain subjects

– India neighboring countries: Bangladesh, Nepal, Pakistan, etc.

Gauri Salokhe


Gauri Salokhe


Why all this?

Gauri Salokhe


Our Vision

Shared layer of interoperability


Value-addedInformation

Services


Services

Aggregated

Database View

Aggregated

Database View

Subject-specific Portals


Open URL or OAI based

services


services

Information System

(n)

Information System

(n)

Web Services

Web Services

DistributedDatasets

DistributedDatasets

DatabaseDatabase

...Database

Database WebsitesWebsites

Metadata ontologies (Application Profiles)Subject ontologies

XML BusXML BusA

OS

Gauri Salokhe


Lessons Learned

Gauri Salokhe


Don’t Give Up!• We had to struggle with acceptance in-house

• Implementation of the AGRIS AP

– many discussions about:• too many elements• too complicated

• ....but we did not surrender

Gauri Salokhe


Don’t Give Up!• AGROVOC Model

– SKOS discussions (Our model was too complicated)

– Some weeks ago Alistair Miles from W3C working group sent out a proposal for Label relations

• Some Failed projects that we can learn from– Antimicrobials Online

– SWTFOS

– ECOTERM

– IPFSAPH

Gauri Salokhe


Ontologies on demand only!

• First AOS workshop decided about Ontology prototypes (Fishery, Food Safety...)– Fishery:

• lot of conceptual work without real demand or application

• bulky construction with no real use case

• NeOn has the involvement of partners and application

– Food Safety: • a food safety ontology was created using AGROVOC, expert

knowledge and text mining, but it did not fit the needs of the application (IPFSAPH)

• These were useful experiences, but not the way to go for developing full fledged ontologies

Gauri Salokhe


Our work and the Semantic Web

Gauri Salokhe


Recent remarks.. • Of Tim Berner’s Lee

– A web of data– Channelling existing structured data into the semantic

web– Using existing large databases– Creating data exchange profiles

http://blogs.zdnet.com/Berlind/?p=518&tag=nl.e622

Gauri Salokhe


Our Vision




Services


Services

Aggregated

Database View

Aggregated

Database View




services


services

Information System

(n)

Information System

(n)

Web Services

Web Services

DistributedDatasets

DistributedDatasets

DatabaseDatabase

...Database

Database WebsitesWebsites

Metadata ontologies (Application Profiles)Subject ontologies

XML BusXML BusA

OS

Gauri Salokhe


Semantic Web“space”• Not a semantic web, but a semantic space in the

web for communities (Semantic Applications)

• Network of Data and Service Providers with agreed procedures and standards (common ontological layers)

Gauri Salokhe


Semantic Web“space”• Data Provider: exposes institutional open

archives of data and information– Development Agencies and NGOs– Research Institutions– Industry Information Centres

• Service Provider: provides services based on these institutional open archives– Libraries and other traditional Aggregators– Thematic or Regional Centres of Excellence– The data providers themselves

Gauri Salokhe


Challenge in all this..

Gauri Salokhe


Challenges related to ...• Priorities

• What do we need most?

• What can we achieve in measurable time?

• Impact • How to reach out effectively?

• killer application/s?

• Community• How can we improve collaboration?

• How can we exploit synergies and comparative advantages?

• Sustainability• How can we assure services for a given range of time?

• Who will give us the money?

Gauri Salokhe


www.fao.org/aims

Gauri Salokhe


The names behind the work at FAO •Stefano Anibaldi

•Andrew Bagdanov

•Claudio Baldassarre

•Caterina Caracciolo

•Marta Iglesias

•Gudrun Johannsen

•Stefka Kaloyanova

•Stephen Katz

•Johannes Keizer

•Soonho Kim

•Irene Onyancha

• Boris Lauser

• Yuan Oktafian

• Stefano Pesci

• Valeria Pesce

• Gauri Salokhe

• Margherita Sini

• Imma Subirats

• Virginie Viollier

• Maria Folch

• Kirsten Geist

• Kris Jelinek

•Ex colleagues: Frehiwot Fisseha, Anita Liang, Fynvola Le Hunte Ward, Jim Weinheimer, Kat Hagadorn

Gauri Salokhe


Partners

ARD Prasad, Aree Thunkijjanukij, Asanee Kawtrakul, Chandima Gunadasa, Chang Chun, Dagobert Soergel, Eero Mikkola, Enrica Porcari, Jai Haravu, Jayanta Chatterjee, Michael Hailu, TVP Prabhakar, Wang Zhong, Ze Li and many more..

Gauri Salokhe


Thank you!

[email protected]