implementing a taxonomy in a content management portal

Implementing a Taxonomy in a Content Management Portal

Content Week 2005Miami, Florida

Monday, January 31, 2005Workshop H

2:45pm – 4:45 pm

Marjorie M.K. HlavaAccess Innovations, Inc.

[email protected]

www.accessinn.com

Introductions

• Name• Project• Expectations for these two short hours• Please fill in the sign up sheet• Would you like

– 1. Copy of this presentation?– 2. Sample software?– 3. Other information?

Copyright © 2005 Access Innovations, Inc.

What will we talk about this afternoon?

• 1.Definitions• 2.Where taxonomy fits in the Information Circle• 3.Where to use a taxonomy• 4.Taxonomies for Communities of Practice• 5.Surrounding theories and applications• 6.How to build and maintain• 7.How is used in enterprise information

Thesaurus Master

Data Feed

MAIto add Metadata

Database Management System Add

Metadata using MAI

Search

Inverted File



1. Definitions


What is a taxonomy?

• A hierarchical thesaurus with authority terms applied at the final node

• A browse-able web interface• A Linnaean System• A browse- able list with the term instance at

the final leaf


Types of Taxonomies

• Naming and organizing things into groups that share similar characteristics

• 1. Flat – just a list• 2. Hierarchical

– Taxonomic view• 3. Faceted

– Sorted by a single charasteristic – Metadata - Dublin Core– COSATI -GILS

• 4. Thesaurus– Term records– Database backend– Easier to modify and maintain


Taxonomy in meta data

• Definition– Taxonomy is a thesaurus in its hierarchical view

with the authority files applied at the final nodes– It allows the browse-able front end to a portal– It provides keyword and name access to the

content in the portal


Taxonomy definition

• A taxonomy is a thesaurus in hierarchical view with authority file terms added at the final nodes

• Thesaurus• Authority file• Hierarchical form• Final nodes


Thesaurus

• Concepts • Methods• Procedures

• Cognitive approach• The knowledge capture piece• The topics or subjects


Authority file

• People• Places• Things

• The tangible approach• Concrete Entities


Hierarchical view

• Gives the Portal view• The view of all the preferred terms in

categorized order• An outline of the thesaurus


Final Nodes• The last position on the hierarchical tree

– Taxonomy• concept

– narrower terms» final node - people, place or thing term» document instance» Letter to George Wiesman Dec 12, 2003» Technical report number TR-1039» Museum artifact 1706 wodden wagon wheel


Term Records – the Database Part

• Associative terms– Related terms

• Equivalence terms– Preferred and non preferred– Use and used for– Synonyms

• Hierarchical terms– Broader narrower terms– Parent Child


Other term record fields

• Scope notes• Cross references• History• Term Status• Category• User defined


2. Where does a taxonomy fit in the information circle?


Information Circle - Overview

Taxonomy

User

Content

Output


Content

Taxonomy

User

Content

Output

•Web Pages•White Papers•Research Reports•Licensed Data Feeds•Intranet•Internal Reports•Lotus Notes files•Databases•Public Relations Documents/Press Releases•Market Research Reports•Customer Relationship Management (CRM)•HR Files•Accounting/Financial Records•Legal Documents•Patents•Museum artifacts


Taxonomy

User

Content

Output

Content – cont’d

HTML – Meta name / KeywordsDB – Field / Meta tag / ElementXML – Entity table for valid values

Content Creation:


Taxonomy

Taxonomy

User

Content

Output

Taxonomy is applied to new and existing content:

Meta Tags

Thesaurus TermsAuthority Terms

DateAuthor

Descriptionetc.

Rule Base Taxonomy


Taxonomy – cont’d

Taxonomy

User

Content

Output

Index data - Manually - Automatically

Suggest new candidate terms

Review


Output

Taxonomy

User

Content

Output

Searchable Data

- Internal Data - External Data


User

Taxonomy

User

Content

Output

Web Browsing/Searching

Database Browsing/Searching

Query Resolution


User – cont’d

Taxonomy

Output

User

Content

User Input - Suggested Candidate Terms - New Documents

Reports Based on User Search - Search Logs - Null Hits (These will also suggest new candidate terms)


New Content

Taxonomy

User

NewContent

Output

The cycle begins again


Information Circle - Overview

Taxonomy

User

Content

Output


3. Where to use a taxonomy

• Link the Taxonomy and Indexing • Always in sync with the industry• Keep up to date with terminology• Automatically index the old data• Filter newsfeeds• Search using the Taxonomy• File using the taxonomy• Spell check using the taxonomy• Link to translation system• Catalog using the taxonomy• Index a book


Thesaurus Master


Database Management

System - Add Metadata

using MAI

Search

Inverted File

AadvarkAlligator

AppleAdvantage

….Zebra

Record locatorAccessinn.com/12345/demofile/recid15

Database records

Each with many

elements

Portal Searching


Search

Inverted File

AadvarkAlligatorApple

Advantage….

Zebra

Record locatorAccessinn.com/12345/demofile/recid15

Database records

Each with many elements

Portal Searching

Many data bases can be reached


4. Taxonomies forCommunities of Practice


Taxonomies in a Community of Practice

• Nature of Communities of Practice (CoP)• Taxonomies in context• Value of taxonomies• Creating a taxonomy• Applying the taxonomy


Nature of CoPs

• Free flowing, loosely structured

• Simple, ad hoc categorization

• Active CoPs need organization

• Search tends to be hit-or-miss

Courtesy of Lillian Gassie, Naval Postgraduate School, Monterey, CA


Taxonomies in Context

A taxonomy aspires to be:• a correlation of the different functional, regional

and (possibly) national languages used by a community of practice

• a support mechanism for navigation• a support tool for search engines and knowledge

maps• an authority for tagging documents and other

information objects• a knowledge base in its own right

Reference: “Taxonomies: the vital tool of information architecture”, www.tfpl.com


Value of Taxonomies

• Improves organization & structure• Facilitates navigation• Facilitates knowledge discovery• Reduces effort• Saves time

“Taxonomies are better created by professional indexers or librarians than by domain experts.”



Naval Postgraduate School’s Homeland Security Taxonomy (1)


Naval Postgraduate School’s Homeland Security Taxonomy (2)


IBM Insight graphical view


Applying a Taxonomy (1)

Manually• Add terms into

meta data fields• Design

navigation & site indexes with taxonomy hierarchy


Incorporating Hierarchical Classification from a Taxonomy


Applying a Taxonomy (2)

System integration• Search & retrieval

systems• Auto-assignment

of metadata• Categorization

systems


Applying the Taxonomy to a Digital Library

Web portal

Locally held

documents

Public repositories

Commercial data sources

Agency data sources

INTERNET (public)

spiders

Meta-Search Tool

Filtered content

Search engineSearch engine

Search engine

Search engineSearch engine

Automated categorization

Library catalogs

Search engine



5. Surrounding theories and applications


Other Vocabulary types

• Uncontrolled lists• Classification System• Subject headings• Controlled vocabulary

– usually synonyms and spelling• Authority files• Thesaurus• Taxonomy


Uncontrolled list - define

• Add terms as they occur• No cross reference• Simple flat structure


Controlled term lists - defined

• State the preferred terms • Provide allowed term entry• Heavily cross referenced• Not generally hierarchical• Popular• Easy to create


Controlled term list - format

• Cars – use Automobiles

• Personal Computer – use Microcomputer


Classification vs Subject Headings

• Classification– single spot or placement– browse physical list– often a numbering system– clear hierarchy– no or few cross references


Classification vs Subject Headings

• Subject headings– generic search– hidden classification system– related terms and cross references in heavy use– Usually the inverted form

• cells, electric

– Alphabetic access


Authority systems - defined

• Lists of terms in the preferred format for use• Frequently have cross references• Widely available• Frequently coded lists• Brand names


Authority lists - examples

• ISO Country Name and Code– International Standards Organization

• ISO Language list• NAICS (SIC)

– Standard Industrial Classification Code (SIC)– Replaced by– North American Industrial Classification System

(NAICS)


What is a thesaurus?

• Jessica L. Milstead. All Rights Reserved• “For writers, it is a tool like Roget’s one with words grouped

and classified to help select the best word to convey a specific nuance of meaning.

• For indexers and searchers, it is an information storage and retrieval tool: a listing of words and phrases authorized for use in an indexing system, together with relationships, variants and synonyms, and aids to navigation through the thesaurus”

• www.jelem.com


Thesaurus - defined

• For information retrieval 1960’s– indexing either intellectual or automatic– in searching– searching but not indexing– indexing but not searching– hierarchical view for searching


Thesaurus - defined

• Monolingual - standard– British – English - ISO 5578– American – English –ANSI/NISO Z39.19

• Multilingual – standard ISO 5579– concept mapping– Eurovoc

• Discipline or Mission based - ad hoc


Thesaurus -standard format

• Main Entries• Top Terms - TT• Broader Terms - BT • Narrower Terms - NT• RELATED TERMS - RT• Scope Notes - SN• History - HI• Date term added/changed - DA


Standards

• Monolingual– NISO / ANSI – Z39.19– ISO 5578

• Multilingual– ISO 5579


ISO Standards

• Set up already - easy to adopt• Multiple broader terms• The standards outline procedures

– ISO -better for implementation– NISO much better reading


Why do we index ?

• Improve precision– define scope of terms

• Improve recall– different terms for same concept

• Guide to a field of expertise• Learning tool• Richer expression


Uses ?

• Indexing*– …process by which subject terms or classification symbols

are assigned to concepts in documents

– A thesaurus is also known as an indexing language

– * not the building of the inverted file in computer sense of indexing


What are we controlling ?

• Synonyms– different terms same concept

• Polysemes or Homonyms– same word different meanings– Lead– Reading


How ?

• Meaning– delineation of scope of a term

• Term equivalence– linking of synonyms

• Disambiguation of homonyms– lead (metal)– lead (element)– lead (management)


Precision options

• Language specificity• Coordination• Compound terms - level of

precoordination• Homographs and scope notes• Word distance indication


Precision options

• Structural relationships• Links and roles• Treatment and aspect codes• Weighting


Disambiguation

Bill Invoice

Bill Legislative

Bill Sport

Bill Person


Disambiguation

Bills Invoices

Bills Legislation

Bill Animal

Bill Person

PT

NT BT

RTRT

BTNT


6. How to build and maintain a taxonomy


How to build a taxonomy

• Collect the terms• Pull out authority terms• Organize into arrays• Choose top terms• Organize hierarchically• Flesh out term records• Test, review, and edit


Or said another way …

• Define scope• Collect terms and relationships• Identify existing taxonomies• Identify resources• Create & refine taxonomy• Apply taxonomy• Review and update


Maintain

• Steady stream of terms– Web logs – Null sets– New announcements– Indexing team– Library– Records managers– Etc.

• Candidate terms• Out of date is nearly useless


Best Results Measures

• Accuracy• Productivity• Hits, Misses and Noise• Precision (Recall)• Relevance• Ease of set up• Time to production


Integration

• Thesaurus– full featured– multiple views– multiple versions– multiple languages

• Automatic indexing– filtering– assisted

• Data Harmony MAI and Thesaurus Master


Visual Taxonomy

• Ways to look– Hierarchical– Alphabetic – by term– Ring diagrams– Topic maps– Related terms

Visual Taxonomy


API to Many Systems for CMS


Apply to the meta data

• Automatic application?• Spider setting internally• External web crawls – use all aliases• Filter data• Enhance search experience


Meta data

• The fields• The elements

– Class codes– Title– Author– Plaintiff– Product– subject / topic

• Meta Name Keywords in HTML


7. How Taxonomies are used in Enterprise Information


Brand is repeated in several spots and tied to search as well

Another way of listing brands

Category list from taxonomy is tied to brand

list and product list

Category code from the taxonomy is tied to the brand

list and the product list


Enterprise Taxonomy Management• Consistent application across entire site• Synonyms are used interchangeably• User doesn’t need to know the taxonomy• Pop up view is helpful• Site map for construction and browsing• Allows hidden sections for internal use


Taxonomies

• Form the basis for knowledge sharing• Add value to discussion• Allow deeper retrieval • Are straightforward to create• Require on-going maintenance


Your Taxonomy

• There is too much information to pile it on the floor.

• It fits in many places in the information flow

Data Feed

Thesaurus Master

MAIto add Metadata

Database Management System Add

Metadata using MAI

Search

Inverted File



Thank you for your time!Questions?

Marjorie M.K. Hlava

Access Innovations, Inc.

505-998-0800

[email protected]

www.accessinn.com

implementing a taxonomy in a content management portal

Technology

hlava access innovations

taxonomy definition

base taxonomy

taxonomy fits

existing content

meta data definition

preferred terms

hierarchical thesaurus