indexing & retrieval. approaches to indexing key word indexing concept indexing social indexing...

86
Indexing & Indexing & retrieval retrieval

Upload: beatrice-turner

Post on 01-Jan-2016

267 views

Category:

Documents


8 download

TRANSCRIPT

Page 1: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing & retrievalIndexing & retrieval

Page 2: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Approaches to indexing

Key word indexing

Concept indexing

Social indexing

Non-text indexing

Page 3: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Keyword Indexing

Page 4: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Keyword indexing (1)

• QuickAdvantages:Entity-oriented - draw terms from entity itself

How to

succeed in

graduate

school

Page 5: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Keyword indexing (1)

• QuickAdvantages:Entity-oriented - draw terms from entity itself

• Inexpensive• No vocabulary lag• Multiple access points• Accuracy• No intellectual effort needed

Page 6: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Keyword indexing (2)

• No control over synonyms, near synonyms

Disadvantages:

• No control over homographs

Page 7: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Keyword indexing (3)

• Dependent on authors for informative and accurate titles

Disadvantages:

Artificial metalloenzymes based on the biotin−avidin technology: enantioselective catalysis and beyond

The golden peaches of Samarkhand

Page 8: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Keyword indexing (4)

• No control over word forms

Disadvantages:

Communicating in the library

or

Communications in libraries

Page 9: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Keyword indexing (5)

• No cross reference structureDisadvantages:

Page 10: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Historical key word indexing methodologies

Uniterm cards

Edge-notched cards

Optical coincidence cards

Key word in context (KWIC)

Spatial indexing

Page 11: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Pre- versus post-coordinate indexingMortimer TaubeChina—FolkloreChina—HistoryChina —PoliticsFrance —FolkloreFrance —HistoryFrance —PoliticsGermany —FolkloreGermany —HistoryGermany —PoliticsRussia —FolkloreRussia —HistoryRussia —Politics(12 terms)

China, France, Germany, Russia, Folklore, History, Politics(7 terms)

Page 12: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Post-coordinate index searchingHistory of France → France + History

Two sets of documents

Boolean AND search yields intersection of the two sets

France History

France AND History

Page 13: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Advantages to Taube's systemNo need to develop a list of authorized terms—pulling terms from documents themselves

No need to articulate rules of punctuation for representing complex concepts (France—History)

No need to delineate citation order (France—history v. History—France)

No need to formulate rules for subheadings ("May subdivide geog.")

Page 14: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Uniterm cardsOne card per term

Document no. 102"Arrest statistics of the Arizona State Police"

state31 102 53 24 75 96 107 68 49 70

34 95 117 59 115 147 109

police11 102 23 85 96 87 68 49 6091 115 107 79

Page 15: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Searching with uniterm cardsQuery: looking for documents about state police

102 Arrest statistics of the Arizona State Police.

state31 102 53 24 75 96 107 68 49 70

34 95 117 59 115 147 109

police11 102 23 85 96 87 68 49 6091 115 107 79

107 A short history of the Wisconsin State Police.115 The modern police state.

Page 16: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Edge-notched cardsOne card per bibliographic item

bearsWhirdeaux, ImaCaring for your pet pterodactyl / by Ima Whirdeaux

Call no. Q54321 .W45Turner, PaigeCaring for your pet grizzly / by Paige Turner

Call no. Q12345 .T8

pet-care

pterodactyls

Page 17: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Pyramid coding for edge-notched cardsCoding the year 1947*20 dots

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9

*They hadn't heard of the Y2K problem yet.

10 dots

9 5 2 0 9 5 2 0 8 4 1 8 4 1 7 3 7 3 6 6

Page 18: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

0 1 2 3 4 5 6 7 8 910 11 12 13 14 15 16 17 18 1920 21 22 23 24 25 26 27 28 2930 31 32 33 34 35 36 37 38 3940 41 42 43 44 45 46 47 48 4950 51 52 53 54 55 56 57 58 5960 61 62 63 64 65 66 67 68 6970 71 72 73 74 75 76 77 78 7980 81 82 83 84 85 86 87 88 8990 91 92 93 94 95 96 97 98 99

Optical coincidence cardsPre-printed cards with numbers for entire database

fleas

Page 19: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Key Word in Context (KWIC) IndexDoc 15 title: "A comparison of OCLC and WLN

hit rates for monographs and an analysis of the types of records retrieved"

CONTEXTttems of remote users: anhit rates for monograph/A

comparison of OCLC and WLNOCLC and WLN hit rates for

onographs/ A comparison ofarison of OCLC and WLN hit

n analysis of the types of s of the types of records

phs and an analysis of theA comparison of OCLC and

KEY WORDSanalysis of the types ofcomparison of OCLC and WLNhit rates for monographs and /monographs and an analysi/OCLC and WLN hit rates forrates for monographs and /records retrieved. A com/retrieved. A comparison /types of records retrieve/WLN hit rates for monogra/

POINTER15151515151515151515

Stopword

Stopword

Page 20: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Key Word Out of Context (KWOC) Indexaardvark 101baggage 123banyan 128, 159, 179coconut 955, 654driving 196, 488, 788elementary 455, 785elephant 128, 465, 783garage 678, 398hardware 849, 483, 399meter 768nadir 877

noxious 112opium 289opus 985, 159,

849people 629, 458quark 137, 492radar 968, 295radio 430, 206,

749stereo 294, 837,

873television 745, 727,

883ultraviolet 958, 774zebra 276

Page 21: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Vector space model (VSM)

Each document represented by a vector

tech

no

log

y

libraries

assi

stiv

e

Vector for document entitled "Assistive technology for libraries"

Page 22: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Vector space model matchingSimilarity between query and document vectors

tech

no

log

y

libraries

assi

stiv

e

Vector for document 2

Vector for query

Vector for document 1

Page 23: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

VSM term weightingAssign high weights to terms that appear frequently in the document but infrequently in the database

Query: "I'm looking for articles about assistive technology for the blind."

Termconclusioninformationblind

Freq. w/indocumentlowhighhigh

No. ofdocumentswith termhighhighlow

Page 24: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

VSM refinementsAdding semantic and syntactical parsing.

Bill is going to the store to make a purchase.

Bill is going to purchase the store.

Bill is going to store his purchase.

Page 25: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Concept indexing

Page 26: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Concept indexingRather than pulling terms from documents, assign concept identifier (e.g. France—History) to documents dealing with history of France

Requires intellectual effort

Takes more time than key word indexing so less economicalAvoids problems of false coordination and synonymy through use of vocabulary control

Page 27: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Vocabulary control (1)

One indexing term or phrase to represent a concept

– Unidentified flying objects not flying saucers

– Point user to correct term with "use" reference

– Reduces number of searches needed to find items about a particular topic

Page 28: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Vocabulary control (2)

One form of a word to represent the concept

– Dictionaries not dictionary

Page 29: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Vocabulary control (3)

One usage of a homographic term

– Fault (geologic) not fault (responsibility for error)

– Usage identified though scope note– Consistency among indexers as well

as one indexer over time– Helps user to avoid false drops

Page 30: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Vocabulary control (4)

Syndetic structure– Broader terms– Narrower terms– Related terms (see also)– User can negotiate structure to find

most appropriate term, as well as identify additional related terms of potential use in finding relevant documents

Page 31: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Social network indexing

• Tags

• Tag clouds

• User-created tags providing access to library resources

Page 32: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

flickr

http://www.flickr.com/

Page 33: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Tags

Page 35: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Tags

Page 36: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Tags

Page 37: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Tags

(177,583 photos)

Page 38: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Tags

Page 39: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Tag clouds

Page 40: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Geotagging

Page 41: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Librarian tagging

Page 42: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Library using flickr

Page 43: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Peace Palace Library (PPL)

Page 44: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Social bookmarking: http://www.delicious.com

Page 45: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

http://www.delicious.com/mauicclibrary

Page 46: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

http://www.delicious.com/mauicclibrary

The economic case for open access in academic publishing

technology

Portable software for USB drives

CU Researcher Finds 10,000-Year-Old Hunting Weapon in Melting Ice Patch

Page 47: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

University of Pennsylvaniahttp://www.library.upenn.edu/

Page 48: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

PennTags

Page 49: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Item list with PennTags

Page 50: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Adding a PennTag

Add to PennTags

Page 51: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Non-text indexing

Page 52: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing Music

Page 54: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing Music - melodic contour

* R U- / - / - \

R U R D

Page 55: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Query by humming

Page 56: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Query by humming (2)

Hummed Queries

Digital Audio

Melodic contour Ranked ListOf

Matching Melodies

Pitch Tracker

Query Engine

MIDI Songs

Melody Database

Source: Ghias, Asif; Logan, Jonathan; Chamberlin, David; and Brian C. Smith. 1995. Query by humming--musical Information retrieval in an audio database. ACM Multimedia 95 - Electronic Proceedings. http://www.cs.cornell.edu/Info/Faculty/bsmith/query-by-humming.html

Page 57: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing Music - melodic contour

* R U R U R D

http://www.musipedia.org/

Page 58: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing Music - melodic contour

* R U R U R D

http://www.musipedia.org/

RURURD

Page 59: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing Music - melodic contour

* R U R U R D

http://www.musipedia.org/

Page 60: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing images

Source: Trust Territory archives.

Page 61: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing images - chair (1)

Page 62: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing images - ?

Page 63: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing images - chair (2)

Page 64: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Biometrics - face

Page 65: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Biometrics - differences

Page 66: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Biometrics - similaritiesLook at ratios of distances between marker points

Page 68: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing images by color

http://www.hermitagemuseum.org/fcgi-bin/db2www/qbicColor.mac/qbic?selLang=English

Page 69: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing images by color

http://www.hermitagemuseum.org/fcgi-bin/db2www/qbicColor.mac/qbic?selLang=English

Page 70: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing images by color

http://www.hermitagemuseum.org/fcgi-bin/db2www/qbicColor.mac/qbic?selLang=English

Page 71: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing images by color

http://www.hermitagemuseum.org/fcgi-bin/db2www/qbicColor.mac/qbic?selLang=English

Page 72: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing images by color

http://www.hermitagemuseum.org/fcgi-bin/db2www/qbicColor.mac/qbic?selLang=English

Page 73: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing images by color

http://www.hermitagemuseum.org/fcgi-bin/db2www/qbicColor.mac/qbic?selLang=English

Page 74: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing images by color

http://www.hermitagemuseum.org/fcgi-bin/db2www/qbicColor.mac/qbic?selLang=English

Page 75: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing images by layout

http://www.hermitagemuseum.org/fcgi-bin/db2www/qbicLayout.mac/qbic?selLang=English

Page 76: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing images by layout

http://www.hermitagemuseum.org/fcgi-bin/db2www/qbicLayout.mac/qbic?selLang=English

Page 77: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing images by layout

http://www.hermitagemuseum.org/fcgi-bin/db2www/qbicLayout.mac/qbic?selLang=English

Page 78: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing images by layout

http://www.hermitagemuseum.org/fcgi-bin/db2www/qbicLayout.mac/qbic?selLang=English

Page 79: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing images by layout

http://www.hermitagemuseum.org/fcgi-bin/db2www/qbicLayout.mac/qbic?selLang=English

Page 80: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing images by layout

http://www.hermitagemuseum.org/fcgi-bin/db2www/qbicLayout.mac/qbic?selLang=English

Page 81: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing images by shape

http://shape.cs.princeton.edu/search.html

Page 82: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing images by shape

http://shape.cs.princeton.edu/search.html

Page 83: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing images by shape

http://shape.cs.princeton.edu/search.html

Page 84: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Indexing images by shape

http://shape.cs.princeton.edu/search.html

Original

Page 85: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Search by Shape – Commercial Usage

http://www.youtube.com/watch?v=grShwnDXyUA

Page 86: Indexing & retrieval. Approaches to indexing Key word indexing Concept indexing Social indexing Non-text indexing

Search by Color Exercise

http://www.hermitagemuseum.org/fcgi-bin/db2www/qbicSearch.mac/qbic?selLang=English

Title?

Artist?

Title?

Artist?

1 2

34 5