known-item searching in a discovery environment jan fransen university of minnesota – twin cities

52
Known-Item Searching in a Discovery Environment * Finding What You Know We Have Jan Fransen University of Minnesota – Twin Cities

Upload: christal-short

Post on 25-Dec-2015

213 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

Known-Item Searching in a Discovery Environment

*Finding What You Know We Have

Jan FransenUniversity of Minnesota – Twin Cities

Page 2: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

Jeff PetersonJan FransenJanet ArthAnne Beschnett / Amy ClausenJosh BishoffJayne BlodgettSunshine CarterMark Engebretson

David Faust / Malaika GrantStephen HearnChris KoehlerEric LarsonDarlene MorrisArvid NelsonStacie Traill

*MNCAT Discovery Implementation Group

Page 3: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*Why all the fuss?

Hessel, H., and J. Fransen. “Resource Discovery: Comparative Results on Two Catalog Interfaces.” Information Technology and Libraries 31, no. 2 (2012).

Satisfaction surveys done by the U of M Libraries Primo Management Group in late 2009

Page 4: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

Hessel, H., and J. Fransen. “Resource Discovery: Comparative Results on Two Catalog Interfaces.” Information Technology and Libraries 31, no. 2 (2012).

Satisfaction surveys done by the U of M Libraries Primo Management Group in late 2009

Page 5: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*But how often do they search for known

items (really)?

*55 percent of searches in University of Illinois at Urbana-Champaign’s log analysis

*44 percent of searches in University of Michigan’s study

Schlembach, M. C., Mischo, W. H., & Bishoff, J. The Use of Transaction Logs to Model User Searching Behaviours.

Chapman, S., Desai, S., Hagedorn, K., Varnum, K., Mishra, S., & Piacentine, J. (2013). Manually Classifying User Search Queries on an Academic Library Web Site. Journal of Web Librarianship, 7(4), 401-421.

Page 6: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*And then…

Page 7: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

Please, PLEASE make the default search for whatever replaces

MNCAT Classic be "Title begins" 95% of the searching I do is for known items that I know the title of. I hate having to go through multiple radio buttons and pull down

menus just to do a simple title begins search!

Better to provide specific pointers on how to

search most

efficiently in the new MNCAT environment

for a "known item."

I don't dislike change in general. I do dislike this

change intensely. Known item searching is foundational to

advanced research, and the replacement being provided

is unsatisfactory.

I think the UMN Libraries need to keep in mind that some of your

most frequent users know exactly what we need

to find (we have a specific title or author), but we have very

limited time for scrolling through lists of irrelevant results.

I find MNCat Classic has much more accurate results when I search for a particular title or

author. The new interface

returns too many results to be useful, or sometimes

none at all when I put the title in quotation marks.

Comments collected from 5/8/13 to 12/26/13Almost all comments were received in May and June.

Page 8: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*2/6/14 Library Staff Forum

Page 9: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*What is working well?

Page 10: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*So how did we get there?

Page 11: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*So how did we get there?

*We knew Primo’s known item search success rates had improved (anecdotally)*We needed to prove it

*We knew the success rate wasn’t 100%*It wasn’t in MNCAT Classic, either

*We needed people to know we were willing to work with them if they could give us examples

Page 12: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*Proof

*MNCAT Classic vs. Fall 2013 MNCAT *(That’s Primo without Central)

*Sample 1: 414 strings taken from actual Feb 2012 MNCAT Classic Title Browse searches*37 journal searches, 377 all-Aleph searches

*Searched exactly as typed

*MNCAT Classic: Basic screen, browse on title

*MNCAT (Primo 3): Keyword search

Page 13: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*What is success?

*Considered found if result was in the first 10

*If searching MNCAT and Did You Mean gave correct suggestion, considered found

*Intent isn’t always obvious—there are some judgment calls

Page 14: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*Results

Both worked equally well 274 66%

MNCAT Classic found on first screen, MNCAT didn’t 35 8%

MNCAT found on first screen, MNCAT Classic didn’t 7 2%

Neither found on first screen 98 24%

Page 15: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*Where did Classic shine?

*Partial words at the end of a long phrase (55%)

*Subtitle or too much information included in search (17%)

*Incorrect titles where the problem is AFTER the first words (7%)

*Series titles or incorrect abbreviations (7%)

Journal of hospitality and tourism technology

(Should be Journal of hospitality and tourism research)

Annals of the American Academy of Political and Social Scien

phonetics and phonology 3

Page 16: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities
Page 17: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities
Page 18: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*Where did MNCAT (Primo) shine?

*Article included

*Misspelling at the beginning of the phrase

europeen journal of archeology

A Cross of Iron: Harry S. Truman and the Origins of the National Security State, 1945-1954

The Clinical journal of pain

The Harvard classics

The lost Gospel of Judas Iscariot : a new look at betrayer and betrayed

Page 19: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*Where else?

*Didn’t look for cases where the MNCAT Classic Browse returned a matching result at the top, but clicking that item led you to many records

*With MNCAT you can facet, so it would be better for that type of search

*ExamplesHamletCriticismneurologyNew York  Times

Page 20: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*Tips for MNCAT Success

*Get the title right

*Spell it right

*Don’t put in more than you need

*If you don’t see what you expect, check for Did You Mean

Page 21: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*And then we told people

*Constant drum beat*Just try it…

*And then tell us when you find something that doesn’t work

*“We can’t fix it if we don’t know it’s broken.”

*YouTube Video!*Focus on library staff, but available to

anyone

*http://z.umn.edu/mncatbattle

Page 22: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*“But I ALWAYS spell it right! And still…”

*Sample 2: Random sample of MARC 245a fields from Aleph

*The most boring test you’ve ever seen

*So boring, we didn’t finish it

*If we own it and you spell it right, you’re almost always going to find it

*We did identify a few instances of odd behavior and were able to fix records or tweak normalization rules

Page 23: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*Adding Primo Central

*December 26*Aleph to Alma

*MNCAT + MNCAT Article Discovery = MNCAT Discovery

*Oh yeah, and moved to Primo 4.4.1

*Phase 2 testing*Same search set

*BUT corrected obvious spelling/typo

Page 24: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities
Page 25: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities
Page 26: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*Results

Material Type Found Owned, but not found Not owned

Journal 89% 5% 6%

Book 81% 0 19%

Conference proceeding 83% 0 17%

Page 27: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*Journals owned and not found

Search String Notesannals of the new york Articles overwhelm. 2nd hit if I choose Libraries Catalogj pharm sc European Journal of Pharmaceutical Sciences is #2, and that's the only

journal that shows up. Journal being sought is probably Journal of Pharmaceutical Science, and this abbreviation appears to be valid.

Journal of neuroimaging : official journal of the American Society of Neuroimaging.

First hit without the subtitle

Nw. U. L.. Rev. Northwestern University Law Review (works unabbreviated); Discovery picks up individual articles from same

Obstet Gynecol American Journal of Obstetrics and Gynecology, #2 (Valid abbreviation)Popular Science 1946 First hit without the yearSCI nursing : a publication of the American Association of Spinal Cord Injury N

Found with no subtitle

screen: the journal of the society for education in film

Found with no subtitle

Page 28: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*More About Journals

Page 29: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities
Page 30: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities
Page 31: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities
Page 32: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*So what’s the problem?

*MNCAT Discovery just doesn’t find journals by title reliably, or buries them under articles?

*People don’t realize that they can just type the name in the Big Search Box?

*People are systematically typing the names wrong?

*People miss the functionality of the E-Journals list?

Page 33: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*What did we do?

Page 34: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*What should that link point to?

*Find E-Journals (the A-Z list)

*Includes the word Journals!

*More tolerant of title variants than Title Starts With

*Better with abbreviations

*Libraries Catalog Browse

*A little more like the SFX A-Z list

*More forgiving of errors late in search string

*Both print and online journals (BUT everything else in the catalog, too)

*Currently sorting by Date-Newest

*Several known bugs

Page 35: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

Title MNCAT DiscoveryBrowse (Popularity

Sort)Find E-Journals

The New England journal of medicine 1 1 (without The) 1

Nature 1 1 1Science 1 4 1Pediatrics 1 1 1Journal of the American Chemical Society 1 1 1JAMA 4 1 1

Proceedings of the National Academy of Sciences 1 3

PLoS ONE 2 1 1Annals of Internal Medicine 1 1 1Harvard business review 1 1 1The New York times 6 1 5Neurology 1 2 1Lancet 1 1 1Health affairs 1 1 1Chest 1 1 1

Journal of personality and social psychology 1 2 1

Journal of biological chemistry 1 1 1

American journal of public health 1 1 1

Journal of the American Veterinary Medical 6 1 1

The American journal of psychiatry 1 1 1

Circulation 2 1 1Archives of internal medicine 1 1 1

Wall Street Journal 11 (print) or 2

(online)1

The American journal of clinical nutrition 1 1 (without The) 1BMJ. British medical journal 1 1 1

Page 36: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*Next Steps

*This summer

*We’ll renormalize, and move to Primo 4.6

*Test the same set again to make sure we gained and didn’t lose in the Known Item game

*Usability Lab

*Test discovery-to-delivery workflow

*Ongoing

*Respond to specific problem queries

*Analyze what people are searching

Page 37: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*Fun with Google Analytics

Page 38: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*What ARE they searching?

*How did they start their search?

*Google Analytics on Libraries home page

*January 21 – April 18, 2014

Page 39: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*Most frequent search string?

1. Harvard Business Review (174)

2. JSTOR (173)

3. Nature (164)

4. Science (151)

5. PubMed (134)

6. Oxford English Dictionary (114)

7. Journals (102)

8. Web of Science (93)

9. Scientific American (90)

10.Pennsylvania Literary Review (86)

Page 40: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*What about the first book?

66 searches

Page 41: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities
Page 42: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*Next book?50 searches#26 on the list

Page 43: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*Most unexpected top 20 result

rates of reaction of sodium borohydride with bicyclic ketones

78 times!

Page 44: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*Number of times users searched

for “hours”

*Hours (3)

*Spring break hours (1)

Page 45: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities
Page 46: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*Number of words in a search string

(average)

*Average = 4.87Minimum 01st Quartile 2Median 33rd Quartile 6Maximum 504

Page 47: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*504?!

\t Buy this book together with History of Syria, Including Lebanon and Palestine by Philip Hitti Origen’s construal of the Bible as a textual incarnation of the Word encourages an assimilationist interpretation of the Hebrew Scriptures as a proto-Christian gospel. Although in partial agreement with this thesis, this study suggests a non-assimilationist reading of Origen’s biblical exegesis. \t+\tA detailed history of Syria, Lebanon, and Palestine from the earliest times until the 20th century.\tSave $44.52 Total List Price: $296.82 Buy both books for only $252.29 Quantity: Customers who bought this book also bought: \tThe Jewish Neo-Aramaic Dialect of Urmi by Geoffrey Khan This volume presents a description of the Neo-Aramaic dialect that was spoken by the Jews of Urmi in north-western Iran but which is now virtually extinct. The material for the volume was gathered firsthand in fieldwork conducted with the last remaining speakers in Israel. The volume consists of a detailed grammatical description, a corpus of transcribed texts, including folktales, historical accounts and portrayals of customs, and an extensive glossary. Quantity: \tThe Archaeology of Cult in Middle Bronze Age Canaan by Jill Katz What was Canaanite religion like during the Middle Bronze Age, at the time of the of the biblical patriarchs? This volume presents a theoretical model for identifying ritual behavior in the archaeological record, providing a test case using the rich material culture and structures that have been unearthed at the biblical city of Gerar (Tel Haror, Israel). Quantity: \tDischronology and Dialogic in the Bible’s Primary Narrative by David Bergen This ground-breaking study offers a reassessment of Moses' book of the law from a narrative theory perspective. Concerned for the long-term viability of his people, Moses legislates a public reading of his document which is deposited next to the ark of the covenant as a national testament. Through the mechanics of narrative mediation, the narrator reveals to the reader of Deuteronomy the contents of Moses' enshrined publication. Deuteronomy's simulcast of Moses' book invites external readers to compare and evaluate their readings with story-world readers who access the same text within the Bible's Primary Narrative. Quantity: \tJacob of Sarug’s Homily on the Tower of Babel by Aaron Butts This edition of Mar Jacob of Sarug's (d. 521) homily on the Tower of Babel develops an extended word-play between “morduto” ‘rebellion’ and “marduto” ‘discipline.’ As is characteristic of Jacob, the characters, their personalities, and their motivations are developed far more than they are in the biblical narrative. The volume constitutes a fascicle of The Metrical Homilies of Mar Jacob of Sarug, which, when complete, will contain the original Syriac text of Jacob's surviving sermons, fully vocalized, alongside an annotated English translation. Quantity: \tInterpretation, Religion and Culture in Midrash and Beyond by Lieve Teugels The third issue of Proceedings of the Midrash session at the SBL Annual meeting published in this series. This volume contains papers on religion in midrash (2006) and modes of biblical interpretation in rabbinic, Syriac and Islamic traditions (2007). Quantity: \t previous | up | next The Spell of the Logos}

Page 48: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities
Page 49: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*How often do people search by

ISBN, ISSN, or another number?

*Not very often

*BUT that’s 550 searches

Page 50: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*How often do they use Boolean operators or

quotation marks?

*A little more often

Page 51: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*Just how many ways are there to search

for DSM 5?

Search String Search Count

DSM 5 29

DSM-5 23

dsm 22

dsm v 10

DSM-V 4

DSM - 5 3

Dsm5 2

Diagnostic and statistical manual of mental disorders : DSM-5 1

Diagnostic and Statistical Manual of Mental Disorders: Dsm-5 1

Page 52: Known-Item Searching in a Discovery Environment Jan Fransen University of Minnesota – Twin Cities

*Questions?

*Jan [email protected]