no advanced user left behind: discovery for all
TRANSCRIPT
![Page 1: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/1.jpg)
No Advanced User Left Behind - Discovery for All
Beth Namei & Caroline Muglia University of Southern California Libraries
ProQuest User Summit | Chicago, IL | October 8, 2015
![Page 2: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/2.jpg)
What we’ll cover today
● Timeline of USC’s implementation of Summon● Portrait(s) of Advanced Users● Results & Snapshots of 3 user studies and search query
analysis conducted from 2010-2015● Pain Points● Recommendations● Q&A
![Page 3: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/3.jpg)
Why USC launched Summon
● Better discoverability
● Relevant results to our users
● Richer user experience with the library’s website
![Page 4: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/4.jpg)
Novice researchers are happy with Summon!Freshman: “For my research papers, I always come to the USC website and Quick Search is really helpful…”Freshman:“The first thing I would do is probably Quick Search it… that’s my first step, that’s the easiest, it’s right there.”Freshman:“I use Google Scholar, but I find that it’s easier to use Quick Search because Google Scholar doesn’t link you to USC LIbraries all the time.”
Source: https://docs.google.com/document/d/1ajpryZbNmmFFceimG1yHgBnO5F6Icl5lr9eXb0d29SE/edit?ts=5612e99d
![Page 5: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/5.jpg)
Why discovery vendors and academic librarians (should) care about advanced users?
![Page 6: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/6.jpg)
Who are advanced users?They have…
● Specialized subject knowledge ● Familiarity with library's collections and resources● Experience researching in their discipline● Commitment to in-depth research● Sophisticated searching skills● Mastered and adapted to less than ideal search tools/systems
They want…● Precision search tools that allow them to hone in on sources● To search for known items using objective info. (author name, projects, article, book or journal titles, etc.)
Sources: “Not in Love, or Not in the Know? Graduate Student and Faculty Use (and Non-Use) of E-Books” - doi:10.1016/j.acalib.2012.08.005; http://ejournals.library.ualberta.ca.libproxy2.usc.edu/index.php/EBLIP/article/view/1802/3366 and http://www.emeraldinsight.com.libproxy1.usc.edu/doi/pdfplus/10.1108/07378830710840509
![Page 7: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/7.jpg)
Advanced users are the majority
http://clipgid.com/professor-teaching-clipart.html http://www.supercollege.com/images/grad_shadows_white_2.jpg
http://img11.nnm.me/9/0/b/6/d/d096d47ae3b91b239668df781ea.jpg
![Page 8: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/8.jpg)
Advanced users are embedded Quality Control
Image from: http://qainsight.net/category/qa-comics/
![Page 9: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/9.jpg)
Advanced users influence novice users
Image from: http://blog.ocad.ca/wordpress/fcdc/files/2014/10/college-clip-art-9iRq8p4ie.gif
![Page 10: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/10.jpg)
Advanced users are decision makers
Image from: http://agilevietnam.com/2012/12/28/understanding-decision-making/
![Page 11: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/11.jpg)
Discovery-centric Library Homepages
2010-2015
![Page 12: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/12.jpg)
2010 - Pre-Summon homepage
![Page 13: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/13.jpg)
July 2010 - Summon is launched as “Quick Search” - becomes the default tab
![Page 14: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/14.jpg)
July 2012 - Summon-centric homepage (catalog tab is removed)
![Page 15: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/15.jpg)
From 2012-2015, the catalog was still a visible option, just less prominentvia a drop-down menu
an icon below the main Summon search box
at the same time we also launched SIRSI Enterprise - a next-generation catalog
![Page 16: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/16.jpg)
July 2015: Complete website redesign & rebranding of our Summon instance
![Page 17: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/17.jpg)
Catalog is less visible than ever before
![Page 18: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/18.jpg)
Catalog vs. Summon Usage @ USC (2008-2015)
![Page 19: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/19.jpg)
3 user studies (2013-2015)
![Page 20: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/20.jpg)
Usability testing of homepageFall 2013
● Interviewed 13 users○ 4 undergraduates (3 freshman, 1 senior)○ 4 faculty○ 5 graduate students
● Accessed current homepage● Accessed mock-up of new homepage
![Page 21: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/21.jpg)
Trust
Image from: http://www.gratis-malvorlagen.de/science-fiction/wissenschaftler-2/
“I don’t trust Quick Search. I only use it if it is absolutely necessary….In order for this site to be successful, you have to have success after success after success because people...aren’t going to waste their time…I will admit to you that the odds are slim and none that Quick Search will get me anywhere close to what I want….If I need to search something I don’t go to any USC search engine….I go to Google where I can get things ten times as fast.”
Advanced user #1: Religion Professor
![Page 22: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/22.jpg)
Reliability
Advanced user #2: English Professor
“I could throw an author and title in [Summon] but I know that works very erratically….I often get inundated by irrelevant things…”
Image from: http://www.flickriver.com/photos/mitopencourseware/sets/72157621971051805/
![Page 23: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/23.jpg)
Annual InSync Library Survey Fall 2013
2,736 survey responses
● 2,256 undergraduate and graduate students
● 235 faculty
![Page 24: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/24.jpg)
“Journal articles are SO much easier to find using google if you already know the citation title.”
Need for Precision Tools
Advanced user #3: Cinematic Arts Graduate Student
“Your catalog seems to have problems returning accurate results. For example, searching the exact title of a book sometimes shows that you do not own the book, when you actually do have it.”
Advanced user #4: Law Professor
image from: http://famonthelam.com/wp-content/uploads/2015/02/dc8oKMgce.jpeg image from: http://blogs.bedfordstmartins.com/bits/files/2012/02/digi-liz.jpg
![Page 25: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/25.jpg)
Problems finding known items“Please, please, please fix the search function for books….The results are awful. I'll type in a book's title, and the book itself will be on page two after pages of reviews. Or I'll be looking for a journal article and it'll be impossible to find in a swarm of totally unrelated results.” Advanced user #5:
Undergraduate, Junior
“Make it easier to search for a book by author through the website. Searching by title is easy. It's difficult to search by author.”
Advanced user #6: Social Work Graduate Student
Image from: http://jeannelking.com/wp-content/uploads/2014/08/Closeup-of-good-enough-suspicious-facial-expression.jpg
Image from http://www.huffingtonpost.com/wait-but-why/generation-y-unhappy_b_3930620.html
![Page 26: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/26.jpg)
Search Log Analysis (2014)
http://bit.ly/Summon-relevance
![Page 27: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/27.jpg)
2014 ACRL Study of Known Item Search Queries in Summon Objectives: To compare the relevancy of results when searching for known items in Summon to Google and Google Scholar.
To better understand user behavior and experience when searching for known items via our main search option.
Sample Size: 384 known item queries were examined → in order to compare the relevancy across 3 search tools we reduced these to 278 (by eliminating non-scholarly items--videos, sound recordings, etc.--and items USC did not own)
![Page 28: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/28.jpg)
What kinds of known items are USC users searching for?
n= 384
![Page 29: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/29.jpg)
How are they searching for these known items?
n= 384
![Page 30: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/30.jpg)
Defining Relevant Results● Relevant = a match for the known
item shows up 1st or 2nd in the list of results.
● Partially Relevant = a match for the known item is found 3rd-10th in the list of results.
● Not relevant = known item is not listed in the first 10 results or no results were returned.
Image from: http://relevantinfo.org/wp-content/uploads/2013/09/good-o-meter.jpg
![Page 31: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/31.jpg)
Relevancy of Results for Scholarly Known Item Searches - March 2015
28%
n= 278
![Page 32: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/32.jpg)
Relevancy of Results for Scholarly Known Item Searches - June 2015
21%
n= 278
![Page 33: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/33.jpg)
in other words...
between 177,500 - 288,450 known item searches did
not lead to relevant results in 2014!
Image from: http://cache1.asset-cache.net/xc/538990835.jpg?v=2&c=IWSAsset&k=2&d=69V2Nb-aNPGQep7gMBCUQ2J-GKIQ445o7LyHHB91Fdj6ANhQcooFq0hMe_YiRJd10
![Page 34: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/34.jpg)
failed results = dead ends for users
Image from: https://gigaom.com/2011/03/28/opinion-preaching-to-the-green-choir-a-dead-end/
![Page 35: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/35.jpg)
Causes of “failed” searches After April 2015 Summon Release
n=57
![Page 36: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/36.jpg)
2 Pain Points (Dead Ends): Linking Problems & User Errors
n=57
![Page 37: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/37.jpg)
360Link 2.0We will be launching the new 360Link 2.0 Sidebar to provide more intuitive and visible help when users are faced with error messages or broken links.
Not a perfect solution though...
![Page 38: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/38.jpg)
360Link Sidebar Usability Study September/October 2015
Graduate student: When faced with an error message after clicking a Full Text Online link from Summon:
“When this happens I assume I did something wrong….I would do a Google search to make sure I have the correct article information”
Instead of reporting a problem when faced with these linking problems, she would request the item via interlibrary loan.
![Page 39: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/39.jpg)
User errors● 17% of all queries had 1-2
(47/278) user input errors○ 36% (17/47) of these led
to failed searches ● Types of errors:
○ Typos (transpositions, substitutions, omissions, insertions)
○ incorrect use of boolean Image from: http://human-error.sarkisozlerik.com/human-error/a-lifetime-by-design.html
![Page 40: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/40.jpg)
Help wanted!“Did you mean” suggestions only showed up 8 times (3% - 8/278) for all searches in our study. ● And only for 4 of the (47)
searches with user errors. ● And of these, only 2 of the
suggestions actually led to the known item.
Image from: http://www.debrafine.com/wp-content/uploads/2015/01/image004.gif
![Page 41: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/41.jpg)
Recommendations for Librarians● Decide whether or not to include ALL
databases/vendors in your discovery system○ Especially those that do not support direct linking or
have persistent linking problems○ Be a squeaky wheel! Make sure vendors know that
addressing these problems is a priority.
● Upgrade to 360Link 2.0 and do usability testing to make sure wording is actually helpful to ALL users
![Page 42: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/42.jpg)
Recommendations for ProQuest● Improve query
reformulation suggestions○ consider automatically
redirecting errors (like Google)?
● Continue to improve the relevancy of results for both known item and topic searches
![Page 43: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/43.jpg)
Other Recommendations for ProQuest● Accommodate copy and pasted and formatted citation queries
○ 4% (17/384) of the queries copied and pasted 3 or more citation elements (not necessarily something advanced users do but other studies have found this happening more and more)
● Offer more browsing options● Improve discovery for non-textual known item searches (videos,
sound recordings, etc.) ○ 78% (7/9) searches for videos USC owns did not find relevant
results (related to average # of words in film titles and minimal/poor metadata)
![Page 44: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/44.jpg)
“digital abundance [is] making it harder for [researchers] to feel confident that they had found everything they would need to find in order to reach and defend their research conclusions….Would it be possible...to develop a discovery tool that is designed not to find the best items but rather to provide some assurance that you hadn’t missed something?”
Keep imagining the future of discovery
Roger Schonfeld, June 1, 2015 The Scholarly Kitchen blog
![Page 45: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/45.jpg)
Questions? Comments?
Image from: http://www.opencolleges.edu.au/informed/features/student-driven-learning/
![Page 46: No Advanced User Left Behind: Discovery for All](https://reader034.vdocument.in/reader034/viewer/2022052705/589ed5dd1a28ab39498b4bed/html5/thumbnails/46.jpg)
Thank you!Caroline Muglia, Collection [email protected]
Beth Namei, Reference & Instruction Librarian [email protected]