Owning The Discovery Experience For Your Patrons
Charleston Conference, 2014
Robert McDonald, Courtney Greene McDonald,Esther Onega & Kate Lawrence
Welcome
Robert McDonald
What We’ll Share
Kate Lawrence, Vice President of User Research, EBSCO Information Services
The Google Influence on Discovery
Esther Onega, Head, Charles L. Brown Science & Engineering Library, The University of Virginia
Discovery Tools, A Case Study: The University of Virginia
Courtney Greene McDonald, Head Discovery & Research Services, Indiana University Libraries
“Everybody, Everything”: Crafting a Systemwide Discovery Strategy
Kate Lawrence, Vice President of User Research, EBSCO Information Services
The Google Influence on Discovery
@bykatelawrence
User Research at EBSCOMore than just usability testing
DataWhat is the story
behind the data?
Secondary ResearchWhat questions have been asked and
answered previously?
Primary Research
Matching the right method to
the research topic
College Students StudyDigging deep to understanding their digital lives
154 3
22 Students:
High School College Graduate
School
Schools included MIT, UNLV, Rice, Georgetown, UMass Amherst, GWU, UCSF, and more
“It’s their oxygen.”Google
The first half of the first page of results is critical.
Page 1 matters most“If it doesn’t exist on Google, does it exist?”
I trust itGoogle is their point of triage, how they discover the world.
“My door to the world.”
Wikipedia is often the first result.
Go to Images, News etc from this page.
Students are more likely to search again than to look at page 2+ of results.
WikipediaBecause college students like to start with an overview
1
2
3
The overview in “layman’s language”
The table of contents – “preview”
The references and external links at the bottom
Skimming and scanning
Skimming and scanning
What gets students interested in researchMotivation
“When it’s about me, it’s just more
interesting and worthwhile.”
Friends
Peers are the most significant
influencer among college students.
Future
Will this help me pay off my debt?
Get a job? Not have to move home with my parents after graduation?
Subject
“I fell in love with computational fluid dynamics. Google wasn’t
good enough any longer.”
RelevanceMentor
“I want my advisor to be proud of me.”
Students “make the turn” towards informavore
Motivation
The Library WebsiteGetting there via Google
“I don’t speak library-ese.”
Core
colle
ction
Abst
ract
s
Subj
ect I
ndex
Onl
ine
Data
base
sAr
chiv
es
Bool
ean
A simplified, elegant library website
When Discovery Means Find
“Everybody, Everything”: Crafting a Systemwide Discovery StrategyCourtney Greene McDonald, Head Discovery & Research Services, Indiana University Libraries
@xocg
One University, Many Front Doors
Systemwide Discovery
IUCATShared online catalogPowered by Blacklight
OnCourseShared learning management systemCurrently SakaiMoving to Canvas in 2016
Indiana University Libraries System
Diverse User Base, Divergent User PathsGoal: Consistency regardless of chosen entry point
Defining opportunities at the campus level
Discovery at Bloomington
Leverage EDS API to pull results into website, a crucial discovery platform
EDS IntegrationResponsivePlanned study of new library website, specifically search results
User InputIncreasing proportion of mobile users – designing for that need
Balancing consistency with choice
Discovery Systemwide
Leverage API to pull results into campus views; EDS Reading List for CMS
EDS IntegrationFlexibilityPlanned study of catalog interface
User InputCampus views: consistency with options; open source community collaboration
Esther Onega, The University of Virginia
Discovery Case Study: The University of Virginia
It all started with an email
Why Discovery?TO: [lb-pubserv] Primooooooo!
On Jun 5, 2013, at 11:26, "Burks, Todd (tcb2e)" <[email protected]> wrote:
Does anybody really like Primo? To be perfectly frank, I don’t. I don’t show it to First-Year students. In fact, I actively discourage them from using it. They’re set up for failure if they use it, and I need them to succeed in order to show the value of our services. Primo doesn’t add any value to the sessions I teach. What about you? Todd P.S. I’d like to have the default VIRGO search NOT include the Primo column on the right. It makes my job more difficult to steer new students away from it. If there are users that use it all the time, they should be able to set that preference without us having to do it for them.
TO: [lb-pubserv] Primooooooo!
On Jun 5, 2013, at 11:26, "Burks, Todd (tcb2e)" <[email protected]> wrote:
Does anybody really like Primo? To be perfectly frank, I don’t. I don’t show it to First-Year students. In fact, I actively discourage them from using it. They’re set up for failure if they use it, and I need them to succeed in order to show the value of our services. Primo doesn’t add any value to the sessions I teach. What about you? Todd P.S. I’d like to have the default VIRGO search NOT include the Primo column on the right. It makes my job more difficult to steer new students away from it. If there are users that use it all the time, they should be able to set that preference without us having to do it for them.
How we got hereFirst, the meeting
Identified problems
Brainstormoptions
Analysis
StartedTask force
Process for Evaluation• Reinitiated ExLibris (Primo) customer service
• Limited results to peer-reviewed
• Included abstract metadata in search results
• Contacted peer institutions
• Arranged open vendor presentations of top WSDS products & established evaluation criteria:• EBSCO (EDS)• ProQuest (Summon)• OCLC (WorldCat Local)
• Analyzed link performance
• User testing
Evaluation CriteriaHow we decide
7 Essential Elements
FunctionalitySpeed, relevancy
APINecessary for incorporation into our
catalog, VIRGO
MetadataSearchable fields, field consistency,
faceting
Quality ControlResults reflect library holdings
CoverageCompatibility of Library holdings with
those of the discovery service
Customer ServiceMUST be available, helpful, and
responsive
RelevancyAre librarians and users finding results
to be useful ?
Evaluation CriteriaHow we decide
API
• Each service has a web API:• The service is engaged by opening a URL containing the
specifics of a given request• The output from the URL is used to construct the contents
of the requesting page• A separate URL query and response is required for:• Each page of search results• Each individual article details page
• Each service has its own unique URLs for sending requests• Each service has its own unique XML reply format for receiving
results.
Comparing Metadata
Metadata
Ferreiro, Alberto. "Simon Magus, Nicolas Of Antioch, And Muhammad."
Church History 72.1 (2003): 53-70.
Primo•No abstract•No subject headings•Incomplete author name•Junk fields
There is a duplicate record for this article further down in the results did have full name and abstract, but due to an incorrect title could not connect through the link resolver.
Comparing Metadata
Metadata
Ferreiro, Alberto. "Simon Magus, Nicolas Of Antioch, And Muhammad."
Church History 72.1 (2003): 53-70.
EBSCO•Abstract•4 subject headings•No junk fields
No duplicate records and the link resolver connected properly.
Comparing Metadata
Metadata
Ferreiro, Alberto. "Simon Magus, Nicolas Of Antioch, And Muhammad."
Church History 72.1 (2003): 53-70.
Summon•Abstract•2 subject headings•More and better identifiers•No junk fields
No duplicate records and the link resolver connected properly.
Limiting Results to Our Holdings
• Results automatically limited by integration with Serials Solutions holdings.
• Updating Serials Solutions holdings automatically updates Summon results.
Yes Yes YesSummon EBSCO Primo
• Manually export and upload of single Serials Solutions holdings file.
• Updating Serials Solutions holdings requires updating EBSCO.
• Manually export, combine, and upload 27 separate Serials Solutions holdings files.
• Updating Serials Solutions holdings requires updating Primo.
Article Detail Page: Primo
Not part of the subjects data
Includes subtitle and title of magazine feature
Article Detail Page: EBSCO [EDS]
Refine?
Article Detail Page: Summon
LC-style subject headings broken into
separate terms (e.g. “Moose – Portrayals”
becomes “Moose” and “Portrayals”)
Library Staff Lunch & Test• 25 worksheets completed
• 14 votes received• Many cookies consumed• No clear winner, but 1 clear loser (Primo)
User Test Results
EBSCO EDS was favored overall,
but only by 3 searches
Qualitative Results – EBSCO [EDS]
Positives Negatives
• Quantity of articles• Facets• Relevancy
• Specificity• Non-English language
articles• Speed
“Seems to come up with much more specified results of these keywords, possibly quite useful for a researcher already well-into the research process and quite a bit familiar with the materials...but not sure how good it would be for a beginning researcher.
“One of the first things I noticed about the articles in these results was that the titles seemed more interesting and engaging, which made me more inclined to explore the article further. The search results included a good variety policy evaluation, study analysis, and general pieces on medical tort reform.”
Qualitative Results – Primo
Positives Negatives
• Diversity and range of articles• Journal titles, specificity, focus
• High quality• Speed
• Dated materials• Duplicates• Medium quality
“The first page of results covers good ground in terms of major works one would need to be familiar with when conducting research in this field, and displays a good diversity in terms of sources.”
“Using Firefox: Yielded 1,327 article results that were not exactly relevant to the paleo diet. The word 'paleo' [paleo diet] was included in some but the results were more on chemical reactions and such and i was looking more for experiments and peer reviewed journals covering the topic. It was not really relevant to the paleo diet at all.”
Qualitative Results – Summon
Positives Negatives
• Perceived selectivity• International diversity• Cross-disciplinary
• Recency• Non-English language
materials• Questionable faceting
“I liked the Summon results a lot. The first three results were directly relevant, and most on the first page of results were at least somewhat relevant. There were also about 600 results (as opposed to about 1200 in Existing Virgo and Ebsco) so it seems that Summon weeded out some of the not relevant results.”
“First article was from 1998, which is too old to be still relevant. Do not know why this is at the top of the articles list. Quality of journals is slightly suspect, although less so than EBSCO.”
Products, Prices & Contracts
ExLibris (Primo)$ • Would insist on annual contract
• Set-up fees paid 3 years ago
EBSCO [EDS]$ • 6-month free trial included
• 1-year contract• No set-up fee• Comes with full EDS
ProQuest [Summon]$$ • 1-, 2-, 3-year contract options
• Significant set-up fee
Questions
Thank you