articles+ usability testing

21
Articles+ Usability Testing Kathleen Bauer, Manager of User Experience Group in Yale Library IT

Upload: yale-university

Post on 28-Nov-2014

63 views

Category:

Education


0 download

DESCRIPTION

Usability testing done on a web-scale discovery tool Summon, branded Search Articles+ at Yale University Library.

TRANSCRIPT

Page 1: Articles+ usability testing

Report on Articles+ Usability TestingKathleen Bauer, Manager of User Experience Group in Yale Library IT

Page 2: Articles+ usability testing

Worked in groups of three librarians.Recruited students at Bass, Thain Family Cafe, Arts, and CSSSIStudents were asked ● their year ● major/department● if they had used Articles+

User Experience in Yale Library IT

Methods

Page 3: Articles+ usability testing

Participants

Participants were ten graduate students, post docs and staff, and ten undergraduates, with majors/departments Early Modern European History, English (2), Astronomy(2), Environmental Science(2), Management, Geology and Geophysics, Physics (2), Political Science (2), History of Science, Music, Biology (2), Spanish, Literature and undeclared.

User Experience in Yale Library IT

Page 4: Articles+ usability testing

Prior Use of Articles+

The majority, 70%, of test participants had not used Articles+.

User Experience in Yale Library IT

Page 5: Articles+ usability testing

Question 1 Image Search

To start, I’d like you to imagine you are preparing a class presentation for which you need several very high-quality images of the Notre Dame Cathedral in Paris, and use Search Articles+ to find one or more images for that purpose. Do you see anything in the list of results you might use?

Success Rate = 85% (17) found images

User Experience in Yale Library IT

Page 6: Articles+ usability testing

Question 1 Image Search

Participants used facets for Content type and Discipline to find images, which display in a scroll bar.

User Experience in Yale Library IT

Page 7: Articles+ usability testing

Question 2 Topic Search

Have you recently looked for articles on a topic for a class assignment or research? What was that topic? Where did you look for articles? Participants searched a range of topics of their choice.

Please take a moment to look at the list of results. Do they look appropriate to your topic?

Success Rate = 75% (15) thought results looked appropriate or relevant.

User Experience in Yale Library IT

Page 8: Articles+ usability testing

Question 2 Limit to Scholarly Material

Let’s assume you need scholarly articles only for a class paper. Can you find a way to get rid of any popular material and news items?

Success Rate = 15% (3) used the Peer Review limit.

Page 9: Articles+ usability testing

Question 2 Limit to Scholarly Material

Participants were more likely to use the Content Type facet for Journals Articles than to use the Peer Review limit.

Page 10: Articles+ usability testing

Question 3 Find a Subject Guide

Participants were asked two different questions. The first version:

Please show me how you would find some basic, background information about the discipline of ethnomusicology. Take a minute to look at the display of results. Do you see anything on the page that you might consult if you needed to find resources at the Yale Library?

Success Rate = 18% (2 of 11) found a guide recommendation in the Topic Explorer.

User Experience in Yale Library IT

Page 11: Articles+ usability testing

Question 3 Find a Subject Guide

Most participants didn’t see the Topic Explorer, which disappeared when they exploded facet values.

User Experience in Yale Library IT

Page 12: Articles+ usability testing

Question 3 Find a Subject Guide

With facets expanded, the Topic Explorer no longer displays, so no librarian/ guide recommendation displays.

User Experience in Yale Library IT

Page 13: Articles+ usability testing

Question 3 Find a Subject Guide

This version of the question, which asked participants to find info on colonial Mexico, had a 66% (6 of 9) success rate.This guide was embedded in results, and was easier to find.

User Experience in Yale Library IT

Page 14: Articles+ usability testing

Would you use Articles+ again?

User Experience in Yale Library IT

Most participants indicated they would used Articles+ again.

Page 15: Articles+ usability testing

What was successful?

● Students naturally and easily used facets, especially content type and disciplines.

● Students liked that they could search for things they know the library owns.Results were perceived as “library approved” and that was a positive for Articles+ over Google.

● It was easy to get results, and most students were satisfied with the relevance of results.

● Articles+ appeals to undergraduates, and they voiced enthusiasm for using it.

● Students said they wished they had known about Articles+.

User Experience in Yale Library IT

Page 16: Articles+ usability testing

Recommendations for Serials SolutionsIssue: There was some trouble getting to local experts because of the far-right display pane issue. It was better when the librarian/Libguide displayed within results.

User Experience in Yale Library IT

● Switch suggested librarian with encyclopedia article abstract (in the Topic Explorer right pane)

● Address issues of the right pane disappearing (with loss of functionality) when the screen is too small or a facet is expanded.

Page 17: Articles+ usability testing

Recommendations for Serials SolutionsIssue: Participants did not notice that facets were in effect, causing problems with new searches.

User Experience in Yale Library IT

Facet breadcrumb example from Findit.

It should be very obvious when limits are in effect. Add facet breadcrumb, and/or notice that facets are in effect.

Page 18: Articles+ usability testing

Recommendations for Serials Solutions

● When a term such as images is in the search the row of images should be at the very top of search results, similar to the Google functionality.

User Experience in Yale Library IT

Page 19: Articles+ usability testing

Recommendations for the Library

● Students want to know about tools like Articles+. Reach out to them!

● Use instruction to increase knowledge of Articles+. ● Create a reference cheat sheet for librarians who will

present or teach about Articles+.

User Experience in Yale Library IT

Issue: Most students didn’t know about Articles+ and said they wished they’d known about it.

Page 20: Articles+ usability testing

Recommendations for the Library

● Make the link to Articles+ more prominent on the library’s home page and elsewhere. Articles+ search can be embedded in Libguides.

● Do an additional round of testing in the fall 2014. Record sessions so that others in the library can view them.

User Experience in Yale Library IT

Page 21: Articles+ usability testing

More Information

Report on Summon Usability Test http://libraryux.tumblr.com/post/86600055718/summon-2-0-testing-results

Remote Unmoderated Testhttps://yalesurvey.qualtrics.com/SE/?SID=SV_cGVskajotpE9IK9

Send questions to [email protected]