how to conduct usability studies: a librarian primer
Post on 17-Oct-2014
591 views
DESCRIPTION
Slides used for RUSA webinar on September 4, 2013TRANSCRIPT
![Page 1: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/1.jpg)
How to Conduct Usability Studies A Librarian Primer
Tao Zhang, PhD Digital User Experience Specialist Purdue University Libraries [email protected]
Sponsored by ALA RUSA #RUSAUX
Ilana Barnes Business Information Specialist Purdue University Libraries [email protected]
![Page 2: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/2.jpg)
Learning Objectives
2
• Participants will be able to describe the usability concept and it's applications to libraries.
• Participants will be aware of the basic work!ows and techniques needed when doing usability tests.
• Participants will be aware of resources for usability studies and where to "nd out more.
![Page 3: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/3.jpg)
Agenda
3
• What is a usability study? • Usability study process • Conducting the study • Analyzing Results • Communicating • Where to "nd out more
![Page 4: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/4.jpg)
About Us
Ilana Barnes Tao Zhang
4
![Page 5: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/5.jpg)
Tell us about you!
What type of library do you work in? A) Public B) Academic C) Other
How familiar are you with Usability Studies/Tests? A) Very familiar B) Somewhat familiar C) Not familiar but excited to learn!
Chat box: Areas you especially want us to talk about
5
![Page 6: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/6.jpg)
What is User Experience?
6
![Page 7: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/7.jpg)
De"nition
Usability study refers to “evaluating a product or service by testing it with representative users. “
- http://www.usability.gov/how-to-and-tools/methods/usability-testing.html
7
![Page 8: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/8.jpg)
How UX "ts into Reference Services
• Decisions that user experience research can help you with: • What should our libraries front page look like? • What do our undergraduates expect from the library? • What would we like our undergraduates to expect from the library
that they do not know or understand or need?
• Understanding your users aims and needs help you help them and show your impact at the same time.
8
![Page 9: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/9.jpg)
How to plan your usability study topic
• What are my objectives? • How much time do I have? • How many teammates do I have? • Who am I presenting this to? • Is this for an existing product?
9
![Page 10: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/10.jpg)
When to Test
• Early or middle stages of development • Types:
Exploratory | Assessment | Comparison | Validation
• Start preparing at least three weeks before actual test
• DO pilot tests
10
![Page 11: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/11.jpg)
Usability Study Process
11
Recrui'ng Preparing Conduc'ng Analyzing Presen'ng
Choosing Features Creating Tasks Preparing Test Script
![Page 12: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/12.jpg)
Recruiting
• Determining the target audience (behavior) • Finding representative members
– Behavioral criteria, demographics – Avoid sampling bias
• Convincing participants to come – Incentives – Informed content (duration of study, data to collect)
• Scheduling – Reminding – Deal with no shows
12
![Page 13: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/13.jpg)
Recruiting
Best people to invite: • People who are going to need the service in the
near future • People who have used a competing/previous
version service in the recent past Number of participants: • 5-8 • Also consider representativeness and tasks
13
![Page 14: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/14.jpg)
Choosing Features
• Input from: Development team | Stakeholders | Users • Content inventory & task analysis
14
![Page 15: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/15.jpg)
Creating Tasks
Tasks should be: • Reasonable and doable • Described in terms of end goals • Speci"c Run through tasks to see how long each will take Determine what quantitative data to collect
15
![Page 16: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/16.jpg)
Discussion
You need to do a usability study of your library’s website, because users "nd it difficult to "nd information they want on the website. What are some possible tasks you can use in the usability study?
16
![Page 17: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/17.jpg)
Writing A Study Script
• Introduction (5 min) • Warm-up interview, demographics survey (5 min) • Scenarios and tasks (20-30 min) • Interview (10-15 min) • Wrap-up (5 min)
17
![Page 18: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/18.jpg)
Conducting the Study
The physical layout with observer
18
![Page 19: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/19.jpg)
Conducting the Study
The physical layout without observer
19
![Page 20: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/20.jpg)
Conducting the Study
TechSmith Morae www.techsmith.com/morae.html
20
Recorder
Observer #1
Participant Test Moderator
Observer
Manager
Observer #2 …
![Page 21: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/21.jpg)
Conducting the Study
21
http://silverbackapp.com/
![Page 22: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/22.jpg)
Conducting the Study
Testing on mobile devices
22
IPEVO Ziggi-HD High-Definition USB Document Camera
DIY Mobile Usability Testing - SXSW Interactive 2012 http://www.slideshare.net/beleniq/diy-mobile-usability-testing-sxsw-interactive-2012
![Page 23: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/23.jpg)
Moderation
• Follow script but allow some !exibility • Encourage participants to “think aloud” • Ask about expectations • Ask “why” a lot • Suggest solutions when participants are stuck • Discuss with observer right after test
23
![Page 24: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/24.jpg)
Two Basic Techniques
Observation Interview
24
![Page 25: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/25.jpg)
Observation
• Watch, Understand • Limit the time users have to perform a task • Encourage users to think aloud with neutral
prompts • Elicit detailed information
25
![Page 26: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/26.jpg)
Interview
• Introduction • Warm-up • General issues • Deep focus • Wrap-up
26
![Page 27: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/27.jpg)
Nondirected Interview
Prepare nondirected questions • Focused on the person answering it • Focused on experience, not extrapolation • Concentrate on immediate experience • Avoid judgmental language • Focus questions on a single topic • Keep questions open-ended • Avoid binary questions
27
![Page 28: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/28.jpg)
Nondirected Interview
Examples “Is this a useful feature?” “Is this feature valuable to the work you do right now?” “Do you think this feature would be better if it was also available on smart phones?”
28
![Page 29: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/29.jpg)
Nondirected Interview
Running the interview • De"ne (technical) terms • Don’t force opinions • Restate answers • Never say the participant is wrong • Work with a partner
29
![Page 30: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/30.jpg)
Analyzing Results
• Collecting observations – Notes from moderator and observer – Quantitative information: task successfulness,
navigational path, number of steps, errors, time to complete, subjective ratings (System Usability Scale)
– Qualitative feedback from participants
• Extracting trends – Repeated concerns – Multiple issues with possible common origin – Identify effects on user experience
30
![Page 31: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/31.jpg)
Analyzing Results
• A list of usability issues – Quantitative and qualitative data as evidence – Possible causes – Effects on user experience
• Assign importance/severity • Suggest solutions
31
![Page 32: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/32.jpg)
Communicating Findings
Usability test report • Project overview • Participants • Testing environment • Tasks • Procedure • Key "ndings
– Issues – Recommendations
32
![Page 33: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/33.jpg)
Communicating Findings
Presentation • Prepare the audience • Use supporting evidence (e.g., highlight videos) • Emphasize user perspectives • Use numbers carefully (because of small sample) • Leave enough time for questions
33
![Page 34: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/34.jpg)
Communicating Findings
Common problems: • “This is not statistically signi"cant.” • “This user is our typical user.” “This user does not
have relevant experience.” • “User X did Y; therefore, everyone must do Y.” • “More than half of users did not like this design;
therefore, we should change it.”
34
![Page 35: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/35.jpg)
What’s Essential?
• Develop a study plan • Find participants • Conduct the sessions • Debrief with observers and decide together • Present results
35
![Page 36: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/36.jpg)
Discussion
Think of an existing design that you want to evaluate for effectiveness • What are my objectives? • What types of resources do I have? • Who am I presenting this to?
36
![Page 37: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/37.jpg)
Other Types of Usability Studies
• Remote usability testing http://www.userzoom.com/
• Eye tracking
37
http://www.normalmodes.com/blog/2009/09/28/eye-tracking-heatmap-gallery-a-preview-discussion-of-ui-considerations/
![Page 38: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/38.jpg)
Where to Find Out More
• 6 Must-See Usability Testing Videos http://blog.usabilla.com/7-must-see-usability-testing-videos/
• Moderating Usability Tests Joe Dumas & Beth Loring (2008)
• A Field Guide to Usability Testing Smashing eBook https://shop.smashingmagazine.com/guide-to-usability-testing-1.html
• Handbook of Usability Testing Rubin & Chisnell (2008)
• Don’t Make me Think by Steve Krug Chapter: “Usability Testing on 10 cents a Day”’
38
![Page 39: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/39.jpg)
Learning Objectives
39
• Participants will be able to describe the usability concept and it's applications to libraries.
• Participants will be aware of the basic work!ows and techniques needed when doing usability tests.
• Participants will be aware of resources for usability studies and where to "nd out more.
![Page 40: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/40.jpg)
Agenda
40
• What is a usability study? • Usability study process • Conducting the study • Analyzing Results • Communicating • Where to "nd out more
![Page 41: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/41.jpg)
Works Cited
• 6 Must-See Usability Testing Videos - The Usabilla Blog. (n.d.). Retrieved from http://blog.usabilla.com/7-must-see-usability-testing-videos/
• Affairs, A. S. for P. (2013). Usability Testing. Retrieved from http://www.usability.gov/how-to-and-tools/methods/usability-testing.html
• Clear, Consistent, Capable, NICE» Blog Archive » Eye Tracking Heatmap Gallery: A Preview & Discussion of UI Considerations. (n.d.). Retrieved from http://www.normalmodes.com/blog/2009/09/28/eye-tracking-heatmap-gallery-a-preview-discussion-of-ui-considerations/
• Dumas. Joe, & Loring, B. (2008). Moderating Usability Tests. • DIY Mobile Usability Testing - SXSW Interactive 2012. (n.d.). Retrieved from
http://www.slideshare.net/beleniq/diy-mobile-usability-testing-sxsw-interactive-2012 • Morae usability testing software from TechSmith. (n.d.). Retrieved from
http://www.techsmith.com/morae.html • Krug, S. (2005). Don’t Make Me Think. • Rubin, J., & Chisnell, D. (2008). Handbook of Usability Testing. • UserZoom | zooming in on the user experience. (n.d.). Retrieved from http://www.userzoom.com/ • Silverback — guerrilla usability testing. (n.d.). Retrieved from http://silverbackapp.com/
41
![Page 42: How to Conduct Usability Studies: A Librarian Primer](https://reader034.vdocument.in/reader034/viewer/2022052303/5441c786b1af9fef4b8b47ec/html5/thumbnails/42.jpg)
Final Notes
Evaluation of this webinar Questions? Tao Zhang [email protected] Ilana Barnes [email protected]
42