usability testing foundation
TRANSCRIPT
Founded in 2010 with offices across the globe
More than 1b words subtitled in 200+ languages
35m+ active users, 8m+ registered users
18,000+ hours of premium video
What is Usability Testing?
ParticipantThink Aloud
Set of tasks Observation
Measuring:EfficiencyEffectivenessSatisfaction
Moderated In-Person
● Probe participant to delve deeper● Observe body language and
reaction● Keep participant on track● Able to clarify confusions
Moderated Remote
● Reach out to participants who are geographically dispersed
● Participants interact with the website comfortably at their home or own location
● Time saving
● Internet connection may be unstable● Know your tools well on yours and
participants’ end● No show rates are higher
Unmoderated Remote
● Participants interact with the website comfortably at their home or own location
● No need to schedule● Simultaneous testing ● Time saving
● Cannot probe or follow-up● No real-time support for participant● Participants are “pro users”
What are your business goals? What are your users’ top tasks?
What do users do most often on your site?
How to find Top Tasks?
Identify top search termsIdentify most visited pagesSpeak to Customer Service teamExamine competitor’s siteGet users to rank list of tasks
Writing Tasks into Task Scenarios
Task:“Search for a horror show from Japan”
Task Scenario:
Must represent realistic user goal
“It is the Halloween holiday and you invited a group of friends to your place. You plan to watch a horror show with them. Find a Japanese horror show.
Tips for writing good task scenariosTask is unambiguous and easy to understand.
There is a clear end point to determine if task is a success or failure
Avoid hidden clues
Not biased
Avoid stacked tasks
Make the task neutral
Don’t be funny or assume about what users will find funny
Your friend recommended you a new show that is still airing on TV. Find information
about the new show.
What’s wrong?
RecruitKnow your end users. Demographic/gender/age?What form of testing? (in-person/remote)External recruiting agency or recruit by yourself?
Screening users
Screen users’ behavior instead of their frequency of use (ex. “What things do you do online?” (list activities with frequency) instead of “Are you a frequent internet user?”)
Users who work in certain companies (competitors)
Users whose job are related to research
Other specific criteria, etc.
How many users should I test?
else 5 is enough(to capture usability problems)*iterative process*not for quantitative studies
As many as possible
Roles to play as a moderator1. Give instructions
2. Inform users to think aloud and remind them
3. Inform them that there’s no right or wrong answer
4. Reflect what you heard and check your understanding
6. Ask open-ended questions
5. Do not assist them in the tasks
7. Take notes, but don’t write all the time
8. Ask for consent: videotaping permission, non-disclosure and give participant rights
Types of activities1st impressions (What’s the first thought that comes to your mind? Can you tell me what does this page do for you?)
Exploratory (What would you expect to see if you click on this? What does this label mean? I noticed that you …., what’s going on your mind right now?)
Directive tasks ( ....Find the working hours of the restaurant for Sunday)
Types of activitiesWrapping up(On a scale of 1- 10, how would you rate the ease of use for this website? What should we do to make it 10 out of 10? Do you have any questions or feedback?)
Post-site questionnaire - System Usability Scale- Post-site word choice
Success RatesTASK SCENARIOS USER 1 USER 2 USER 3 USER 4 USER 5 Success
Rate
1. 1 0 1 1 1 80%
2. 1 0 0 0 0 20%
3. 1 1 1 0 0 60%
0= fail 1= success