Usability Testing

Art Center Interactive Design 4 | Week 5 | Joy Liu

UsabilityQuality of a user's experience when interacting with products or systems

How to measure usability? Is your product user-friendly?

Effective Efficient Easy to Learn Error Tolerant Engaging

the completeness and accuracy with which users achieve their goalsEffective

Efficient Easy to Learn Error Tolerant Engaging

RaeVynn Sands, Flickr, http://www.flickr.com/photos/cronewynd/4689772454

Does the product accomplish what I need?

Can I accomplish all the major tasks?

Is the product missing anything essential to my goal?

Is the product unnecessarily complex? cumbersome to user?

How complete is the result?

What are the core elements of this product?

Is it effective?

the speed (with accuracy) with which this work can be done

Effective Efficient Easy to Learn Error Tolerant Engaging

Insert ANY video of parallel parking fail here

How deep is the system?

How many clicks does it take to complete _____?

Does the system give feedback?

Is this loading? Or did the screen freeze?

Where am I? Where am I going?

How can I make this simpler by activating GPS? gesture?

Is the system full of inconsistency?

Is it efficient?

how well the product supports both initial orientation and deeper learning

Effective Efficient Easy to Learn Error Tolerant Engaging

Martin Gillet, Flickr, https://www.flickr.com/photos/mgillet/6875700642 Francesco Crippa, Flickr, http://www.flickr.com/photos/fcrippa/7967670670/

How easy is it to complete the tasks?

Can most people learn to use this product very quickly?

Can I complete tasks without the help of a technical person?

Do I need to learn a lot of things before I can get going?

How much hand-holding is needed for return users?

Do I remember where things are in the system?

Do I remember what the icons mean in a system?

Is it easy to learn?

how well the product prevents errors and helps the user recover from any that do occur

Effective Efficient Easy to Learn Error Tolerant Engaging

iancvt55, Flickr, http://www.flickr.com/photos/iancvt55/13088416513/

Are error messages clear?

How often are mistakes being made?

Is it my fault that the screen froze?

Do users know how to recover from an error?

How do I get out of this mess?

Where can I go for help? HELP!!!

Is it error tolerant?

how pleasant, satisfying or interesting an interface is to use

Effective Efficient Easy to Learn Error Tolerant Engaging

Fernando Messino, Flickr, https://www.flickr.com/photos/krudo/5194217402

How do I keep users coming back?

Are people satisfied of the product?

What is the impression of the product?

What pleases as many users as possible?

Do users enjoy/like using this product?

Will existing users recommend this product to others?

Is it engaging?

Usability Testing (onsite/remote)Observing user performing tasks with your product.

Create a Test Plan - Step 1

Define Scope & Objective

Exactly what are you testing? How much of the product will the test cover?

E.g. Can someone make a purchase on Site X’s v2 new prototype?

Create a Test Plan - Step 2

Define Scenario

What is the user mindset when going through this part of the product?

E.g. You’ve been looking for a new computer chair. Your coworker recommended Site X to you.

Create a Test Plan - Step 3

Define Test Metrics

What data are you collecting and measuring? Task completion? Errors? Time on task? User feedback?

E.g. Likelihood to user, likelihood to recommend, overall satisfaction, ease of use, confusions, likes or dislikes, etc.

Create a Test Plan - Step 4

Recruit Participants

What type of participants will be tested? How many participants are needed? Screener?

E.g. Male/Female, 22–45, works 30+ hours a week, must have a job that requires sitting 6+ hours a day.

Create a Test Plan - Step 5

Record Testing Results

Video, audio, or other tools? Permission to record? Who’s the note-taker and who’s the facilitator?

If you are doing the test yourself, make sure you have a recording device to help you capture details.

Create a Test Plan - Step 6

Run the test

When and where? How long?

No more than 90 minutes. Must reset the environment for each participant.

How many test users does it take?

The answer is 5, except when it's not. Most arguments for using more test participants are wrong, but some tests should be bigger and some smaller.

— Nielsen Norman Group

Moderating techniquesName Pros Cons

ConcurrentThink Aloud

• Understand participants’ pain points as they occur

• Real-time feedback and emotional response

Interfere with usability metrics, i.e. accuracy and time on task

RetrospectiveThink Aloud

Doesn’t interfere with usability metrics

• Session length increase • Difficulty in remembering

feedback and thoughts

ConcurrentProbing

Understand participants’ thoughts as they attempt to work through a

task

Interfere with natural thought process and progress

RetrospectiveProbing

Doesn’t interfere with usability metrics

Difficulty in remembering feedback and thoughts

Jennifer Romano Bergstrom, Ph.D., UX Project Leader, Fors Marsh Group; President, UXPA DC. “Moderating Usability Tests"

‣http://www.usability.gov/how-to-and-tools/resources/templates.html

‣http://remoteresear.ch/tools/

‣http://www.uxbooth.com/articles/the-art-of-guerilla-usability-testing/

‣http://www.slideshare.net/andybudd/guerilla-usability-testing

‣http://uxdesign.smashingmagazine.com/2013/01/08/improving-your-website-usability-tests/

Resources & Reading