ISTD 2003

Interactive Systems Technical Design

Lecture #5

Usability engineering

[Source: Jakob Nielsen, Usability Engineering, Morgan Kaufmann, 1993]

ISTD 2003

What is Usability?

Social acceptability

Practical acceptability

Reliability Compatibility

CostUsefulness …

UtilityUsability

Learnability

Efficiency Memorability

Errors

Satisfaction

System acceptability

ISTD 2003

Usability Attributes

• Learnability: The system should be easy to learn so that the user can rapidly start getting some work done with the system.

• Efficiency: The system should be efficient to use, so that once the user has learned the system, a high level of productivity is possible.

• Memorability: The system should be easy to remember, so that the casual user is able to return to the system after some period of not having used it, without having to learn everything all over again.

• Errors: The system should have a low error rate, so that users make few errors during the use of the system, and so that if they do make errors they can easily recover from them. Further, catastrophic errors must not occur.

• Satisfaction: The system should be pleasant to use, so that users are subjectively satisfied when using it; they like it.

ISTD 2003

Usability Engineering Lifecycle1. Know the user

a. Individual user characteristicsb. The user’s current and desired tasksc. Functional analysisd. The evolution of the user and the job

2. Competitive analysis3. Setting usability goals

a. Financial impact analysis

4. Parallel design5. Participatory design6. Coordinated design of the total interface7. Apply guidelines and heuristic analysis8. Prototyping9. Empirical testing10. Iterative design

a. Capture design rationale

11. Collect feedback from field use

ISTD 2003

Usability Heuristics

• Simple and natural dialogue• Simplify, less is more• Match the users’ task in as natural way as possible

• Speak the users’ language• Employ users’ language and terminolory• Mappings and metaphors

• Minimize user memory load• Consistency• Feedback

• Response time (0.1 -> 1.0 -> 10 seconds)• System failure

ISTD 2003

Usability Heuristics (cont.)

• Clearly marked exits• Do not trap the user

• Shortcuts• Accelerators for experienced users

• Good error messages• Clear language, precise, constructive, polite

• Prevent errors• Confirmations (”do you really want to delete all files?”)• Avoid modes

• Help and documentation• Most users do not read manuals• Search -> Understand -> Apply

ISTD 2003

Case Study: Assessing Usability of Pointing Devices

OKOK

W1=4W2=3

D1=5

D2=3.5

cursor

A) B)

button

In which case it is more difficult to point the button with the cursor?

ISTD 2003

Fitts’ Law

Index of difficulty = log2(2D/W)

D = distance to the object

W = width of the object

Time to point = C1 + C2 (Index of difficulty)

C1, C2 = device specific constants

A) Index of difficulty = log2(2*5/4) = log2(2.5)

B) Index of difficulty = log2(2*3.5/3) = log2(2.33)

ISTD 2003

Usability Testing

• User testing with real users

• Fundamental usability assessment method

• Provides direct information about …• How the system is used• What are the exact problems

ISTD 2003

Usability Testing (cont.)

Methodological pitfalls

• Reliability: would the test produce the same result if repeated?• Caused by huge individual differences between test

users

• Validity: does the result actually reflect the usability issues that one wants to test?• Caused by using wrong users or tasks

ISTD 2003

Usability Testing (cont.)

1. Test plan• Goals, schedule, budget, pilot tests, …

Usability_Problems_Found(i) = N(1 – (1- P)i)

i = number of test users

N = total number of usability problems

P = probability of finding any singleusability problem with any single

test user

ISTD 2003

Usability Testing (cont.)

2. Getting test users• Novice vs. expert users

• Between-subjects vs. within-subjects testing

3. Choosing experimenters• Previous experience on usability testing is valuable

4. Ethical aspects of tests with human subjects• ”System is being tested, not the user”

• Respect the test user and his/her privacy

5. Choosing test tasks• Representative tasks

6. Conducting the test1. Preparation

2. Introduction

3. The test itself

4. Debriefing

ISTD 2003

Other Usability Assessment Methods

• Heuristic evaluation• Performance measurement• Thinking aloud• Observation• Questionnaires• Interviews• Focus groups• Logging actual use• User feedback

ISTD 2003

Heuristic Evaluation(”Discount Usability Engineering”)

Evaluation of a system using a set of heuristic rules or guidelines

• Lifecycle stage: Early design, ”inner cycle” of iterative design

• Users needed: None

• Main advantage: Finds individual usability problems. Can address expert user issues.

• Main disadvantage: Does not involve real users, so does not find

”surprises” related to their needs.

ISTD 2003

How to Conduct Heuristic Evaluation

• Involve multiple evaluators (3-5)• Each evaluator independently analyzes the system

wrt. a list of recognized usability principles• Aggregate evaluators’ feedback

ISTD 2003

Ten Usability Heuristics by Nielsen

• Visibility of system status • The system should always keep users informed about what is going on,

through appropriate feedback within reasonable time.

• Match between system and the real world • The system should speak the users' language, with words, phrases and

concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.

• User control and freedom • Users often choose system functions by mistake and will need a clearly

marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.

• Consistency and standards • Users should not have to wonder whether different words, situations, or

actions mean the same thing. Follow platform conventions.

• Error prevention • Even better than good error messages is a careful design which prevents a

problem from occurring in the first place.

ISTD 2003

Ten Usability Heuristics by Nielsen (cont.)• Recognition rather than recall

• Make objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.

• Flexibility and efficiency of use • Accelerators -- unseen by the novice user -- may often speed up the

interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.

• Aesthetic and minimalist design • Dialogues should not contain information which is irrelevant or rarely needed.

Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.

• Help users recognize, diagnose, and recover from errors • Error messages should be expressed in plain language (no codes), precisely

indicate the problem, and constructively suggest a solution. • Help and documentation

• Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.

ISTD 2003

Performance Measurement

Quantitative and/or qualitative measurements when the system is used for completing a specific task or tasks:

number of errors, time needed, proportion of satisfied/frustrated users, etc.

• Lifecycle stage: Competitive analysis, final testing

• Users needed: At least 10

• Main advantage: Hard numbers. Results easy to compare.

• Main disadvantage: Does not find individual usability problems.

ISTD 2003

Thinking Aloud

User uses the system while continuously thinking out loud

• Lifecycle stage: Iterative design, formative evaluation

• Users needed: 3-5

• Main advantage: Pinpoints user misconceptions. Cheap test.

• Main disadvantage: Unnatural for users. Hard for expert users to verbailize.

ISTD 2003

Observation

Observation of the user working, interfering with them as little as possible

• Lifecycle stage: Task analysis, follow-up studies

• Users needed: 3 or more

• Main advantage: Ecological validity; reveals user’s real tasks. Suggests functions and features.

• Main disadvantage: Appointments hard to set up. No experimenter control.

ISTD 2003

Questionnaires

Users are presented with a written questionnaire

• Lifecycle stage: Task analysis, follow-up studies

• Users needed: At least 30

• Main advantage: Finds subjective user preferences. Easy to repeat.

• Main disadvantage: Pilot work needed (to prevent misunderstandings). Data may not be

reliable.

ISTD 2003

Interviews

Users are interviewed and presented with a set of questions

• Lifecycle stage: Task analysis

• Users needed: 5

• Main advantage: Flexible, in-depth attitude and experience probing.

• Main disadvantage: Time consuming. Hard to analyze and compare.

ISTD 2003

Focus Groups

Users are brought together to discuss and identify issues

• Lifecycle stage: Task analysis, user involvement

• Users needed: 6-9 per group

• Main advantage: Spontaneous reactions and group dynamics.

• Main disadvantage: Hard to analyze. Low validity.

ISTD 2003

Logging Actual Use

Automatic collection of statistics about the detailed use of the system

• Lifecycle stage: Final testing, follow-up studies

• Users needed: At least 20

• Main advantage: Finds highly used (or unused) features. Can run continuously.

• Main disadvantage: Analysis programs needed for huge mass of data. Violation of users’ privacy.

ISTD 2003

User Feedback

Collection of feedback from the users using the system

• Lifecycle stage: Follow-up studies

• Users needed: Hundreds

• Main advantage: Tracks changes in user requirements and views.

• Main disadvantage: Special organization needed to handle replies.