1

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

Usability Testing: Theory and Practice

Johan Åberg

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

Content

• Introducing usability

• Course structure

• Background

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

INTRODUCING USABILITY

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

Creating usability�

• � has nothing to do with common sense

• Common sense resulted in e.g...

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

2

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

Unfortunate white space

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

Creating usability�

• ... Does not mean that you can start with yourself

• This resulted in e.g...

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

Usability – medieval helpdesk

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

COURSE STRUCTURE

3

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

Theoretical part

• 7 sessions

• Covering 2 text books + articles

• Session leaders (2 per session)– Present material

– Lead discussion

• Everybody– Active participation

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

Textbooks

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

Practical part

• Plan, carry out, and present the results of a usability study– Propose plan for examiner in advance

• Sit in on another student’s usability study– Observe moderation issues and give constructive

feedback (cc to examiner)

– Textbook on moderation issues

• Choose system to study– Research possibilities

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

BACKGROUND

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

Usability Testing

• Users perform realistic tasks in the prototype or system

• Measuring metrics

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

Questions

• Will the users like the product?

• Is the new product more efficient than the previous?

• Comparison with competing products?

• Most important usability problems?

• Is the product getting better with each new iteration?

4

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

Criteria for selecting participants

• Self-reported expertise

– E.g. beginner, intermediate, expert, �

• Usage frequency

– Number of visits per month�

• Amount of experience

– Days, months, years, �

• Activities

– Used a certain function, �

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

Number of participants

• Aim of the test

• Tolerance of error marginNumbersuccessful

Number of participants

Lower 95% confidence

Upper 95% confidence

4 5 36% 98%

8 10 48% 95%

16 20 58% 95%

24 30 62% 91%

40 50 67% 89%

80 100 71% 86%

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

Terminology 1

• Within-subjects

• Between-subject

• Balance for possible learning between tasks

Participant Task 1 Task 2 Task 3 Task 4

P1 T1 T2 T3 T4

P2 T3 T1 T4 T2

P3 T2 T4 T1 T3

P4 T4 T3 T2 T1

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

Terminology 2

• Independent variable – that which you manipulate or control, e.g.

• Characteristic of participant (age, gender, relevant experience)

• Different solutions or prototypes being tested

• Tasks

• Dependent variable – that which you measure, e.g.• Task success

• Time

• SUS score

• �

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

Datatypes

• Nominal (categorical)• E.g. Male, Female; Design A, Design B

• Ordinal• E.g. Ranking of 4 designs, from Most beatiful to Least

beutiful

• Interval• E.g. 7-point scale: ”This design is beautiful. Completely

agrees � Completely disagrees

• Ratio• E.g. Time, Task success %

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

Datatypes

• Are these scales equal?

• The upper is ordinal. Only compute frequency.

• The lower can be considered as interval. You can compute a mean.

5

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

Confidence intervals

• Assume that this is your time data from a study with 5 participants

• What does this mean?

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

Confidence intervals

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

Comparing means

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

Trend Lines

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

Chi-square test

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

Show error bars!

6

HUMAN-CENTERED SYSTEMS

Johan Åberg, HCS

How to show error bars?