Amela Karahasanović

Senior Scientist, SINTEF IKT

Adjunct Associate Professor, DESIGN

amela@sintef.no

www.sintef.no

Sintef IKT, Forskningsveien 1

www.sintef.no

•User modeling

•Usability and user

experience

•Methods for empirical

evaluation in HCI and SE

•Software maintenance

•Program comprehension

Technology for a better society

•Largest Scandinavian independent research organisation

•More than 2100 employees from 67 countries

•Generated knowledge for 2.8 billions of NOK in 2010

•Eight research divisions

•About 16% of our turnover is derived from international projects

2

HCI research group

– Psychology

– Software Engineering

– Media science

3

Wednesday 18th September 13:00 – 14:00

http://gemini-centre-user-experience.origo.no/

– 3D technology

– Internet of things

– Collaboration tool

4

Usability

5

Plan for today

Chapter 10 + some examples

• What is usability testing

• Usability testing versus traditional research

• Types of usability testing

• User-based testing

• Questions for discussion

6

What is usability?

"The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use." (ISO 9241-11)

– Effectiveness - can users complete tasks, achieve goals with the product, i.e. do what they want to do?

– Efficiency - how much effort do users require to do this? (Often measured in time)

– Satisfaction – what do users think about the products ease of use? ….which are affected by: – The users - who is using the product? e.g. are they highly trained and

experienced users, or novices? – Their goals - what are the users trying to do with the product - does it

support what they want to do with it? – The usage situation (or 'context of use') - where and how is the

product being used?

7

Usability cont.

• Usability is composed of: – Learnability: How easy is it for users to accomplish basic

tasks the first time they encounter the design? – Efficiency: Once users have learned the design, how

quickly can they perform tasks? – Memorability: When users return to the design after a

period of not using it, how easily can they re establish proficiency?

– Errors: How many errors do users make, how severe are these errors, and how easily can they recover from the errors?

– Satisfaction: How pleasant is it to use the design? (Nielsen,1993 )

8

What is usability testing?

• Involving representative users, representative tasks, representative environments

– Testing paper prototypes

– Wizard of Oz

– Testing working version before it is released

– Testing working system

– Different devices (smart phones, laptops…)

9

Goal of usability testing

• Improve the quality of an interface by finding flaws in it

– Flaws that cause problems for the majority of people (not preferences)

• What works fine (keep it)

10

Kjdsgklglohofho fhihfdoho

11

Usability versus research?

• Approach similar to one used in classic research

• Different goals

• Usability testing is involved in building a successful product (resources, time, trade-offs)

– Practical, large impact

– Optimize interface after each iteration

• Usability study – Research study - Research on usability methods

12

13

Usability engineering

Activities aiming to improve the ease of use of an interface

• Expert-based testing (usability inspection)

• Automated testing (usability inspection)

• User-based testing (usability testing)

14

Expert-based testing

• Structured inspections done by interface experts • Before tests with users • Confusing wording, inconsistent layout, obvious flaws • Heuristic review

– Compare interface with the rules

• Consistency inspections – Series of screens or web pages inspected

• Cognitive walkthrough – Experts perform the tasks (high-frequency and

important/seldom)

• Guidelines review – Web Content Accessibility Guidelines

15

16

Automated usability testing

• Software that compare interface with the guidelines

• Produce report and/or fix the code • Manual check often needed • <alt> tag (alternative for graphics) but not if the

text is appropriate – 'picture here'

• Number of fonts, avg. font size, deepest level of a menu

• Software applications: RAMP, InFocus

17

User-based testing

– Select representative users

– Select the setting

– Decide what tasks users should perform

– Decide what type of data to collect

– Before the test session (informed consent, etc.)

– During the test session

– Debriefing after the session

18

Questions – Part I

Go to the alarm app on you phone

– Difference between a summative and a formative usability test of this app

– Should we first test it with users or with experts? What do you think? Why?

– What are the differences between a usability test and an experiment with this app?

19

When to test

– Users perform tasks (early or later in the development)

– Formative testing • Low-level fidelity prototypes

• How the user perceives an interface component?

• Low-cost of paper prototypes, users comfortable to criticize

– Summative testing • Evaluate the effectiveness of specific design choices

• High fidelity prototype

– Validation test • Before release, compare to benchmarks

– Formal approach 20

Usability evaluation Talhonia

time

21

Usability evaluation Talhonia – cont.

22

How many users?

– 5 users will find approximately 80% of problems

– 7 for small projects, 15 for medium-large projects

– Goal to find the major flows , that will cause the most problems and must be fixed

– How many users can we afford? How many users can we get? How many users do we have time for?

23

Location

– Lab

– Portable usability lab

– Remote studies (time, place)

24

Tasks

• Clear; no need for further explanation

• Tested

• One clear answer – way to do things

• Tasks that are performed often

• Critical tasks – logging

• No private, financial information

• Must be clear how to go to the next task

25

Oppgave 4: Du er hjemme og du ser NRK Nett-TV på din PC ( http://www.nrk.no/nett-tv ).

Se på episoden 10:12 av Matlyst og svar følgende spørsmål:

• Hva heter kvinnen som er kledd i lilla genser og brunt forkle?

• Når en potet frøs om vinteren, hva ble den da?

26

Measurement

– Task performance

– Time performance

– User satisfaction (validated survey tool)

– Average time to recover from an error

– Time spent using help

– Number of visits to the search feature

– Time spent on specific web page

– Typing speed

– Qualitative data

0

5

10

15

20

25

30

35

40

45

1 3 5 6 7 10 11 12 13 17 18 20 21 23 24 25 26

Unified Library Application - Quit

Unified Library Application

Title Information

ReturnItem

Reserve Title

Lend Item

Insert Title Window

Find Title

Compilation

Subject_id 15 Task 2

Sum of Seconds

Minutes

Visited

27

Testing session

• Preparation

• The session

• After the session

• Example – R2D2 project

28

Starten av dagen: • Sjekke at alt er på plass i laben; gavekort, utstyr, godteri, brus, batterier, pen og papir osv. • Skrive ut eksperimentet material som skal deles til deltagerne • Skrive ut tlf.nr. til de som kommer (mail fra Nordstat) • Informere resepsjonen at vi venter deltakere. Hvis deltaker kommer veldig tidlig, be resepsjonisten

henvise dem til sofa i resepsjonen. • Logge seg på PC og teste alt virker (ingen updates skal dukke opp i løpet av eksperimentet) • Gå inn på facebook og twitter og sjekk at ”thomtor” er logget ut • Heng IKKE FORSTYR på døra • Ha alltid mobilen med deg Eksperimentet (før deltakeren kommer) • Start SINTEF verktøyet (r2d2exp.jar). Det kan ta litt tid – IKKE KLIKK FLERE GANGER! • Start opp camtasia studio (dobbelt klikk) • Start opp uLog (høyre klikk på ikonet i ”system tray” og start) • Eksperiment (hent deltaker) • Ta med deltageren ned; hilse; introduksjon • Konfidensialitet skjema signeres • Husk å sette mobilen på vibrasjon • Start opptak i camtasia studio (høyreklikk, klikk ‘record the screen’, klikk record button) • …

29

30

Making sense of the data

– Write up the results and help influence the design of the specific interface

– Presentation to developers and managers

– Report should • Include all flaws

• Priorities

• For each flaw: describe the problem, present the data, priority, suggest a fix, estimate time/efforts for the fix

– Report structure • How you did usability testing and how you prepared

• What happened during the testing

• The implications and recomendations

31

Questions – Part II Alarm app usability test

• What is important to remind participants before the usability test?

• What is the "think aloud" protocol? Should it be used in summative or formative studies?

• Properties of good tasks?

32

Assignment

Make a PLAN for a summative evaluation of the ‘Contacts’ functionality on your mobile phone in a usability lab with 10 participants.

33