Download - Pragmatic Usability Evaluation. Overview zHeuristic Evaluation zEvaluators zUsability Metrics
![Page 1: Pragmatic Usability Evaluation. Overview zHeuristic Evaluation zEvaluators zUsability Metrics](https://reader036.vdocument.in/reader036/viewer/2022082817/56649db05503460f94a9dfdf/html5/thumbnails/1.jpg)
Pragmatic Usability Evaluation
![Page 2: Pragmatic Usability Evaluation. Overview zHeuristic Evaluation zEvaluators zUsability Metrics](https://reader036.vdocument.in/reader036/viewer/2022082817/56649db05503460f94a9dfdf/html5/thumbnails/2.jpg)
Overview
Heuristic EvaluationEvaluatorsUsability Metrics
![Page 3: Pragmatic Usability Evaluation. Overview zHeuristic Evaluation zEvaluators zUsability Metrics](https://reader036.vdocument.in/reader036/viewer/2022082817/56649db05503460f94a9dfdf/html5/thumbnails/3.jpg)
Heuristic Evaluation
‘heuristic’ - ‘used of problem solving techniques that proceed by trial and error’ (related to Greek ‘eureka’)
Longman Concise English Dictionary, 1985
a method of usability evaluation where an analyst finds usability problems by checking the user interface against a set of supplied heuristics or principles
Lavery, Cockton and Atkinson, 1996
![Page 4: Pragmatic Usability Evaluation. Overview zHeuristic Evaluation zEvaluators zUsability Metrics](https://reader036.vdocument.in/reader036/viewer/2022082817/56649db05503460f94a9dfdf/html5/thumbnails/4.jpg)
Who should do heuristic evaluation?
Use more than one evaluatorIdeally should not be the designerIdeally should be a usability specialist
technical authors also usefulEach carries out independent inspection, then
aggregate findingsEvaluators may need help
unless ‘walk-up-and-use’ application could provide typical usage scenario
![Page 5: Pragmatic Usability Evaluation. Overview zHeuristic Evaluation zEvaluators zUsability Metrics](https://reader036.vdocument.in/reader036/viewer/2022082817/56649db05503460f94a9dfdf/html5/thumbnails/5.jpg)
Effectiveness of increasing number of evaluators
0
25
50
75
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
number of evaluators
per
cen
tag
e o
f u
sab
ilit
y p
rob
lem
s fo
un
d
(taken from Nielsen, 1993)
![Page 6: Pragmatic Usability Evaluation. Overview zHeuristic Evaluation zEvaluators zUsability Metrics](https://reader036.vdocument.in/reader036/viewer/2022082817/56649db05503460f94a9dfdf/html5/thumbnails/6.jpg)
Evaluator Performance
Nielsen 1992: same interface
evaluated by 3 groups novice (knowledge
about computers only)
usability experts usability and
specialist domain experts
22%
41%
60%
0%
25%
50%
75%
100%
pro
ble
ms
no
vic
es
sin
gle
ex
pe
rts
do
ub
lee
xp
ert
s
![Page 7: Pragmatic Usability Evaluation. Overview zHeuristic Evaluation zEvaluators zUsability Metrics](https://reader036.vdocument.in/reader036/viewer/2022082817/56649db05503460f94a9dfdf/html5/thumbnails/7.jpg)
Performing Heuristic Evaluation
Go through interface, compare against recognised usability principles first pass flow of interaction and general
scope subsequently focus on specific elements
Typically 1 - 2 hours in totalOutput: list of usability problems cross-
referenced to usability guidelines
![Page 8: Pragmatic Usability Evaluation. Overview zHeuristic Evaluation zEvaluators zUsability Metrics](https://reader036.vdocument.in/reader036/viewer/2022082817/56649db05503460f94a9dfdf/html5/thumbnails/8.jpg)
Usability Metrics
learnability
memorability
errors
subjective satisfaction
efficiency
![Page 9: Pragmatic Usability Evaluation. Overview zHeuristic Evaluation zEvaluators zUsability Metrics](https://reader036.vdocument.in/reader036/viewer/2022082817/56649db05503460f94a9dfdf/html5/thumbnails/9.jpg)
Setting Usability Metrics
skill and intuition...better than last versionbetter than the competitionclient targetsset a range of levels
unacceptable minimum target ideal
![Page 10: Pragmatic Usability Evaluation. Overview zHeuristic Evaluation zEvaluators zUsability Metrics](https://reader036.vdocument.in/reader036/viewer/2022082817/56649db05503460f94a9dfdf/html5/thumbnails/10.jpg)
Usability Profile
unacceptable minimum target ideal
learnability
efficiency
memorability
errors
satisfaction
![Page 11: Pragmatic Usability Evaluation. Overview zHeuristic Evaluation zEvaluators zUsability Metrics](https://reader036.vdocument.in/reader036/viewer/2022082817/56649db05503460f94a9dfdf/html5/thumbnails/11.jpg)
Usability Testing - Planning
draw up a test plan see separate handout ‘Checklist for
usability test plans’ informal use by the test team formal use for QA procedures
consider whether to use video/audio recording
![Page 12: Pragmatic Usability Evaluation. Overview zHeuristic Evaluation zEvaluators zUsability Metrics](https://reader036.vdocument.in/reader036/viewer/2022082817/56649db05503460f94a9dfdf/html5/thumbnails/12.jpg)
Usability Testing - Users
test users should represent target users remember sales staff as a special user group may need to give basic training
getting hold of users internal users should be easy customers from user groups may help paid volunteers : students, classified ads..
take account of older users if relevant
![Page 13: Pragmatic Usability Evaluation. Overview zHeuristic Evaluation zEvaluators zUsability Metrics](https://reader036.vdocument.in/reader036/viewer/2022082817/56649db05503460f94a9dfdf/html5/thumbnails/13.jpg)
Pay-off ratio for user testing(after Nielsen, 1993)
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10 11 12
number of test users
rati
o o
f b
en
efi
t to
co
st
![Page 14: Pragmatic Usability Evaluation. Overview zHeuristic Evaluation zEvaluators zUsability Metrics](https://reader036.vdocument.in/reader036/viewer/2022082817/56649db05503460f94a9dfdf/html5/thumbnails/14.jpg)
Usability Testing - designing test tasks
representative & provide reasonable coverage
do-able but not trivialconsider relating to a larger scenarioprovide a written task descriptionpresent in increasing level of difficultydecide whether to use verbal protocols
![Page 15: Pragmatic Usability Evaluation. Overview zHeuristic Evaluation zEvaluators zUsability Metrics](https://reader036.vdocument.in/reader036/viewer/2022082817/56649db05503460f94a9dfdf/html5/thumbnails/15.jpg)
Relative effectiveness
Karat et al (1992)expert individual & group walkthroughs
used guidelines and tasksusability testing
users identified and described problemstesting identified most problems
including some severe ones missed by experts
![Page 16: Pragmatic Usability Evaluation. Overview zHeuristic Evaluation zEvaluators zUsability Metrics](https://reader036.vdocument.in/reader036/viewer/2022082817/56649db05503460f94a9dfdf/html5/thumbnails/16.jpg)
Effectiveness continued
walkthroughs useful when resources linited, or for early design
team walkthroughs better than individuals
techniques are complementarycost effectiveness similaralso formal experimental trials
![Page 17: Pragmatic Usability Evaluation. Overview zHeuristic Evaluation zEvaluators zUsability Metrics](https://reader036.vdocument.in/reader036/viewer/2022082817/56649db05503460f94a9dfdf/html5/thumbnails/17.jpg)
Usability testing - procedure
preparation remember to switch off screen-savers, email, etc.
introductiontestingdebriefing
questionnaires if used also ask about the testing process
write up quickly
![Page 18: Pragmatic Usability Evaluation. Overview zHeuristic Evaluation zEvaluators zUsability Metrics](https://reader036.vdocument.in/reader036/viewer/2022082817/56649db05503460f94a9dfdf/html5/thumbnails/18.jpg)
usability metrics
learnability time to reach specified
level of proficiency e.g. complete a specified,
representative task
note that learning is a continuum
memorability test users on commands
after trial session
errors number of errors in
completing specified task
subjective satisfaction rating scales physiological measures
efficiency times for experts to
complete specified task(s)
frequency of ‘non-productive’ actions
ratio of used to unused commands