holistic usability measure (hum)
DESCRIPTION
Holistic Usability Measure (HUM). Juan E. Gilbert, Ph.D. IDEaS Professor & Chair Human-Centered Computing Division School of Computing Professor Automotive Engineering Clemson University [email protected] http://www.JuanGilbert.com/ http://www.HumanCenteredComputing.org/ - PowerPoint PPT PresentationTRANSCRIPT
Juan E. Gilbert, Ph.D.IDEaS Professor & Chair Human-Centered Computing Division
School of Computing
Professor Automotive EngineeringClemson [email protected]
http://www.JuanGilbert.com/http://www.HumanCenteredComputing.org/
http://www.clemson.edu/computing
Holistic Usability Measure (HUM)
2 of 18
Usability! “If you could build a system
that resulted in world peace, but no one could use it ... it would be useless. Usability matters.” – Gilbert
However, “there has been no generally agreed-upon method of measuring usability, which makes it difficult to compare among different findings.” - Polkosky
Holistic Usability What is Holistic Usability?
Usability Effectiveness, Efficiency, User Satisfaction
What really counts? Effectiveness, Efficiency, or User Satisfaction?
Holistic Usability A single measure of the overall usability
System Usability Scale (SUS) 10 statements to which
users rate their level of agreement
Half positive, half negative
5-pt scale of agreement
Tullis and Albert’s Figure 6.8, based on Brooke 1996
Taken from http://jldrury.googlepages.com/8IssuesSelf-ReportedMetrics10-26-08.ppt
System Usability Scale (SUS) Each item's score contribution will range from 0 to 4. For items 1,3,5,7,and 9 the score contribution is the
scale position minus 1. For items 2,4,6,8 and 10, the contribution is 5 minus
the scale position. Multiply the sum of the scores by 2.5 to obtain the
overall SUS score.
System Usability Scale (SUS) For items 1,3,5,7,and 9 the
score contribution is the scale position minus 1.
For items 2,4,6,8 and 10, the contribution is 5 minus the scale position.
Multiply the sum of the scores by 2.5 to obtain the overall SUS score.
Scores are on 100 point scale
Taken from http://jldrury.googlepages.com/8IssuesSelf-ReportedMetrics10-26-08.ppt
System Usability Scale (SUS) SUS SCORE 0-60 GRADE = F
SUS SCORE 60-70 GRADE = DSUS SCORE 70-80 GRADE = CSUS SCORE 80-90 GRADE = BSUS SCORE 90-100 GRADE = A
System Usability Scale (SUS) Nice for comparing systems
Short questionnaire
Easy to calculate
Doesn’t work across all systems
Holistic Usability Measure
10 of 18
Usability Goals
What is the goal of your interface/system?
Who decides the primary goal of the system? Designer Client Manager
11 of 18
Usability Goals
Did you build the right system/interface? Who decides?
How do you know if your system is better than another that does the same tasks?
You must test it!
12 of 18
Background During requirement analysis, before design
Information design; what is to be built What is the system/interface suppose to accomplish? Who are the users? Etc.
These are typical, but most people miss Usability metrics These are typically an afterthought As a result, usability gets cut
13 of 18
Background
Quantitative Metrics Speech recognition accuracy Task completion time
Subjective Metrics User friendliness Ease of use
14 of 18
Background
Which metrics are most important? It depends on your system/interface Decide how you will test your system before
you design it
Given several different metrics How do you measure the overall effectiveness
of the system/interface?
15 of 18
Holistic Usability Measure
HUM Assign each metric a weight (1 – 99%) based
on its importance
Design and implement the system/interface
Evaluate it Convert all metrics to a similar numerical scale
16 of 18
Holistic Usability Measure
HUM = W1*Metric1 + W2*Metric2 + ... + Wn*Metric n
where W1 + W2 + ... + Wn = 1
and 0 ≤ W1, W2, ..., Wn ≤ 1
HUM yields a percentage measure rating the usability according to the designer’s intent.
17 of 18
Conclusions HUM is a tool for testing
It reports usability with respect to the usability goals
Multiple ways to design same system. Which one is the best? HUM is ideal for comparing more than one implementation of the
same VUI
HUM is easy to use and it makes sense
Recall, “there has been no generally agreed-upon method of measuring usability, which makes it difficult to compare among different findings.” – Polkosky (SpeechTECH Magazine November/December 2005)
Juan E. Gilbert, Ph.D.IDEaS Professor & Chair Human-Centered Computing Division
School of Computing
Professor Automotive EngineeringClemson [email protected]
http://www.JuanGilbert.com/http://www.HumanCenteredComputing.org/
http://www.clemson.edu/computing
Thank You