discount evaluation evaluating with experts. agenda part 4 preview heuristic evaluation perform he...
Post on 20-Dec-2015
215 views
TRANSCRIPT
![Page 1: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/1.jpg)
Discount Evaluation
Evaluating with experts
![Page 2: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/2.jpg)
Agenda
Part 4 preview Heuristic Evaluation Perform HE on each other’s
prototypes Cognitive Walkthrough Perform CW on each other’s
prototypes
![Page 3: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/3.jpg)
Project part 4
I’ll make comments on your evaluation plans by Tuesday
Perform the evaluationsClearly inform your users what you
are doing and why. If you are audio or video recording, I
prefer you use a consent form.Pilot at least once – know how long its
going to take.If you need equipment, ask me
![Page 4: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/4.jpg)
Part 4 write up
State exactly what you did (task list, how many, questionnaires etc.)
Summarize data collected Summarize usability conclusions
based on your data Discuss implications for the prototype
based on those conclusions
![Page 5: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/5.jpg)
Discount Evaluation Techniques
Basis:Observing users can be time-
consuming and expensiveTry to predict usability rather than
observing it directlyConserve resources (quick & low cost)
![Page 6: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/6.jpg)
Approach - inspections
Expert reviewers usedHCI experts interact with system and
try to find potential problems and give prescriptive feedback
Best if• Haven’t used earlier prototype• Familiar with domain or task• Understand user perspectives
Does not require working system
![Page 7: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/7.jpg)
Heuristic Evaluation
Developed by Jakob Nielsen
Several expert usability evaluators assess system based on simple and general heuristics (principles or rules of thumb)
(Web site: www.useit.com)
![Page 8: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/8.jpg)
How many experts?
Nielsen found thatabout 5 evaluations found 75% of the problems
Above that you get more, but at decreasing efficiency
![Page 9: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/9.jpg)
Evaluate System
Reviewers need a prototype May vary from mock-ups and storyboards to a
working system
Reviewers evaluate system based on high-level heuristics.
Where to get heuristics? http://www.useit.com/papers/heuristic/ http://www.asktog.com/basics/firstPrinciples.html
![Page 10: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/10.jpg)
Heuristics
use simple and natural dialog
speak user’s language
minimize memory load
be consistent provide feedback
provide clearly marked exits
provide shortcuts provide good error
messages prevent errors
![Page 11: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/11.jpg)
Neilsen’s Heuristics
visibility of system status
aesthetic and minimalist design
user control and freedom
consistency and standards
error prevention
recognition rather than recall
flexibility and efficiency of use
recognition, diagnosis and recovery from errors
help and documentation
match between system and real world
![Page 12: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/12.jpg)
Groupware heuristics
Provide the means for intentional and appropriate verbal communication
Provide the means for intentional and appropriate gestural communication
Provide consequential communication of an individual’s embodiment
Provide consequential communication of shared artifacts (i.e. artifact feedthrough)
Provide Protection Manage the transitions between tightly and loosely-coupled
collaboration Support people with the coordination of their actions Facilitate finding collaborators and establishing contact
Baker, Greenberg, and Gutwin, CSCW 2002
![Page 13: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/13.jpg)
Ambient heuristics
Useful and relevant information “Peripherality” of display Match between design of ambient display
and environments Sufficient information design Consistent and intuitive mapping Easy transition to more in-depth information Visibility of state Aesthetic and Pleasing Design
Mankoff, et al, CHI 2003
![Page 14: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/14.jpg)
Process
Perform two or more passes through system inspectingFlow from screen to screenEach screen
Evaluate against heuristics Find “problems”
Subjective (if you think it is, it is)Don’t dwell on whether it is or isn’t
![Page 15: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/15.jpg)
Debriefing
Organize all problems found by different reviewersAt this point, decide what are and
aren’t problemsGroup, structureDocument and record them
![Page 16: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/16.jpg)
Severity Rating
Based on frequency impact persistence market impact
Rating scale 0: not a problem 1: cosmetic issue, only fixed if extra time 2: minor usability problem, low priority 3: major usability problem, high priority 4: usability catastrophe, must be fixed
![Page 17: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/17.jpg)
Advantages
Few ethical issues to consider Inexpensive, quick
Getting someone practiced in method and knowledgeable of domain is valuable
![Page 18: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/18.jpg)
Challenges
Very subjective assessment of problemsDepends of expertise of reviewers
Why are these the right heuristics?Others have been suggested
How to determine what is a true usability problemSome recent papers suggest that many
identified “problems” really aren’t
![Page 19: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/19.jpg)
Let’s practice: PAL
![Page 20: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/20.jpg)
Heuristics
use simple and natural dialog
speak user’s language
minimize memory load
be consistent provide feedback
provide clearly marked exits
provide shortcuts provide good error
messages prevent errors
![Page 21: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/21.jpg)
Your turn
Prepare Determine your heuristics Demo your prototype to all of us
Evaluate HA eval Mini-e Mini-e eval GroupOne GroupOne eval HA
Debrief On your own, include in Part 4
![Page 22: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/22.jpg)
Cognitive Walkthrough
More evaluation without users
![Page 23: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/23.jpg)
Cognitive Walkthrough
Assess learnability and usability through simulation of way novice users explore and become familiar with interactive system
A usability “thought experiment” Like code walkthrough (s/w
engineering) From Polson, Lewis, et al at UC Boulder
![Page 24: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/24.jpg)
CW: Process
Construct carefully designed tasks from system spec or screen mock-up
Walk through (cognitive & operational) activities required to go from one screen to another
Review actions needed for task, attempt to predict how users would behave and what problems they’ll encounter
![Page 25: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/25.jpg)
CW: Assumptions
User has rough plan User explores system, looking for
actions to contribute to performance of action
User selects action seems best for desired goal
User interprets response and assesses whether progress has been made toward completing task
![Page 26: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/26.jpg)
CW: Requirements
Description of users and their backgrounds
Description of task user is to perform Complete list of the actions required
to complete task Prototype or description of system
![Page 27: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/27.jpg)
CW: Methodology
Step through action sequenceAction 1Response A, B, ..Action 2Response A...
For each one, ask four questions and try to construct a believability story
![Page 28: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/28.jpg)
CW: Questions
1. Will users be trying to produce whatever effect action has?
2. Will users be able to notice that the correct action is available? (is it visible)
3. Once found, will they know it’s the right one for desired effect? (is it correct)
4. Will users understand feedback after action?
![Page 29: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/29.jpg)
CW: Answering the Questions
1. Will user be trying to produce effect?Typical supporting evidence
• It is part of their original task• They have experience using the system• The system tells them to do it
No evidence?• Construct a failure scenario• Explain, back up opinion
![Page 30: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/30.jpg)
CW: Next Question
2.Will user notice action is available?Typical supporting evidence
• Experience• Visible device, such as a button • Perceivable representation of an action
such as a menu item
![Page 31: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/31.jpg)
CW: Next Question
3.Will user know it’s the right one for the effect?Typical supporting evidence
• Experience• Interface provides a visual item (such as
prompt) to connect action to result effect • All other actions look wrong
![Page 32: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/32.jpg)
CW: Next Question
4.Will user understand the feedback?Typical supporting evidence
• Experience• Recognize a connection between a
system response and what user was trying to do
![Page 33: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/33.jpg)
CW: Questions
1. Will users be trying to produce whatever effect action has?
2. Will users be able to notice that the correct action is available? (is it visible)
3. Once found, will they know it’s the right one for desired effect? (is it correct)
4. Will users understand feedback after action?
![Page 34: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/34.jpg)
Let’s practice: PAL
![Page 35: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/35.jpg)
User characteristics
Technology savy users Familiar with computers Comfortable with basic cell phone
operations Familiar with many features of cell
phone, but not expert
![Page 36: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/36.jpg)
Task:Party helper
Heather goes to a reception with some conference attendees. Heather doesn’t know many of them but wants to make a good impression since she’ll be looking for a job in a few months, so she wants to use PAL to remind her of names of folks she meets during the evening.
![Page 37: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/37.jpg)
Action list
Assume PAL is running…
1. When introduced to someone new, press “earmark” button and repeat the name.
2. Assume no intervening introductions…
3. When there is a break, press and hold the back navigation button until bar reaches earmark.
4. Listen to name being repeated
![Page 38: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/38.jpg)
CW Summary
Advantages Explores important
characteristic of learnability
Novice perspective Detailed, careful
examination Working prototype
not necessary
Disadvantages Can be time
consuming May find problems
that aren’t really problems
Narrow focus, may not evaluate entire interface
![Page 39: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/39.jpg)
Your turn
Prepare: Describe users and their experience Determine task(s) and action lists
Evaluation: HA evaluate GroupOne GroupOne evaluate Mini-e Mini-e evaluate HA
As a group, for each action, answer the 4 questions with evidence
![Page 40: Discount Evaluation Evaluating with experts. Agenda Part 4 preview Heuristic Evaluation Perform HE on each other’s prototypes Cognitive Walkthrough Perform](https://reader031.vdocument.in/reader031/viewer/2022032704/56649d485503460f94a23fc2/html5/thumbnails/40.jpg)
CW: Questions
1. Will users be trying to produce whatever effect action has?
2. Will users be able to notice that the correct action is available? (is it visible)
3. Once found, will they know it’s the right one for desired effect? (is it correct)
4. Will users understand feedback after action?