formative evaluation cs3724: hci. problem scenarios summative evaluation information scenarios...
TRANSCRIPT
Problem scenarios
summativeevaluation
Information scenarios
claims about current practice
analysis ofstakeholders,field studies
Usability specifications
Activityscenarios
Interaction scenarios
iterativeanalysis ofusability claims andre-design
metaphors,informationtechnology,HCI theory,guidelines
formativeevaluation
DESIGN
ANALYZE
PROTOTYPE & EVALUATE
Scenario-B
ased Design
User Interface Metrics
• Ease of learning• learning time, …
• Ease of use• perf time, error rates…
• User satisfaction• surveys…
Not “user friendly”
Claims Analysis
• + advantages
• - disadvantages
• E.g. file management:
+ fast learning+ spatial learning aid memory- switching between devices- long to locate a known file…
+ flexible, power+ fast for experts - Learning commands - aesthetics lacking?...
Usability Evaluation
• Analytic Methods:• Heuristic Evaluation
• Models: GOMS, Fitts Law
• Empirical Methods:• Usability Testing
– Wizard of Oz
• Controlled Experiment
Nielsen’s 10 Heuristics1. Visible status, feedback2. User control, undo, exits3. Familiar, speak user’s language4. Consistent, standards5. Recognition over recall6. Efficient, expert shortcuts7. Aesthetic, minimalist8. Prevent errors9. Error recovery10. Help, task-based
Nielsen’s 10 Heuristics1. Visible status, feedback wysiwyg
2. User control, undo, exits wizards
3. Familiar, speak user’s language Acrobat error msg
4. Consistent, standards Word, PPT
5. Recognition over recall web nav, phone menu
6. Efficient, expert shortcuts Word bold
7. Aesthetic, minimalist phone book
8. Prevent errors HomeFinder
9. Error recovery undo, back
10. Help, task-based IIS help doc
Early Usability Testing
• Formative: helps guide design
• Early in design process• when architecture is finalized, then its too late!
• A few users (3-5)
• Identify usability problems, incidents
• Qualitative feedback from users
• Quantitative usability measures
Usability Test Procedure• Wizard of Oz: act out the system• Benchmark task set• No cheating! Don’t teach them how to use the UI• Repeat:
• Give user a task• Ask user to “think aloud”• Observe, note mistakes and problems• Avoid interfering, hint only if completely stuck
• Interview • Verbal feedback• Questionnaire
• 30 min - 1 hour / user
Analyze Results• Initial reaction:
• “stupid user!”, “that’s developer X’s fault!”, “this sucks”
• Mature reaction:• “how can we redesign UI to solve that usability problem?”• the user is always ‘right’
• Identify usability problems• Learning issues: e.g. can’t figure out or didn’t notice feature• Performance issues: e.g. arduous, tiring to solve tasks• Subjective issues: e.g. annoying, ugly
• Problem severity: critical vs. minor• Redesign
Pizza Ordering System
• Scenario:
• Volunteers!
• Design ideas
• Wizard of Oz usability test
• Redesign
In action: Pizza Ordering System
• Scenario: VT students, studying/busy/tired, poor, wants discounts, starving, credit card / hokie card/ no cash, 2 in together, half-n-half, split cost
• Volunteers!
• Design ideas
• Wizard of Oz usability test
• Redesign
• Size of pizza:– large– Small
• First half:
• Toppings:– Peperoni– Mushrooms– Pineapple
• Price: 0.00
• 2nd half:
• Toppings:– Peperoni– Mushrooms– pineapple