Scott Klemmerwww.hci-class.orgHEURISTIC EVALUATIONWHY AND HOWMultipleEmpirical Formal waystoevaluateAssess with real usersModels and formulas to calculate measures Software measures Expertise and heuristicfeedbackAutomated Critique 2Whentogetdesign critique?Before user testing. Dont waste users on the smallstuff. Critique can identify minor issues that can be resolved before testing, allowing users to focus on the big issues.Before redesigning. Dont throw out the baby with the bathwater. Critique can help you learn what works and what should change.When you know there are problems, but you need evidence. Perhaps you've received complaints from customers or found yourself stumbling around your own site. Critique can help you ar ticulate problems and provide you with ammunition for redesign.Before release. Smooth off the rough edges.http://www.etre.com/usability/inspectionBeginReviewwithaClearGoalHeuristicEvaluationDevelopedbyJakobNielsenHelpsSmallfindusabilityproblemsinadesignset(3-5)ofevaluatorsexamineUIindependently check for compliance withusability principles (heuristics) different evaluators will find different problemsevaluators only communicate afterwards findings are then aggregatedCanperformonworking UI orsketchesJacob Nielsen, http://www.useit.com/papers/heuristic Slide summar y adapted from James LandayTenDesignHeuristicsShow system statusFamiliar metaphors & language Control & freedom ConsistencyError preventionRecognitionFlexibility & Aesthetic &over recallefficiencyminimalistdesignrecoverRecognize, diagnose, &from errorsHelpMy somewhat revised names for Jacob Nielsens heuristics, http://www.useit.com/papers/heuristicEvaluatorsProcessStepthroughdesign severaltimesExamine details, flow, and architectureConsult list of usability principles..and anything else that comes tomindWhichprinciples? Nielsens heuristics Category-specific heuristics from e.g., design goals, competitive analysis, existing designsUseviolationstoredesign/fixproblemsJacob Nielsen, http://www.useit.com/papers/heuristic Slide summar y adapted from James LandayWhyMultipleEvaluators?everythingNoevaluatorfindsSomefindmorethanotherssuccessfulEVALUATORSunsuccessfulhardeasyPROBLEMSJacob Nielsen, http://www.useit.com/papers/heuristic Slide summar y adapted from James LandayDecreasing Caveat: graphsproblems foundReturnsforaspecificexamplebenefits / costHeuristic Eval:Cost-effectiveInonecase: benefit-costratio of48estimated benefit $500,000; cost $10,500value of each problem ~$15Khow might we calculate this value? in-house -> productivity; open market -> salesSevereproblemsfoundmoreoftenSingle evaluatorachievespoorresults only finds 35% of usability problems 5 evaluators find ~ 75% of problemsJacob Nielsen, http://www.useit.com/papers/heuristic Slide summar y adapted from James LandayHeuristicsvs.User TestingHeuristicEvaluationoftenfaster 1-2 hours each evaluatorHEresultscomepre-interpretedUsertestingismoreaccurate(bydef.) takes into account actual users and HE may miss problems & find false positivestasksValuabletoalternatemethods find different problems dont waste participantsJacob Nielsen, http://www.useit.com/papers/heuristic Slide summar y adapted from JamesLandayPhasesofHeuristicEvaluation1.Pre-evaluationtraining: giveevaluatorsneededdomainknowledgeandinformationonthescenario2.Evaluation: individualsevaluateandthenaggregate results3.Severityrating: determinehowsevereeachproblemis(priority). Candofirstindividuallyandthenasa group4.Debriefing: review withdesignteamJacob Nielsen, http://www.useit.com/papers/heuristic Slide summar y adapted from James LandayHow-to:Heuristic EvaluationAt leasttwo passes for each evaluatorget feel for flow and scope of system to focus on specific elementsIffirst tosecondsystem is walk-up-and-use or evaluatorsare domain experts, no assistance needed otherwise might supply evaluators with scenariosEach evaluator produces list of problems explain why with reference to heuristic or other information be specific and list each problem separatelyJacob Nielsen, http://www.useit.com/papers/heuristic Slide summar y adapted from James LandayHow-to:HeuristicEvaluationWhy separate listings for each violation? risk of repeating problematic aspect may notbe possible to fix all problemsWhere problems may be foundsingle location in UItwo or more locations that need to be compared problem with overall structure of UIsomething is missing ambiguous with early prototypes; clarify in advance sometimes features are implied by design docs and just havent been implemented relax on thoseJacob Nielsen, http://www.useit.com/papers/heuristic Slide summar y adapted from James LandaySeverityRatingIndependentlyestimateafterreviewAllocate resourcestofixproblemsEstimateSeverityneedformoreusabilityeffortscombinesfrequencyimpact persistenceJacob Nielsen, http://www.useit.com/papers/heuristic Slide summar y adapted from James LandaySeverityRatings01234-----dont agree that this is acosmetic problemminor usability problem major usability problem;usability problemimportantto fixfixusabilitycatastrophe;imperativetoJacob Nielsen, http://www.useit.com/papers/heuristic Slide summar y adapted from James Landay16SeverityRatingsExampleIssue: Unable to edit ones weightSeverity: 2Heuristics violated: User control and freedomDescription: when you open the appforthe first time, you have to enter yourweight, butbe useful if if one yearyou cannot update it. It couldyou mistyped your weight, or or two after the first use ofthe app, your weight has changed.Courtesy Sebastien Robaszkiewicz, cs147 2011DebriefingConductwithevaluators, observers,anddevelopmentteammembersDiscussSuggest addressgeneralcharacteristicsofUIpotentialimprovementstomajorusabilityproblemsDev. teamratesefforttofixBrainstormsolutionsJacob Nielsen, http://www.useit.com/papers/heuristic Slide summar yadapted from James Landay