11/30/16

1

ExpertEvaluationsNovember30,2016

Admin

• Finalassignments• Highqualityexpected

• Slides• Presentationdelivery• Interface(remember,focusisonahigh-fidelityUI)• Reports• Responsive

• Putyourbestfootforward:sharingyourprojectswiththeMayor’sOffice• Explicitlydemonstrateknowledgegainedacrosssemester• Practice,practice,practiceyourtalks!

• RepresentativesfromtheMayor’sOfficenextweek

• ROOMCHANGEfornextweek:425Shillman Hall

11/30/16

2

Admin

• T6• Publically-accessiblelinktoyourprototype(forsharingwithMayor’soffice)

• S-LTimesheetform• http://bit.ly/2g6pTLB

• S-LSurvey:DueWednesdayDec7@6pm• https://www.surveymonkey.com/r/Fall2016_S-LStudentEvaluation

• TRACECourseEvaluations• Pleasecomplete!

ExpertEvaluations

•What• Expertsusetheirknowledgeofusers&technologytoreviewsoftwareusability• Critiques(crits)canbeformalorinformalreports.

• Types• PredictiveModelling• CognitiveWalkthrough• HeuristicEvaluation

• Usedatwhatstage(s)oftheUCD&softwarelifecycles?• Throughoutthelifecycle• Onprototypesofalllevelsoffidelity

11/30/16

3

PredictiveModelsinHCI

•What?• Abstractionsofuserbehaviorthatallowexpertstoestimatehowuserswillinteract• Equations,formulas

• Advantages• Quicker&lessexpensivethanauserstudy• Basedonempiricaldata

• Disadvantages?• Usefulnesslimitedtosystemswithpredictabletasks

• e.g.,telephoneansweringsystems,mobiletextentry,etc.• Basedonexperterror-freebehavior

PredictiveModels

• GOMSModel• Card,Moran&Newell:1980s• Familyofmodelingtechniquestoanalyzecomplexityofinteractivesystems• Taskdecomposition:reducesuserinteractionintobasicactions

• Cognitive,physical,perceptual• Goals

• Stateuserwantstoachieve(bookatrip)• Operators

• Elementarycognitiveprocesses,physicalactions,&perceptualactsperformedtoachievegoals(e.g.,doubleclickmouse,locateicon)

• Empiricallyderivedestimates• Methods

• Sequenceofstepstoaccomplishgoal(eg.dragmouseoverfield,typeinkeywords,pressthegobutton)

• SelectionRules• Howdecidewhichmethodtoselect

11/30/16

4

PredictiveModels

• GOMSModel:uses• Functionalitycoverage:makesurethatmethodsexisttosupporteachusergoal• Predictexecutiontimeforeachgoal• Locatebottlenecks,compareapplicationdesigns• Designinghelpsystems:addressissuesofchallengeidentified

• Assumesexpertuse,withnomistakes

• Verypredictabletasks

PredictiveModels

• KLM:KeystrokeLevelModel• GOMSvariant• Allowsquantitativepredictionsabouthowlongittakesaskilleduserusertoperformatask.• Eg,SearchforaphraseinWord• Taskdecomposition• Empirically-derivedresponsetimesforbasicoperations

• Pressingakey,typingacharacter,pointingmouse• Allowsanalysttocomparesystemsintermsofpredictedperformance• BasedonMHP- ModelHumanProcessor

11/30/16

5

KSLMAccountsfor

• Keystroking TK•Mousebuttonpress TB• Pointing(typicallywithmouse) TP• Handmovementbetweenkeyboardandmouse TH• Drawingstraightlinesegments TD• “Mentalpreparation” TM• SystemResponsetime TR

11/30/16

6

PredictiveModels

• Fitts’Law• Originalmodeldevelopmentin1954• Predictsthetimetomovetoatarget(movementtime,MT)• E.G.,timetomoveamouseorotherpointertoatarget• functionofthedistancefromthetargetobject&theobject’ssize

•Movementassumedtoberapid,error-free,andtargeted• usefulforevaluatingsystemsforwhichthetimetolocateanobjectisimportant,e.g.,acellphone• AdaptationsforHCI

TaskEnvironment

• Modelsmovementofarm-handtoatarget• HandisAcmfromthetarget(Amplitude)• TargetisWcmwide(tolerance)• Assumemovementfollowsstraighthorizontalpath

W

A

11/30/16

7

Components

• ID- Indexofdifficulty

• IDisaninformationtheoretic quantity• FormulationforHCIbyScottMacKenzie• Largertarget=moreinformation(lessuncertainty)

13

ID = log2 (a/w + 1.0)width (tolerance)of target

distance to move

Interpretationoflog2(A/W+1)

• Arm-handmovementrequiremoretimewhen• Distancetotarget(A)increases• Errortolerance(W)decreases• Targetisfurtherawayandofsmallersize

• Arm-handmovementrequireslesstimewhen• Distancetotarget(A)decreases• Errortolerance(W)increases• Targetiscloserandoflargersize

11/30/16

8

Components

• MT- Movementtime

• MTisalinearfunctionofIDk1 andk2 areexperimentalconstants

15

MT = k1 + k2*IDMT = k1 + k2 *log2 (a/w + 1.0)

Equation

• Runempiricalteststodeterminek1 andk2 inMT=k1 +k2*ID• Regressionanalysisonmovementtimedata

• Constantsvarybyinputdevices&deviceuses• E.g.,one-handed&two-handedtextentryonmobiledevices

https://www.interaction-design.org/literature/book/the-glossary-of-human-computer-interaction/fitts-s-law

Silfverberg,M.,MacKenzie,I.S.,&Korhonen,P.(2000,April).Predictingtextentryspeedonmobilephones.InProceedingsoftheSIGCHIconferenceonHumanFactorsinComputingSystems(pp.9-16).ACM.

11/30/16

9

UsesforFitts’Law

• Menuitemsize• Iconsize• Scrollbartargetsizeandplacement• Up/downscrollarrowstogetherorattopandbottomofscrollbar

18

CognitiveWalkthrough

• HowcomparetoFitts’Law?• Stillpredictinginteraction• Analystactuallyusessystemhim/herselftoidentifyissues:InspectionMethod

• Detailedreviewoflikelyuserinteractionswithsystem• Focusofevaluation?• easeoflearning• newusersaccomplishingtasks

• Startwith1. Prototypeordetailedsystemspecification2. (representative)taskdescriptions&scenarios3. Actionsneededtocompletetasks4. Descriptionofusers(theknowledge,experienceetc.thatevaluatorscan

assume)

11/30/16

10

CognitiveWalkthrough

• Steps1. Designerpresentsanaspectofthedesign&usagescenarios.2. Expertisbriefed• assumptionsaboutuserpopulation,contextofuse,taskdetails

3. 1+expertswalkthroughthedesignwiththescenarios,tasks,&actionlists• Guidedbysetofquestions

CognitiveWalkthrough

• Foreachaction,stepthroughand“trytotellabelievablestory”about:

• Willuserssee actionisavailable?• Willusersknow theactionisonetheyneed?• Ifactiontaken,willuserassociateandinterprettheresponsefromtheactioncorrectly?• Doeffectsofactionsmatchgoals?

• Noteanyproblemsthatarise

11/30/16

11

UsabilityHeuristics

• Generalprinciples,rulesofthumb• Manytochoosefrom

• Neilsen’s 10principles• Shneiderman’s 8goldenrules• Norman’srulesfromDesignofEverydayThings• Mac,Windows,Android,Java,etc.guidelines

• Helpdesignerschoosedesignalternatives

• Helpevaluatorsfindproblemsininterfaces(“heuristicevaluation”)

HeuristicEvaluation

• Moreholisticthancognitivewalkthrough,whichistask-specific

• JacobNielsen:early1990s.• Heuristicsdistilledfromanempiricalanalysisof249usabilityproblems• “Systematicinspectionofauserinterfaceforusability”Nielsen’93

• Byexperts

• Discounttechnique• Cheap(donew/HCIteam)• Canuseearly• Flexible(throughoutdesignprocess)

• ? evaluatorsfind75%ofproblems• 5• Asingleevaluatormissesmostproblems!

11/30/16

12

NielsenExperiments

HeuristicEvaluation

• OftenusesNeilson’stenheuristics• Newheuristicsdeveloped

• mobiledevices,wearables,virtualworlds,etc.

• Heuristicevaluatorexplicitlydocumentsusabilityissues• Notesviolations

• Written• Vocalize+Observer

11/30/16

13

HeuristicEvaluation

• Distinctfromtraditionalusertesting• Observercananswerevaluators’questionsabout

• Domain• Helpsthembetterassessusability

• howinterfaceworks• Butaftertheyhavetriedtounderstanditthemselves &commentedonusabilityissue

• Evaluatorexplicitlydocumentsusabilityproblems• Vs.observerinferringproblemsfromuserexperiments

• Goal:expertopinion ofhowusersmightreceivethesystem

HeuristicEvaluation:3Stages

•Briefingsessiontotellexpertswhattodo

•Evaluationperiodof1-2hoursinwhich:• Eachexpertworksseparately;• Takeonepasstogetafeelfortheproduct;• Takeasecondpasstofocusonspecificfeatures.

•Debriefingsessioninwhichexpertsworktogethertoprioritizeproblems.

11/30/16

14

HeuristicEvaluation

•Advantages• Fewethical&practicalissuestoconsiderbecauseusersnotinvolved.• Canbequickerthanauserstudy• Actionableresults

•Challenges?• Canbedifficult&expensivetofindexperts,especially“doubleexperts”(HCI+domain)• Importantproblemsmaygetmissed• Manytrivialproblemsareoftenidentified• Expertshavebiases

HeuristicEvaluation

• Identify&useheuristics• Assessseverityofproblems

• 1:Cosmeticproblem(onlyfixifextratime)• 2:Minorproblem(low-priorityfix)• 3:Majorproblem(importanttofix,high-priority)• 4:Catastrophe(mustfix)

11/30/16

15

HeuristicEvaluation

• Howassesstheseverity?• Answerthefollowingquestions:

• Howcommonisproblem• Doesthisissuehappeninmultipleaspectsofthedesign?

• Willproblempersist• Willuserskeeprunningintothisissue?

• Howeasyforusertoovercome• Isitabarriertothemdoingwhattheyneedtodo?

• Howseriouslywillproblembeperceived?• Asmallannoyanceormajordisturbance?

Schneiderman’s 8GoldenRules1.StriveforConsistency

• PrincipleofLeastSurprise• Similarthingsshouldlookandactsimilar• Differentthingsshouldlookdifferent

• Size,location,color,terminology,prompts,ordering,…

• KindsofConsistency• Internal• External• Metaphorical

https://www.cs.umd.edu/users/ben/goldenrules.html

11/30/16

16

Consistency internal (drop down arrow to reveal more options)

& external (similar text formatting functionality in PPT and Word)

Consistency violated“Shapes” in different places

11/30/16

17

Schneiderman’s 8GoldenRules2.CatertoUniversalUsability• Supportdiverseusers• Novice– Expert• Age,Disabilities,etc.

• Help&Documentationfornovices

• Provideeasily-learnedshortcutsforfrequentoperations• Speedinteractions• Keyboardaccelerators• Commandabbreviations• Shortcuts

windows.microsoft.com/en-us/windows7/using-your-keyboard

Schneiderman’s 8GoldenRules3.OfferInformativeFeedback• Feedbackforallactions

• Keepuserinformedofsystemstate• Cursorchange• Selectionhighlight• Statusbar

• Responsetime• <0.1s:seemsinstantaneous

• 0.1-1s:usernotices,butnofeedbackneeded

• 1-5s:displaybusycursor

• >1-5s:displayprogressbar

designingwebinterfaces.com/6-tips-for-a-great-flex-ux-part-5

11/30/16

18

Schneiderman’s 8GoldenRules4.DesignDialogstoYieldClosure

• Actionsequences• Beginning• Middle• End• E.g.,e-commerce

• Shopping,checkout,confirmation

• Informativefeedbackuponcompletionofasetofactions• Provides• afeelingofrelief• indicationthatusercanprepareforthenextgroupofactions

11/30/16

19

11/30/16

20

designingwebinterfaces.com/6-tips-for-a-great-flex-ux-part-5

Schneiderman’s 8GoldenRules5.PreventErrors

Schneiderman’s 8GoldenRules5.PreventErrors

• Selectionislesserror-pronethantyping

• Disableillegalcommands

• DescriptionError• whentwoactionsaretoosimilar• e.g.,similarlookingbuttons• differentthingsshouldlookandactdifferent

• ModeError• Limituseofmodes• Visibilityofmode• Spring-loadedortemporarymodes

11/30/16

21

Schneiderman’s 8GoldenRules6.PermitEasyReversalofActions

Schneiderman’s 8GoldenRules6.PermitEasyReversalofActions

•Beprecise;restateuser’sinput• Not“Cannotopenfile”,but“Cannotopenfilenamedpaper.doc”

•Giveconstructivehelp•whyerroroccurredandhowtofixit

•Bepoliteandnon-blaming• Not“fatalerror”,not“illegal”

•Hidetechnicaldetails(stacktrace)untilrequested

11/30/16

22

Schneiderman’s 8GoldenRules7.SupportInternalLocusofControl

• Usersoftenchoosesystemfunctionsbymistake• needaclearlymarked"emergencyexit"• Supportquickexitswithoutextendeddialogue

• Provideundo• Longoperationsshouldbecancelable• Alldialogsshouldhaveacancelbutton• Userpreemptive• Easyaccesstoinfo,shortcuts,etc.

Schneiderman’s 8GoldenRules8.Reduceshort-termmemoryload

• Minimizewhatuserisrequiredtorememberbetween• Screens• Usesessions

• Includetask-relevantinformationandfeaturesinasinglescreen,wherepossible

• Makeobjects,actions,andoptionsvisibleoreasilyretrievable

11/30/16

23

And,oneofNielsen’sHeuristics*AestheticsandMinimalistDesign

• LessisMore•Visibility,reducenoise•Omitextraneousinfo,graphics,features

Lab:HeuristicEvaluationDueatconclusionofclasstoday

11/30/16

24

HeuristicEvaluation

• Evaluate1 prototype(assigned)• Notanonymous,mayaskforclarification• Butattempttofigureoutsystemforyourself

• Asateam• Introduceyoursystemtoyourevaluators• Persona(especiallygoals&attributes)• DesignRequirements• Features

• Takenotes

HeuristicEvaluation

• UseSchneiderman’s 8GoldenRules• Makeanumberedlistofusabilityproblemsandsuccessesyoufind

• Foreachproblemandsuccess:• describetheproblemorpositivefeature• identifytherelevantusabilityheuristics

• Discussviolationorconformance• Estimateseverity

• Cosmetic,Minor,Major,Catastrophe

11/30/16

25

HeuristicEvaluation

• Youmayuseyournotesandanycoursereadingstoassistyouinyourevaluation.

• Recommendsolutionsfortheproblems

• Bethorough• Atleast10usefulcomments(positiveornegative)abouttheinterfacethatyouevaluate

HeuristicEvaluation

• Mustbereadable&easytounderstand• Don'tburytheproblemsyoufoundinreamsoffree-flowingprose:beconcise,neat,organized

• Wherepossible,includescreenshotstoillustrateyourpoints

11/30/16

26

HeuristicEvaluation

• Attheendofclass,emailaPDFcopyofyourreportto• theappropriateteammembersand• CC'Prof.Parker&Farnaz

Team URL Evaluated By

1 http://homepro.com.s3-website-us-east-1.amazonaws.com/index.html Team2

2 https://cs5340team2-millayryan.rhcloud.com/ Team1

3 Nativeapp(provided inclass) Team7

4 http://ishashah112.com/HCI-Project/www/index.html#/login Team5

5 http://projecthci.s3-website-us-east-1.amazonaws.com/index.html Team6

6 https://github.ccs.neu.edu/pages/nitins51/housing-hub/ Team4

7 https://hci-findhome.herokuapp.com/project/#/ Team3

11/30/16

27

Debriefs

• Problemsidentified• Potentialsolutions