usability testing: analyzing data

• Compile

• Summarize data

• Analyze data

• Develop recommendations

• Produce the final report

Wednesday, March 28, 12

Data Analysis


next 2 weeks:


Step 1: Compile


Get it into a pile.


Get your piles sorted.


Ideally:


compile as you test.


further test the test.


keep the data legible.


• Type handwritten notes

• Verify data captures

• Back up files


Organize raw data.


Do whatever it takes to get it done.


• lists

• tallies

• matrices

• stories

• structure models

• flow diagrams


Goal:


Step 2: summarize


1. task accuracy


number of errors made per task


• categorize errors by type

• ommission

• commission


# of participants who performed:

• successfully

• successfully but with some assistance

• assistance types, tasks, products


Vital Statistics:


% performing successfully, including those who needed assistance

• regardless of time limits

• regardless of assistance needed


If this percentage is very low, you’ve got a

serious problem.


% performing successfully

• including those who corrected errors


% performing successfully within benchmark

• indicative of efficiency


2. task timings


summarize preference data


Limited choice questions:

• Sum by choice

• Chart trends


Free-form questions:• Group into meaningful categories

• Ex: particular screen, doc section, etc.


Summarize other measures

• Unnecessary returns to main navigation

• Number/type of hints and prompts

• Times site map was accessed

• Points of hesitation (and for how long)


Summarize by Group/Version


(see pages 255 - 259)


3: analyze data


What didn’t succeed?


What were the difficulties?


What were the error sources?


Prioritize problemsby severity


Severity descriptions & definitions: p. 262


usability testing: analyzing data

Technology

data captures

data analysiswednesday

data legible

raw data

assistance assistance

assistance neededwednesday

limited choice questions

number of errors