so much ux data! now what?
DESCRIPTION
workshop for UXPA DC on April 12, 2014, entitled "All this UX data! Now what?" Attendees learned how to deal with large amounts of user experience data from tests, and how to combine certain data to tell a succinct story.TRANSCRIPT
So much UX data! Now what?
Jennifer Romano BergstromApril 12, 2014
UXPA-DC Workshop| Arlington, VA@romanocog
2
Measuring the UX
• How does it work for the end user?
• What does the user expect?
• How does it make the user feel?
• What is the user’s story and habits?
• What are the user’s needs?
Usability = “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” ISO 9241-11
+ emotions and perceptions = UX
@romanocog @uxpadc
3
Usability vs. User Experience (UX)?
The 5 Es to Understanding Users (W. Quesenbery): http://www.wqusability.com/articles/getting-started.htmlUser Experience Design (P. Morville): http://semanticstudios.com/publications/semantics/000029.php
Whitney’s 5 Es of Usability
Peter’s User Experience Honeycomb
@romanocog @uxpadc
4Krug, S. Don’t Make Me Think
What People do on the Web @romanocog @uxpadc
5
UX data
OBSERVATIONAL+ Ethnography
+ Time to complete task+ Reaction time
+ Selection/click behavior+ Ability to complete
tasks+ Accuracy
IMPLICIT+ Facial expression analysis
+ Eye tracking+ Electrodermal activity (EDA)
+ Behavioral analysis+ Linguistic analysis of
verbalizations+ Implicit associations
+ Pupil dilation
EXPLICIT+ Post-task satisfaction
questionnaires+ In-session difficulty
ratings+ Verbal responses
+ Moderator follow up+ Real-time +/- dial
@romanocog @uxpadc
6
Explicit data
*Satisfaction Questionnaire Q7: Please rate how difficult it was to log in on this device. 1= not difficult at all to 5= extremely difficult.
Participant Ratings0
3
6
9
How likely would you be to recommend this site to a friend?
Not likely at all or Slightly likely
Moderately Likely
Very Likely
Participant Ratings0
3
6
9
How likely would you be to use this site in the future?
Not likely at all or Slightly likely
Moderately Likely
Very Likely
“Love the picture in the middle of it.”“It looks very clean and very simple.”
“It looks pretty organized, it's a nice design.”
When asked how they would save information, four of six participants said they would bookmark the page or take a screenshot of the information. Only two mentioned that they would use the site functionality to save for later use.
83%
9%9%
Percentage of Difficulty Rat-ings*
1 & 2
3
>=4
@romanocog @uxpadc
7
Observational data
Details page about Military Physician Assistant career, only accessible through Military section.
Details page about Civilian Physician Assistant career.
Evidence: Six participants clicked on Salary & Wages when looking for states with the most jobs before looking under Job Growth. • Five participants thought they had
completed the task once they found the Salary & Wages map on the wrong page.
Time on Page0
5
10
15Average time spent on landing page
Seco
nds
@romanocog @uxpadc
88
Implicit data
8
00:00.00 00:03.88 00:07.75 00:11.63 00:15.50 00:19.38 00:23.25 00:27.13 00:31.00 00:34.88 00:38.75 00:42.63 00:46.50 00:50.38 00:54.256
6.57
7.58
8.59
P29 iPhone
Begin date and time selection.
End date and time selection.
Trouble with scrolling.
More trouble with scrolling.
@romanocog @uxpadc
99
Implicit data
9
00:00.00 00:03.88 00:07.75 00:11.63 00:15.50 00:19.38 00:23.25 00:27.13 00:31.00 00:34.88 00:38.75 00:42.63 00:46.50 00:50.38 00:54.256
6.57
7.58
8.59
P29 iPhone
Begin date and time selection.
End date and time selection.
Trouble with scrolling.
More trouble with scrolling.
@romanocog @uxpadc
10
Eye tracking @romanocog @uxpadc
11
Heat maps @romanocog @uxpadc
12
@romanocog @uxpadcGaze plots
13
@romanocog @uxpadcGaze plots
14
@romanocog @uxpadcGaze plots
15
Gaze plots and commentsM
“Man, this is a long paragraph.”
“There's a lot of information, it'd be a lot better in list form. Ideally, you want to get your information quick without reading through all this.”
@romanocog @uxpadc
16
Usability test of a low-fi prototype
@forsmarshgroup @romanocog
@romanocog @uxpadc
• video
17
Usability test of a high-fi prototype
@forsmarshgroup @romanocog
@romanocog @uxpadc
• video
18
More data
Fixation count Heat map of the Round 2 home page from 7 participants during the “scam” task
Left navigation
Center icons
Mean time elapsed before AOI is fixated
79.55 (18.54). N = 8
29.36 (9.2). N = 6
10
3
6
9 “How clear is the information on this page?”**
# o
f Part
icip
ants
Not at all to Slightly Clear
Moderately Clear
Very to Extremely Clear
@romanocog @uxpadc
19
• “Where and how you click is a bit counter-intuitive. [It’s] not super obvious which button to click to get to next sections.”
• “I feel like the ‘Next’ should be at the bottom and not the top.”
Intuitive ‘Next’ button location
Non-intuitive ‘Next’ button location
Usability test across devices @romanocog @uxpadc
20
Combining data
• “I’m not expecting them to email or call me. I don’t expect any person to notify me.”
• “I’m not sure when I’d get an answer. It’s not like Yahoo! Answers where it’s immediate.”
Participants had different expectations about what would happen next. • Five expected to hear
back via email• One expected an
immediate response• One expected to see the
answer posted somewhere on the site
• Two said they would call for assistance.
@romanocog @uxpadc
video
So how much data do I need?
22
Considerations
• Qualitative• Quantitative• Iterative testing• User groups
@romanocog @uxpadc
Questions
Jennifer Romano Bergstrom@romanocog
UXPA-DC Workshop| Arlington, VA