Eye Tracking the User Experience of
Mobile: What You Need to Know
Jon Strohl, David Hawkins
October 17, 2014
User Focus| Washington, D.C.
@forsmarshgroup
2
• Why mobile is important
• Advantages and limitations of eye
tracking
Agenda
• Case study
• How to eye track the UX of mobile
@jonstrohl @uxhawk #userfocus2014
3 http://www.pewinternet.org/three-technology-revolutions/
http://www.pewinternet.org/fact-sheets/mobile-technology-fact-sheet/
Technology Revolutions
% o
f A
me
rica
n a
du
lts
90% of American adults have a cell phone
58% have a smartphone
42% have an tablet
32% have an e-reader
@jonstrohl @uxhawk #userfocus2014
4 http://www.pewinternet.org/2013/09/19/cell-phone-activities-2013/
% o
f A
me
rica
n a
du
lts
% o
f A
me
rica
n a
du
lts
Cell Phone Activities @jonstrohl @uxhawk #userfocus2014
Traditional UX research is
good at explaining what
people say and do, not
what they think and feel.
Why should we use eye tracking?
@jonstrohl @uxhawk #userfocus2014
6
UX Data
SATISFACTION
& DIFFICULTY
RATINGS
MODERATOR
FOLLOW-UP
VERBAL
RESPONSES
REAL-TIME
+/- DIAL
TIME
ON PAGE/
TASK
REACTION
TIME
SELECTION/
CLICK
BEHAVIOR
SUCCESS/
FAIL
RATE
CONVERSION
RATE
ELECTRODERMAL
ACTIVITY
(EDA)
FACIAL
EXPRESSION
ANALYSIS
EYE
TRACKING
OBSERVATIONAL SUBJECTIVE
IMPLICIT BEHAVIORAL
ANALYSIS
PUPIL
DILATATION
@jonstrohl @uxhawk #userfocus2014
7 *Gazeplot of four participants when provided the task: “Let’s say you want to compare
Eye-Tracking Limitations
• A tool, not a methodology
• Task dependent
• Does not necessarily add
value for all research
questions
• Time consuming
• Higher costs
@jonstrohl @uxhawk #userfocus2014
8
• Versatile tool
• Informs us about:
– Engagement
– Difficulty
– Findability
– Processing order
– Comprehension
Eye Tracking Advantages
• Great proxy measure for how
attention is allocated
@jonstrohl @uxhawk #userfocus2014
9
*Heatmap generated by using the number of fixations that occurred in the first thirty seconds after providing the task: “Let’s say
you want to enter a TV show that you just watched into the app. Please enter a TV show you recently watched.”
Eye tracking paper Compare eye movements
across devices
Tablet App
Homepage Tablet Web Homepage Android App Homepage
He, Siu, Strohl, & Chaparro (2014). Mobile. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design.
Morgan Kaufmann.
@jonstrohl @uxhawk #userfocus2014
10 He, Siu, Strohl, & Chaparro (2014). Mobile. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design.
Morgan Kaufmann.
Users read what they need to read
*Gazeplot of fixations that occurred when participant was on the username entry page while completing the task: “Let’s say you
want to enter a TV show that you just watched into the app. Please enter a TV show you recently watched.”
@jonstrohl @uxhawk #userfocus2014
Issue: Vague error messages increase visual workload.
11
UX Best Practice: Specifically explain the error, and place the messages near where the error
occurs so the user can quickly fix the error and move on.
• “How do I advance to the
next screen?”
• “It seems like it's stuck on
the screen.”
M
Gaze Plot: After getting an
error message, the
participant had to search all
over the screen to find the
missing field.
Gazeplot of fixations that occurred after the participant clicked “Ok.”
Report with other data sources @jonstrohl @uxhawk #userfocus2014
12
Modern eye tracking
How to set up a mobile eye tracking study
13
Do I need eye tracking?
Think about measures
Select your eye tracker Calibrate
Collect Data Cleanse &
Prepare Data
Visualize
Qual
Analyze
Quant
Eye Tracking Process @jonstrohl @uxhawk #userfocus2014
14
*Fixation count heatmap during the task: “Let’s say you are interested in comparing the costs of several colleges. Please compare the cost of schools that you are interested in attending.”
*Bojko, Aga (2013). Eye Tracking the User Experience: A Practical Guide to Research. Rosenfeld Media.
Measures of attraction
Area noticeability measures
Percentage of participants
who fixated on an AOI
Number of fixations prior to first fixation
on an AOI
Time to first fixation on an
AOI
Area interest measures
Number of fixations on an
AOI
Total dwell time on an AOI
Percentage of time on an AOI
Emotional arousal
measures
Pupil diameter
@jonstrohl @uxhawk #userfocus2014
15 *Bojko, Aga (2013). Eye Tracking the User Experience: A Practical Guide to Research. Rosenfeld Media.
Measures of performance
Target findability measures
Percentage of participants
who fixated on the target
Number of fixations prior to first fixation on
the target
Time to first fixation on the
target
Target recognizability
measures
Number of gaze visits to the
target prior to target selection
Time from first fixation on
target to target selection
Cognitive processing measures
Average fixation duration
Mental workload measures
Pupil diameter
@jonstrohl @uxhawk #userfocus2014
16
Do I need eye tracking?
Think about measures
Select your eye tracker
Calibrate System
Collect Data Cleanse &
Prepare Data
Visualize
Qual
Analyze
Quant
Eye Tracking Process @jonstrohl @uxhawk #userfocus2014
17
Devices to Test
18
Choosing an eye tracker @jonstrohl @uxhawk #userfocus2014
19
Do I need eye tracking?
Think about measures
Select your eye tracker
Calibrate System
Collect Data Cleanse &
Prepare Data
Visualize
Qual
Analyze
Quant
Eye Tracking Process @jonstrohl @uxhawk #userfocus2014
20
Calibrate the system @jonstrohl @uxhawk #userfocus2014
21
Do I need eye tracking?
Think about measures
Select your eye tracker
Calibrate System
Collect Data Cleanse &
Prepare Data
Visualize
Qual
Analyze
Quant
Eye Tracking Process @jonstrohl @uxhawk #userfocus2014
22
Prepare data @jonstrohl @uxhawk #userfocus2014
23
Cleanse data
Original data Final data
Start with original data that was recorded
Outliers Capture
rate
issues
Offsets
Check for outliers in the data
Check for capture rate issues
Check for offsets
Data is ready to be analyzed
@jonstrohl @uxhawk #userfocus2014
24
Do I need eye tracking?
Think about measures
Select your eye tracker
Calibrate System
Collect Data Cleanse &
Prepare Data
Visualize
Qual
Analyze
Quant
Eye Tracking Process @jonstrohl @uxhawk #userfocus2014
25
Create gaze plots @jonstrohl @uxhawk #userfocus2014
26
Create heat maps @jonstrohl @uxhawk #userfocus2014
27
Do I need eye tracking?
Think about measures
Select your eye tracker
Calibrate System
Collect Data Cleanse &
Prepare Data
Visualize
Qual
Analyze
Quant
Eye Tracking Process @jonstrohl @uxhawk #userfocus2014
28
Create Areas of Interest (AOIs)
@jonstrohl @uxhawk #userfocus2014
29
Pros Cons
• Versatile
• Informs us about:
― Engagement
― Difficulty
― Findability
― Processing order
― Comprehension
• Time consuming
@jonstrohl @uxhawk #userfocus2014
• Capture more of the UX
• Higher cost
• Mobile device stand
changes the natural
environment
• Won’t answer all
research questions
Thank you!
• Twitter: @forsmarshgroup
• LinkedIn: http://www.linkedin.com/company/fors-marsh-group
• Blog: www.forsmarshgroup.com/index.php/blog
Jonathan Strohl
@jonstrohl
User Focus 2014 | Washington, DC