"keeping it real” while thinking aloud. ensuring credibility of eye-tracking usability...
DESCRIPTION
The presentation concentrates on the issues of think-aloud method as well as misconceptions about eyetracking in user research and usability testing.TRANSCRIPT
„Keeping it real” while thinking aloudEnsuring credibility of eye-tracking usability studies employing think aloud method.
Tobii User meeting, 2007-04-18
Keeping it real while thinking aloud
Agenda
1. About Symetria
2. Think aloud method
3. Misconceptions about eyetracking
4. Problems with eyetracking in think aloud studies
5. Context of situations as an important analysis factor
6. Concept of checkpoints
Keeping it real while thinking aloud
About Symetria
Usability E-marketing Media buying
Usability evaluation
User-Centered Design
Eyetracking analysis (layouts, ads)
Usability consulting
Marketing communication on the Web
Web campaigns
E-mail marketing
Market analysis
Media planning
Ad production
Media buying
Monitoring
Design & planningProduction &
promotionBroadcasting
Keeping it real while thinking aloud
Think aloud as conducted by Symetria
1. Trunk test:– What is this site?– What is it for?– What is your first impression?
2. Tasks:– The more users talk, the better– User can stop and explain– No heavy time pressure – Task fulfillment levels as usability measures
Keeping it real while thinking aloud
Eyetracking misconceptions
• Eyetracking results can be obtained by simply summarizing data for each page
• Every piece of data is important
• The more data gathered, the more accurate the results
Keeping it real while thinking aloud
Eyetracking & think aloud convergence – problems
Result analysis errors
User specific problems
Speech and gaze correlation
Keeping it real while thinking aloud
Eyetracking & thinking aloud convergence – problems
Speech and gaze correlation
– Users tend to look at the elements they are describing, influencing the results
– Questions asked influence reactions – this is not always intended
Keeping it real while thinking aloud
Eyetracking & thinking aloud convergence – problems
User specific problems
– Overeager users – concentrate too much on talking
– Distractible users – forget about the task,
– Tendency to describe the page element by element
Keeping it real while thinking aloud
Eyetracking & thinking aloud convergence – problems
Result analysis errors
• Summarizing data without inspecting the context
• Mixing gaze data for different types of activities
User describes User operates page
screen actions reactions task 1 task 2 task n
Keeping it real while thinking aloud
Eyetracking misconceptions corrected
• Eyetracking is only a tool enhancing the study
• Gaze data cannot be summarized without analyzing the context
• It is not important to track the whole study, but specific moments
• Users that concentrate on talking make results unreliable
Keeping it real while thinking aloud
„Keeping it real” during the study
• Don’t push users to talk
• Ask users to describe their reactions, not their actions– The recording is enough to understand what happened– It is not important for user to know why he/she did something
• In case of overeager users, suggest concentrating on the tasks
• Plan ahead what questions to ask on pages which will be analyzed
Keeping it real while thinking aloud
Planning eyetracking analysis for thinking aloud studies
• Typical situations can be distinguished for each task
• Online store – user looks for a 2GB mp3 player:Home pageRecommended buyingsProduct descriptionProduct listAnother Product descriptionShipment infoProduct descriptionShopping cartCheckout
Keeping it real while thinking aloud
Concept of checkpoints
• An instance of user interaction with a page/type of pages that happens in a specific context defined before the study
• Example:
„Product list page” checkpoint
Type of checkpoint: task fulfillment
Purpose of interaction: user is looking for a mp3 player
Condition: User is seeing a product list for the first time
Hypothesis: user looks at text labels rather than icons
Keeping it real while thinking aloud
Types of checkpoints
• Free interaction checkpoints:– Home page– Products page
• Task fulfillment checkpoints:– Registration form– Product description
• Question-response checkpoints:– What page is currently displayed?– How would you go around contacting the site’s
owner?
Keeping it real while thinking aloud
Conclusion
• Plan situations in which eyetracking data will be comparable between the users
• Plan questions to ask
• During the study, look for situations identified before
• Don’t summarize results automatically
