asking users and experts bobby kotzev adrian sugandhi
Post on 21-Dec-2015
214 views
TRANSCRIPT
Interviews, Survey, or Focus Group?
Interview Take significant time and need more resources
Survey Large data set
Focus group Seeking multiple point of views in a shorter period
of time
Preparing for Interviews:Things to be aware
Be alert to unconscious biases Honesty
Social desirability Prestige response bias Sensitive or highly personal topics – survey?
Consider the interviewee as he/she is “Doing you a favor”
Preparing Interviews:Outcome Analyses
Outcome Analyses Plan outcome-based interviews
Associate questions with the goals Capture desired outcomes
Differentiate between outcome and solution Organize the outcomes Rate the outcomes for importance and
satisfactions Importance + (Importance-Satisfaction) = Opportunity
Use the outcomes to jump-start innovation
Preparing Interviews:Pre-interview Identify the objective of the study Select type of interviews Medium of interview (in person/phone) Decide how you will analyze the data Write the questions
Avoid long questions (Brevity) Avoid compound sentences Avoid using jargon Avoid leading questions Avoid Biases Clarity Avoid future prediction questions Avoid inaccessible topics
Test your questions
Preparing Interviews:Players in the Activity
Participants 6-10 of each user type (diversity)
Interviewers Ensure participants understand the questions Must be skilled
The note-taker Interviewers can focus more on body language
The videographer Whenever possible, record the interview session
Conducting Interviews: Sections
Introduction 5-10 Minutes
Warm-up 5-10 Minutes
Main section 85-100 Minutes
Cool off period 5 Minutes
Closing
Preparing for Interviews: Additional Preparations
Run pilot study Be professional
Dress similarly and appropriately Prepare informed consent Check and familiarize yourself with recording
equipment Record answers exactly
Unstructured Interviews
A conversation that focus on a particular topic Open questions Can generate rich data which the interviewer
haven’t thought about Generates a lot of unstructured data Impossible to replicate Hard to analyze
Unstructured Interviews: Prepare
Have an agenda Be prepared to follow new lines Pay attention to ethical details Respond with sympathy but make sure to
avoid putting ideas in the user’s head Analyze data as soon as possible after the
interview
Semi-Structured Interviews
Starts as a structured interview but can change direction and inject open-ended questions depending on responses
Avoid preempting answers Probes – “Do you want to tell me anything
else about…”
Interviewer’s Role Do not interrupt Keep on track
Unstructured interview easy goes off-track Silence is golden
“permission” to provide more detail Remain attentive Asking the tough questions
Wait until you develop rapport Using examples Watch for generalities Do not force choices
Interviewer’s Role (cont’d)
Watch for markers Key events to probe for more rich information
Select the right type of probes Know when to move on Reflecting
Summarize, reword, or reflect responses Empathy and antagonism Transitions
Transition smoothly from one topic to another
Monitoring the Relationships with the Interviewee/Participant
Watch participant’s body language Nervous, tenses – go to easier question or restate
purpose/motivation of the study Fighting for control
Ask yourself why one refusing your questions Hold your opinions Dos and Don’ts
Interview: Data Analysis and Interpretation
Must be analyzed shortly after each interview Categorizing Affinity diagram Qualitative analysis tools
Look for patterns
Interview: Communicate the Findings
Over time By topic By participant Vehicles for communicating the results
Summarized poster Identify follow-up activities based on the results Table of recommendations (uncovered issues and
next steps)
Questionnaires
Good, established technique for collecting demographic data and users opinion
Can be Closed and Open. Can be distributed to large number of people
Questionnaires: Designing Start with basic demographic data Make questions clear and specific When possible ask closed questions and offer a range of
answers Include no-opinion option Ordering matters Avoid complex and compound questions Use intuitive, consistent scaling Avoid jargons Provide clear instructions how to complete the questioner
Questionnaires: Question Types
Checkboxes Likert Scales
Semantic differential scales
Gender: MaleFemale
Questionnaires: Administering
Well designed Include stamped self addressed envelope Provide a short version Contact and respondents trough a follow-up,
mail, e-mail phone Offer incentives like payments
Questionnaires: Online Forms Pros:
Reach large audience Response is fast No postage costs Data is already in a electronic format and can be imported
easily in DB Errors can be corrected easily and fast
Cons: Obtaining random sample is difficult – web users represent
only a certain demographic. Participants are self selecting – proclaimed nonscientific
Questionnaires: Online Forms (cont’)
Construct Online questioners based on their paper forms
Run pilot studies
Asking Experts
Experts will be power users but also could be involved with the development of similar, successful products so they have an deeper understanding.
Asking Experts: Inspections
Heuristic evaluations evaluate user-interface against a predetermined guidelines
and principles
Visibility of system statusAre users kept informed about what is going on?Is appropriate feedback provided within reasonable time about a user action?
User Control and freedom Are there ways of allowing users to easily escape from places they unexpectedly find themselves in?
Consistency and standards Are the ways of performing similar actions consistent?
Asking Experts: Inspections (cont’)
Different heuristics are needed for different products, devices and software
Using heuristics and experts can reveal issues fast and inexpensively – 5 experts can reveal 75% of issues - graph
Performing heuristic evaluation Spend 1-2 hours with the product. At least two passes – one to get the feel, second to walk
trough the interface and address all aspects of usability. Have a specific task in mind when evaluating functional
products
Asking Experts: Inspections (cont’)
Heuristic evaluation of websites Internal consistency Simple dialog Shortcuts Minimizing the user memory load Preventing errors Feedback Internal locus of control Layout Internal consistency
Asking Experts: Problems
Reporting issues which are not there – (Bill Bailey 2001)
Experts Problem Reporting
34%
22%
44% problems discovered
problems missed
reported non-problems
Asking Experts: Walkthroughs
Walk through a task and to notice problems “Cognitive walkthroughs” – simulating the user
problem solving process for a task Figure if the users will know what to do, how to do it and
weather the action was correct or not. Record assumptions, side issues and summaries of results
Pluralistic walkthroughs Users developers and usability experts get together to step
through a scenario