![Page 1: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/1.jpg)
Interviews and Questionnaires
a.k.a. How to talk to your users
![Page 2: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/2.jpg)
Agenda
Questions Interviewing techniques Questionnaire design Evaluation Plan discussions
![Page 3: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/3.jpg)
First: a little exercise
Interview each other about instant messaging use– 4 or 5 questions, 3 minutes
Person 1: the silent type Person 2: the going off track type
![Page 4: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/4.jpg)
Today’s focus is asking people about stuff…
Interviews Questionnaires Assessment of that data
![Page 5: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/5.jpg)
Interviews & Questionnaires
Subjective view of participants Quantitative – very structured
– Questionnaires often quantitative, but not entirely
– Structured Interviews Strict set of questions, deviation would compromise
study Qualitative – less or no structure
– Semi-structured interviews Some deviation encouraged
– Unstructured interviews i.e. the ethnographic interview Little guide, very explorative
![Page 6: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/6.jpg)
Interviews
Potentially lots of detail can vary questions as needed Inexpensive Time consuming to perform and analyze Some interpretation required Subject to interviewer biases
![Page 7: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/7.jpg)
Questionnaires
Expensive to create …but cheap to administer Easier to get quantifiable results Can gather info from many more people Protects participant identity Only as good as the questions asked
![Page 8: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/8.jpg)
Structured Interviews
More similar to questionnaires Require a lot of training for any hope at inter-
interviewer reliability But that means that they tend to give much
more repeatable results
![Page 9: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/9.jpg)
Unstructured Interviews
Have a plan, but keep interview open to different directions
Get participant to open up and express themselves in their terms and at own pace
Create interpretations with users– Be sure to use their terminology
Take lots of time, but learn a lot as well
![Page 10: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/10.jpg)
Semi-Structured Interviews
Predetermine data of interest - know why you are asking questions - don’t waste time
Plan for effective question types How do you perform task x? Why do you perform task x? Under what conditions do you perform task x? What do you do before you perform…? What information do you need to…? Whom do you need to communicate with to …? What do you use to…? What happens after you…?
– See Gordon & Gill, 1992; Graesser, Lang, & Elofson, 1987
![Page 11: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/11.jpg)
Asking Questions
Understand your goals Consider the ordering of the questions Avoid complex/long/multiple questions Avoid jargon; talk in participant’s language Be careful of stereotypes, biases
![Page 12: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/12.jpg)
Clarity is important
Questions must be clear, succinct, and unambiguous
How much time have you spent reading news on the Web recently? Some A lot Every day Rarely Etc.
None
0 to 5 hours
6 to 10 hours
11 to 20 hours
More than 20 hours
![Page 13: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/13.jpg)
Avoid question bias
Leading questions unnecessarily force certain answers.Do you think parking on campus can be made
easier?
What is your overall impression of…1.Superb
2.Excellent
3.Great
4.Not so Great
![Page 14: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/14.jpg)
Be aware of connotations
Do you agree with the NFL owner’s decision to oppose the referee’s pay request?
Do you agree with the NFL owner’s decision in regards to the referee’s pay demand?
Do you agree with the NFL owner’s decision in regards to the referee’s suggested pay?
![Page 15: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/15.jpg)
Leading questions
People want to do well, give you what you are looking for
Be aware of your own expectations before creating questions and while interviewing
Use value neutral terms
What do you like about this system?
Vs.
Tell me what you thought about this system.
![Page 16: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/16.jpg)
Avoid hypotheticals
Avoid gathering information on uninformed opinions
Subjects should not be asked to consider something they’ve never thought about (or know or understand)
Would a device aimed to make cooking easier help you?
![Page 17: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/17.jpg)
Handle personal info carefully
Ask questions subjects would not mind answering honestly.– What is your age?– What is your waist size?
If subjects are uncomfortable, you will lose their trust
Ask only what you really need to know
![Page 18: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/18.jpg)
What’s wrong with this picture?
How much easier is it to use this email client than Outlook?
I see you choose to use your keyboard shortcuts more than the mouse. Is that faster for you?
Your choice of red is different than any other user we saw. Why did you do that?
![Page 19: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/19.jpg)
Planning your interview:
Introduction Warmup Main session Cool-off Closing
Record everything exactly in your participants’ languages
(don’t forget to test your recording equipment)
![Page 20: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/20.jpg)
The warmup or “grand tour” question
The first question helps set the tone for the interview
– Familiarize the participant to talking– Encourage the participant that their true opinion does
matter Question should be
– Easy to answer– But not answered easily
More than just a “yes” or “no” response Examples:
– Tell me about the work you do?– What made you buy the computer?
![Page 21: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/21.jpg)
Prompts
“Nudge” a participant in a direction, or to get additional response
– Silent: remain silent until they say more– Echo: repeat back and then ask “then what happens”
etc.– Make agreeing sounds: you say “uh huh” and the
other person continues– Tell Me More: could you tell me more about that?– Clarifying: summarize and ask for confirmation or
clarification, often leads to new discussion
![Page 22: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/22.jpg)
Contents of a survey
General/Background info– Demographic data– Also functions as a “warm up”– Correlate responses between groups
Objective questions
Open-ended/subjective
![Page 23: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/23.jpg)
Background examples
Demographic data:– Age, gender– Task expertise
i.e. Have you ever worked in a restaurant?
– Motivation– Frequency of use
How often do you…
– Education/literacy What training have you had in …?
![Page 24: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/24.jpg)
Closed Format
Advantages– Clarify alternatives– Easily quantifiable– Eliminate useless answer
Disadvantages– Must cover whole range– All should be equally likely– Don’t get interesting,
“different” reactions
Restricting set of choicesQuantifiable
![Page 25: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/25.jpg)
Many forms of response
Dichotomous Multiple Choice Multiple Response Rank/Match Likert Rating
![Page 26: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/26.jpg)
Questionnaire Styles
LaTeX
FrameMaker
WordPerfect
Word
Rank from1 - Very helpful2 - Ambivalent3 - Not helpful0 - Unused
___ Tutorial___ On-line help___ Documentation
Which word processingsystems do you use?
![Page 27: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/27.jpg)
Likert-type scale
Typical scale uses 5, 7 or 9 choices Above that is hard to discern Doing an odd number gives the neutral
choice in the middle You may not want to give a neutral option
Characters on screen were:
hard to read easy to read 1 2 3 4 5 6 7
![Page 28: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/28.jpg)
1. What is your age? _______________2. How long have you used the internet?
<1 year1-3 years3-5 years>5 years
3. How do you get information about courses?EmailWeb siteFlyersRegistration bookletAdvisorOther students
4. How useful is the Internet in getting information about courses?______________________________________________________________________________________________________________________
What’s wrong with this picture?
![Page 29: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/29.jpg)
On line questionnaires
Email or internet Change checkboxes into dropdowns, etc Take advantage of the technology –
check input Ensure its as accessible as paper
(browser and email client compatibility) Ensure confidentiality – how is this
different from paper?
![Page 30: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/30.jpg)
Free Web Survey Tools
Survey Monkey– http://www.surveymonkey.com
Survey Share– http://www.surveyshare.com/
phpESP– http://phpesp.sourceforge.net– Open Source surveys using PHP.
![Page 31: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/31.jpg)
Analyzing your quantitative data
“Code” open ended responses or interview questions to make quantitative
– Categorize all responses
Look for trends in the data– Count, average, tabulate– Make charts, etc– Run statistical analysis– Use lo-fi methods (post-its, affinity diagrams, etc)
![Page 32: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/32.jpg)
Analyzing qualitative data
Find interesting cases, responses Look for patterns of responses
– Use post-its, affinity diagrams, etc.
Look for any useful suggestions, improvements, explanations that help you improve your design
Gather illustrative quotes from users that demonstrate your conclusions
![Page 33: Interviews and Questionnaires a.k.a. How to talk to your users](https://reader035.vdocument.in/reader035/viewer/2022062308/56649dde5503460f94ad67aa/html5/thumbnails/33.jpg)
Evaluation discussion
Someone else should be able to pick up your plan and execute it.
Be as SPECIFIC as possible– What criteria are important?– What tasks EXACTLY?– What data? How will you record?– What questions will you ask?