survey
TRANSCRIPT
Designing a SurveyDesigning a SurveyEDUU600EDUU600
Adapted from McMillan & SchumacherAdapted from McMillan & Schumacher
Make items clearMake items clear Avoid double-barreled questions (avoid Avoid double-barreled questions (avoid
and)and) Respondents must be competent to Respondents must be competent to
answer (and provide reliable information)answer (and provide reliable information) Questions should be relevantQuestions should be relevant Short, simple items are bestShort, simple items are best Avoid negative itemsAvoid negative items Avoid biased items or termsAvoid biased items or terms
Writing Questions and Writing Questions and StatementsStatements
Types of ItemsTypes of Items
Closed Form – Structured response where Closed Form – Structured response where subject chooses between predetermined subject chooses between predetermined responsesresponses
Open Form – subject writes in any responseOpen Form – subject writes in any response Scaled items Scaled items
– Gradations, levels, or valuesGradations, levels, or values– Likert scaleLikert scale– Semantic differential scaleSemantic differential scale– Ranked itemsRanked items– Checklist itemsChecklist items
Data Collection TechniquesData Collection TechniquesPros and ConsPros and Cons
Paper/pencilPaper/pencil– Economical/standardEconomical/standard– Norms inappropriateNorms inappropriate– Must be able to readMust be able to read
Alternative AssessmentAlternative Assessment– Holistic/authenticHolistic/authentic– Subjective ratingSubjective rating– Costly/time-consumingCostly/time-consuming
Questionnaire or SurveyQuestionnaire or Survey– Economical/easy to scoreEconomical/easy to score– Response rate/inability to probe and clarifyResponse rate/inability to probe and clarify– Biased/ambiguous itemsBiased/ambiguous items
Interview and ObservationInterview and Observation
InterviewInterview– Flexible/include nonverbal responsesFlexible/include nonverbal responses– Costly/time consumingCostly/time consuming– Can be anonymousCan be anonymous– Effect of interviewer and interviewer biasEffect of interviewer and interviewer bias
ObservationObservation– Captures natural behaviorCaptures natural behavior– Costly/time consumingCostly/time consuming– Observer biasObserver bias– Not anonymousNot anonymous
Designing a QuestionnaireDesigning a QuestionnaireStrongly Strongly AgreeAgree
AgreeAgree NeutralNeutral DisagreeDisagree Strongly Strongly DisagreeDisagree
AlwaysAlways Most of the Most of the TimeTime
SometimesSometimes RarelyRarely NeverNever
Very happyVery happy Somewhat Somewhat happyhappy
Neither sad Neither sad or happyor happy
Somewhat Somewhat sadsad
Very SadVery Sad
Use a Likert Scale, (or combination of Likert, ranking and differential)
Like Dislike
Important Unimportant
Smileys for KidsSmileys for Kids
Questions must be readQuestions must be read Child can respond by circling correct Child can respond by circling correct
responseresponse May respond verbally or by pointingMay respond verbally or by pointing
Open-ended QuestionsOpen-ended Questions Questions to which there is not one definite Questions to which there is not one definite
answer. answer. May be a good way to break the ice with a surveyMay be a good way to break the ice with a survey Gives respondents an opportunity to answer in Gives respondents an opportunity to answer in
their own words. their own words. Example: "Are there any other comments about Example: "Are there any other comments about
the course you would like to add?" the course you would like to add?" Responses to open-ended questions can be very Responses to open-ended questions can be very
useful, often yielding quotable material. useful, often yielding quotable material. Drawback is that the responses are more difficult Drawback is that the responses are more difficult
to catalogue and interpret to catalogue and interpret
From types of survey questions - http://coe.sdsu.edu/eet/Articles/surveyquest/index.htm
Closed-ended QuestionsClosed-ended Questions Have a finite set of answers from which the Have a finite set of answers from which the
respondent chooses. respondent chooses. One of the choices may be "Other" but need to One of the choices may be "Other" but need to
provide a write-in responseprovide a write-in response Easy to standardizeEasy to standardize Data gathered from closed-ended questions lend Data gathered from closed-ended questions lend
themselves to statistical analysis themselves to statistical analysis More difficult to write than open-ended questionsMore difficult to write than open-ended questions Design choices must include allDesign choices must include all the possible the possible
answers a respondent could give for each answers a respondent could give for each questionquestion
From types of survey questions - http://coe.sdsu.edu/eet/Articles/surveyquest/index.htm
Likert ScaleLikert Scale
When you want to know respondents' When you want to know respondents' feelings or attitudes about somethingfeelings or attitudes about something
Respondents must indicate how closely Respondents must indicate how closely their feelings match the question or their feelings match the question or statement on a rating scale. statement on a rating scale.
Number at one end of the scale represents Number at one end of the scale represents least agreement, or "Strongly Disagree”least agreement, or "Strongly Disagree”
Number at the other end of the scale Number at the other end of the scale represents most agreement, or "Strongly represents most agreement, or "Strongly Agree" Agree"
From types of survey questions - http://coe.sdsu.edu/eet/Articles/surveyquest/index.htm
Multiple ChoiceMultiple Choice
When you want respondents to pick When you want respondents to pick the best answer or answers from the best answer or answers from among all the possible options,among all the possible options,
Multiple-choice questions are easy to Multiple-choice questions are easy to lay out on a written survey. lay out on a written survey.
Include specific directions about how Include specific directions about how many answers to select directly after many answers to select directly after the question. the question.
From types of survey questions - http://coe.sdsu.edu/eet/Articles/surveyquest/index.htm
Other Closed QuestionsOther Closed Questions
Ordinal: When you need all possible Ordinal: When you need all possible answers to be rank orderedanswers to be rank ordered
Categorical: When the possible Categorical: When the possible answers for a question are categoriesanswers for a question are categories– Each respondent must "belong" in Each respondent must "belong" in
exactly one of themexactly one of them Numerical: When the answer must be Numerical: When the answer must be
a real numbera real number
From types of survey questions - http://coe.sdsu.edu/eet/Articles/surveyquest/index.htm
When to Use What Type Question
Type of question... Best Used for...
Open-endedBreaking the ice in an interview; when respondents' own
words are important; when the surveyor doesn't know all the possible answers.
Closed-endedCollecting rank ordered data; when all response choices
are known; when quantitative statistical results are desired.
Likert-scale To assess a person's feelings about something.
Multiple-choiceWhen there are a finite number of options (remember to
instruct respondents as to the number of answers to select).
Ordinal To rate things in relation to other things.
CategoricalWhen the answers are categories, and each respondent
must fall into exactly one of them.
Numerical For real numbers, like age, number of months, etc.
http://coe.sdsu.edu/eet/Articles/surveyquest/index.htm
InfoPoll Survey TipsInfoPoll Survey Tips
Write a short questionnaire Write a short questionnaire Use simple words Use simple words Relax your grammar Relax your grammar Assure a common understanding Assure a common understanding Start with interesting questions Start with interesting questions Don't write leading question Don't write leading question Avoid double negatives Avoid double negatives Balance rating scales Balance rating scales
http://www.accesscable.net/~infopoll/tips.htm
InfoPoll Survey TipsInfoPoll Survey Tips
Don't make the list of choices too long Don't make the list of choices too long Avoid difficult concepts Avoid difficult concepts Avoid difficult recall questions Avoid difficult recall questions Use Closed-ended questions rather than Use Closed-ended questions rather than
Open-ended ones Open-ended ones Put your questions in a logical order Put your questions in a logical order Pre-test your survey Pre-test your survey Name your survey to pique interest Name your survey to pique interest Include cover memo or introduction Include cover memo or introduction
http://www.accesscable.net/~infopoll/tips.htm
Online Survey EnginesOnline Survey Engines Survey Monkey - (Basic Account Free) Survey Monkey - (Basic Account Free)
http://www.surveymonkey.com Survey Suite (14 days free) - Survey Suite (14 days free) -
http://intercom.virginia.edu/cgi-bin/cgiwrap/intercom/SurveySuite/ss_index.pl
Other Surveys - Other Surveys - http://www.surveymonkey.com/Pricing.asp
Zoomerang - Zoomerang - http://info.zoomerang.com/ Advanced Survey - Advanced Survey -
http://www.advancedsurvey.com/ Online Survey Builder - Online Survey Builder -
http://freeonlinesurveys.com/ Create Survey - Create Survey - http://www.createsurvey.com/
ResourcesResources Types of Survey Questions - Types of Survey Questions -
http://coe.sdsu.edu/eet/Articles/surveyquest/index.htm Writing questions - Writing questions -
http://www.ryerson.ca/~mjoppe/ResearchProcess/WriteBetterQuesthttp://www.ryerson.ca/~mjoppe/ResearchProcess/WriteBetterQuestion.htmion.htm
U of Texas - Survey Questions - U of Texas - Survey Questions - http://www.utexas.edu/academic/diia/assessment/iar/how_to/methhttp://www.utexas.edu/academic/diia/assessment/iar/how_to/methods/survey.phpods/survey.php
How to write a good survey - How to write a good survey - http://www.accesscable.net/~infopoll/tips.htmhttp://www.accesscable.net/~infopoll/tips.htm
Super Survey - http://knowledge-base.supersurvey.com/survey-Super Survey - http://knowledge-base.supersurvey.com/survey-questions.htmquestions.htm
Writing Effective Survey Questions - Writing Effective Survey Questions - http://www.custominsight.com/articles/effective-survey-http://www.custominsight.com/articles/effective-survey-questions.aspquestions.asp
Designing Surveys that Count - Designing Surveys that Count - http://www.keene.edu/crc/forms/designingsurveysthatcount.pdf http://www.keene.edu/crc/forms/designingsurveysthatcount.pdf