designing a questionaire

Designing the QuestionnaireDesigning the Questionnaire

Questionnaire building Questionnaire building is an art!is an art!

A questionnaireis always custom-built!

The process of questionnaire designThe process of questionnaire designPLANNING WHAT TO MEASURE

Revisit the research objectives

Decide on the research issue of your questionnaire

Get additional information on the research issue from secondary data sources and exploratory research

Decide on what is to be asked under the research issue

FORMATTING THE QUESTIONNAIRE

In each issue, determine the content of each question.

Decide on the format of each question

QUESTION WORDING

Determine how the question is worded

Evaluate each research question on the basis of comprehensibility, knowledge and ability, willingness/inclination of a typical respondent to

answer the question

FORMATTING THE QUESTIONNAIRE

In each issue, determine the content of each question.

Decide on the format of each question

The process of questionnaire designThe process of questionnaire design

SEQUENCING AND LAYOUT DECISIONS

Lay out the questions in a proper sequence

Group all the questions in each subtopic to get a single questionnaire

PRETESTING AND CORRECTING PROBLEMS

Read through the whole questionnaire to check whether it makes sense and it measures what it is supposed to measure

Check the questionnaire for error

Pretest the questionnaire

Correct the problems

Designing the QuestionnaireDesigning the Questionnaire

Logical Steps to Develop a Good QuestionnaireLogical Steps to Develop a Good Questionnaire Plan what to measurePlan what to measure Formulate questions to obtain the needed informationFormulate questions to obtain the needed information Decide on the order and wording of questions and the Decide on the order and wording of questions and the

layout of the questionnairelayout of the questionnaire Using a sample, test the questionnaire for omissions and Using a sample, test the questionnaire for omissions and

ambiguityambiguity Correct the problems (pretest again, if necessary)Correct the problems (pretest again, if necessary)

Designing the Questionnaire (Contd.)Designing the Questionnaire (Contd.)

Planning What to MeasurePlanning What to Measure Specify research objectives and information to Specify research objectives and information to

be collected be collected Relevance of the studyRelevance of the study

Translating Research Objectives Into Translating Research Objectives Into Information RequirementsInformation Requirements

Designing the Questionnaire (Contd.)Designing the Questionnaire (Contd.)

Formatting the QuestionFormatting the QuestionDecision to be made regarding the degree of freedom to be Decision to be made regarding the degree of freedom to be given to the respondents in answering the questionsgiven to the respondents in answering the questions

AlternativesAlternatives Open ended with no classificationOpen ended with no classification Open ended where the interviewer uses precoded Open ended where the interviewer uses precoded

classifications to record the responseclassifications to record the response Close ended or structured format in which a Close ended or structured format in which a

question or a supplementary card presents the question or a supplementary card presents the responses to be consideredresponses to be considered

Open Ended QuestionsOpen Ended Questions Open Ended Questions Are Good for the Following Open Ended Questions Are Good for the Following

CircumstancesCircumstances Introduction to a survey or to a topicIntroduction to a survey or to a topic When it is important to measure the saliency of an issue to a When it is important to measure the saliency of an issue to a

respondentrespondent When there are too many responses to be listed, or they can't When there are too many responses to be listed, or they can't

be foreseenbe foreseen When verbatim responses are desired to give the flavor of When verbatim responses are desired to give the flavor of

people's answers or to cite examplespeople's answers or to cite examples When the behavior to be measured is sensitive or disapprovedWhen the behavior to be measured is sensitive or disapproved

Open Ended Questions (Contd.)Open Ended Questions (Contd.)

AdvantagesAdvantages Wide range of responsesWide range of responses Responses obtained without any influenceResponses obtained without any influence Free choicesFree choices


DisadvantagesDisadvantages Variability in the clarity and depth of the responses Variability in the clarity and depth of the responses Articulateness of the respondent in personal Articulateness of the respondent in personal

interviewinterview Willingness to compose a written answer for a Willingness to compose a written answer for a

mail surveymail survey Interviewer's ability to record the verbatim answers Interviewer's ability to record the verbatim answers

quicklyquickly


Disadvantages (Cont.)Disadvantages (Cont.) These type of questions are also time consumingThese type of questions are also time consuming Involves subjective judgements during summarization Involves subjective judgements during summarization

and are prone to errorand are prone to error ExpensiveExpensive The answer to this type of question expands or contracts The answer to this type of question expands or contracts

depending on the space or time available for itdepending on the space or time available for it Respondents may not use the same frame of reference, Respondents may not use the same frame of reference,

when the options are not availablewhen the options are not available

Closed-response QuestionsClosed-response Questions

There Are Two Basic Formats for Closed Ended There Are Two Basic Formats for Closed Ended or Structured Questionsor Structured Questions

Choice from a list of responsesChoice from a list of responses Appropriate single-choice rating on a scaleAppropriate single-choice rating on a scale

Closed-response Questions (Contd.)Closed-response Questions (Contd.)

AdvantagesAdvantages Easier to answerEasier to answer Require less effort by the interviewerRequire less effort by the interviewer Tabulation and analysis is easierTabulation and analysis is easier Less potential error in the way the question is Less potential error in the way the question is

asked and the way it is recordedasked and the way it is recorded The responses are directly comparable from The responses are directly comparable from

respondent to respondentrespondent to respondent


Limitations Limitations Disagreement among researchers on the type of Disagreement among researchers on the type of

responses that should be listedresponses that should be listed The answer to a closed response question will be The answer to a closed response question will be

received no matter how relevant or irrelevant the received no matter how relevant or irrelevant the question is in that contextquestion is in that context

May not produce meaningful resultsMay not produce meaningful results Dichotomous questions are prone to a large amount of Dichotomous questions are prone to a large amount of

measurement error because the alternatives are polarizedmeasurement error because the alternatives are polarized


Limitations (Contd.)Limitations (Contd.) Good questions are hard to developGood questions are hard to develop Provides fewer opportunities of self expressionProvides fewer opportunities of self expression The list of alternative responses provides The list of alternative responses provides

answers that might have not been considered by answers that might have not been considered by the respondent who are reluctant to admit the respondent who are reluctant to admit ignorance, and thereby selecting a "reasonable" ignorance, and thereby selecting a "reasonable" responseresponse

Number of Response CategoriesNumber of Response Categories

Generally five to seven categoriesGenerally five to seven categories Ideally the multiple choices should be Ideally the multiple choices should be

mutually exclusivemutually exclusive

Order of Response CategoriesOrder of Response Categories

Responses are likely to be affected by the Responses are likely to be affected by the order in which they are presented.order in which they are presented.

What factor influences your fast-food What factor influences your fast-food restaurant choice most ?restaurant choice most ?

Convenient locationConvenient location Quality of foodQuality of food Menu selectionMenu selection Fast serviceFast service Reasonable pricesReasonable prices Brand nameBrand name CleanlinessCleanliness

Order of Response CategoriesOrder of Response Categories To prevent order bias, place the average or To prevent order bias, place the average or

expected response at various positions in the expected response at various positions in the sequence of categoriessequence of categories

Range of Response CategoriesRange of Response Categories Respondents who do not know the answer Respondents who do not know the answer

might take categories as cues.might take categories as cues.

How many long-distance calls do make in a How many long-distance calls do make in a week?week?

less than 5less than 5 less than 10 less than 10 5-105-10 or or 10-20 10-20 More than 10.More than 10. More than 20. More than 20.

Handling Uncertainty and IgnoranceHandling Uncertainty and Ignorance

Concerns the handling of “don’t know” and Concerns the handling of “don’t know” and neutral responsesneutral responses

May be advisable to provide the interviewer May be advisable to provide the interviewer with an additional “no answer” category to with an additional “no answer” category to identify these people correctlyidentify these people correctly

Using Both Open-Response And Closed-Using Both Open-Response And Closed-Response QuestionsResponse Questions

Probe:Probe:Using an open-response question to follow up a Using an open-response question to follow up a closed-response questionclosed-response question

Two general purposes for the use of probes:Two general purposes for the use of probes: Pinpoint questions that were particularly difficult Pinpoint questions that were particularly difficult

for respondentsfor respondents Aid researcher interpretation of respondent Aid researcher interpretation of respondent

answersanswers

Question WordingQuestion WordingCare Has to Be Taken ThatCare Has to Be Taken That Is the vocabulary simple, direct, and familiar to all Is the vocabulary simple, direct, and familiar to all

respondents?respondents? Do any words have vague or ambiguous meanings?Do any words have vague or ambiguous meanings? Are any questions " double-barreled”?Are any questions " double-barreled”? Are any questions leading or loaded?Are any questions leading or loaded? Are the instructions potentially confusing?Are the instructions potentially confusing? Is the question applicable to all respondents?Is the question applicable to all respondents? Are the questions of appropriate length?Are the questions of appropriate length?

Question WordingQuestion Wording Avoid ambiguous wordsAvoid ambiguous words

How many times per month do you How many times per month do you visit a fast-food restaurant?visit a fast-food restaurant?

NeverNever OccasionallyOccasionally SometimesSometimes OftenOften

Question WordingQuestion Wording Are any questions loaded?Are any questions loaded?

1) Don’t you think, because its so greasy, 1) Don’t you think, because its so greasy, fast-food is one of the worst types of fast-food is one of the worst types of food? food?

2) Do you prefer a hamburger that is 2) Do you prefer a hamburger that is grilled on a hot stainless-steel grill or grilled on a hot stainless-steel grill or cooked by passing the raw meat through cooked by passing the raw meat through an open gas flame?an open gas flame?

Question WordingQuestion Wording Are any questions "double-barreled”?Are any questions "double-barreled”?

Are you satisfied with the price and the Are you satisfied with the price and the service of Taco Bell?service of Taco Bell?

Question WordingQuestion Wording Is the question applicable to all respondents?Is the question applicable to all respondents?

Why do you like fast-food?Why do you like fast-food?

Assumes that respondent likes fast-food.Assumes that respondent likes fast-food.

Asking Sensitive QuestionsAsking Sensitive Questions

Consumption of Kellogg’s Frosted FlakesConsumption of Kellogg’s Frosted FlakesThe casual approachThe casual approach

““Have you eaten ‘Frosted Flakes’ within the last week?Have you eaten ‘Frosted Flakes’ within the last week?

The numbered cardThe numbered card ““Would you please read off the number on this card that Would you please read off the number on this card that

corresponds to what you had eaten for breakfast in the last corresponds to what you had eaten for breakfast in the last week?”week?”

(Hand card to respondent)(Hand card to respondent)1.1. PancakesPancakes2.2. Frosted FlakesFrosted Flakes3.3. Other (what)?Other (what)?

Asking Sensitive Questions (Cont.)Asking Sensitive Questions (Cont.)

The everybody approachThe everybody approach ““As you know, many people have been eating As you know, many people have been eating

Frosted Flakes for breakfast. Do you eat Frosted Frosted Flakes for breakfast. Do you eat Frosted Flakes?”Flakes?”

The “other people” approachThe “other people” approach ““Do you know of any adult who eats Frosted Do you know of any adult who eats Frosted

Flakes?”Flakes?” ““How about yourself?”How about yourself?”


The sealed ballot techniqueThe sealed ballot technique In this version you explain that the survey respects people’s In this version you explain that the survey respects people’s

right to anonymity with respect to their eating habits, and that right to anonymity with respect to their eating habits, and that they themselves are to fill out the answer to the question, seal they themselves are to fill out the answer to the question, seal it in an envelope, and drop it in a box conspicuously labeled it in an envelope, and drop it in a box conspicuously labeled “sealed ballet box” that is carried by the interviewer“sealed ballet box” that is carried by the interviewer

The Kinsey approachThe Kinsey approach Stare firmly into respondent’s eyes and ask in simple clear-cut Stare firmly into respondent’s eyes and ask in simple clear-cut

language as that to which the respondent is accustomed, and language as that to which the respondent is accustomed, and with an air of assuming that everyone had done everything, with an air of assuming that everyone had done everything, “Do you eat Frosted Flakes for breakfast?”“Do you eat Frosted Flakes for breakfast?”


Randomized Response TechniqueRandomized Response Technique The respondent is asked to answer one or two randomly The respondent is asked to answer one or two randomly

selected questions without revealing which question has been selected questions without revealing which question has been answeredanswered

Questions:Questions: SensitiveSensitive InnocuousInnocuous

Since the interviewer records a “yes” or “no” answer Since the interviewer records a “yes” or “no” answer without knowing which question has been answered, without knowing which question has been answered, the respondent feels free to answer honestlythe respondent feels free to answer honestly

Randomized Response TechniqueRandomized Response Technique

P[Yes] = P[Yes|P[Yes] = P[Yes|S.QS.Q] * P[S.Q] + P[Yes|] * P[S.Q] + P[Yes|I.QI.Q] * ] * P[I.Q]P[I.Q]

wherewhere

S.Q = Sensitive QuestionS.Q = Sensitive Question

I.Q = Innocent QuestionI.Q = Innocent Question

Sequence And Layout DecisionsSequence And Layout Decisions

Open with an easy and non threatening Open with an easy and non threatening questionquestion

The questionnaire should flow smoothly and The questionnaire should flow smoothly and logically from one topic to the nextlogically from one topic to the next

Proceed from broad general questions to the Proceed from broad general questions to the more specificmore specific

Organization of a Typical QuestionnaireOrganization of a Typical QuestionnaireLocation Type Function Example

Starting questions Broad, generalquestions

To break the ice andestablish a report withthe respondent

Do you own apersonal computer?

Next few questions Simple and directquestions

To reassure therespondent that thesurvey is simple andeasy to answer

What brands ofpersonal computersdid you considerwhen you bought it?

Questions up to athird of thequestionnaire

Focused questions Relate more to theresearch objectivesand convey to therespondent the area ofresearch

What attributes didyou consider whenyou purchased youpersonal computer?

Major portion of thequestionnaire

Focused questions;some may be difficultand complicated

To obtain most of theinformation requiredfor the research

Rank the followingattributes of apersonal computerbased on theirimportance to you

Last few questions Personal questionsthat may be perceivedby the respondent assensitive

To get classificationand demographicinformation about therespondent

What is the highestlevel of educationyou have attained?

Sequence And Layout Decisions Sequence And Layout Decisions (Contd.)(Contd.)

Sensitive questions should not be placed in the Sensitive questions should not be placed in the beginning of the questionnairebeginning of the questionnaire

Use good quality of paperUse good quality of paper Physical layout should be appealing and Physical layout should be appealing and

interestinginteresting Order biasOrder bias

Order Bias: Does The Question Create Order Bias: Does The Question Create The Answer?The Answer?

1. No question asked1. No question asked 2.82.8

2. Asked only about advantages2. Asked only about advantages 16.716.7

3. Asked only about disadvantages3. Asked only about disadvantages 0.00.0

4. Asked about both advantages and 4. Asked about both advantages and disadvantagesdisadvantages

5.75.7

Percentage of Respondents “Very Much Interested” in Buying New ProductQuestions Preceding Buying Interest Question

Pretesting and Correcting ProblemsPretesting and Correcting Problems Pretest DesignPretest Design Pretesting Specific Questions ForPretesting Specific Questions For

VariationVariation MeaningMeaning Task difficultyTask difficulty Respondent interest and attentionRespondent interest and attention

Pretesting the QuestionnairePretesting the Questionnaire Flow of the questionnaireFlow of the questionnaire Skip patternsSkip patterns LengthLength Respondent Interest and AttentionRespondent Interest and Attention

ExercisesExercises

Why did you purchase a Sony brand Why did you purchase a Sony brand Walkman?Walkman?

Price was lower than other alternativesPrice was lower than other alternatives Feel it represents the highest qualityFeel it represents the highest quality Availability of local serviceAvailability of local service Sound is betterSound is better Warranty is betterWarranty is better Looks goodLooks good

ExercisesExercises

Have you ever listened to a Long John Have you ever listened to a Long John Silvers radio commercial?Silvers radio commercial?

Which of the following restaurants do you Which of the following restaurants do you visit frequently?visit frequently?

Burger KingBurger King Pizza HutPizza Hut James Coney IslandJames Coney Island

ExercisesExercises

Do you agree that, since fast-food Do you agree that, since fast-food restaurants produce a disproportionate restaurants produce a disproportionate amount of waste, they should be subject amount of waste, they should be subject to an additional environmental tax?to an additional environmental tax?

YesYes NoNo

ExercisesExercises

How often do you eat fast food?How often do you eat fast food?

DailyDaily Every second dayEvery second day Once a weekOnce a week Every two weeksEvery two weeks

designing a questionaire

Documents