Questionnaires and Survey Design

Target population

Sampling frame

Not included in sampling frame

Not eligible

for survey

Cannot be contacted

Refuse to respond

Incapable of responding

SAMPLED POULATION

Considering survey methods

• There is no best method• Each method has strengths and

weaknesses• Ask yourself1.”Who are the survey respondents?”2. Is the research question more amenable

to one method than to another?”

Other Considerations

1. Administrative factors such as cost, time, geographic distribution

2. Questionnaire issues such as branching, complexity of questions, sensitivity issues, rapport with respondent

3. Data quality issues such as bias, knowledge about refusals and non-contacts, control of replies

Mailed questionnaires

• Cost is relatively low• Response rate may be poor and biased

towards more educated and those with an interest in the topic

• No knowledge about non-response• Long time between data collection and

analysis• Geographic distribution can be wide

Telephone surveys• Most widely used type• Numbers are selected randomly from a phone

book or by generating random numbers (RDD: (Random Digit Dialling)

• Problems include unlisted numbersLandline coverage is decreasing so increasing bias against the young (no directory for cell phones)

• Geographic distribution can be wide• Data collection period can be short• Questions must be short and simple

Face-to-face surveys

• Cost is high• Data collection period is long• Geographic distribution must be clustered• Good rapport• Take a long time• Good control and quality of responses• Sensitivity issues fair quality

Internet• Low cost (no paper, no data entry costs)• data collection is quick• Geographic distribution may be wide• Questionnaires may be complex because

the skips are programmed in • Pop-up instructions, videos, voice-overs,

animation are available to make it more fun and dynamic

• If self-selected, results may be biased.