6/3/2015bill reimer1 sampling and questionnaires bill reimer october 31, 2005...
Post on 19-Dec-2015
229 views
TRANSCRIPT
04/18/23 Bill Reimer 1
Sampling and Questionnaires
Bill ReimerOctober 31, 2005
[email protected]://reimer.concordia.ca/teaching
04/18/23 Bill Reimer 3
Sampling Objective Get a lot of information with little
cost
PopulationSample
04/18/23 Bill Reimer 4
Sample sizes for a population of 10,000
96 150 267600
2401
166 259461
1037
4147
0500
10001500200025003000350040004500
10% 8% 6% 4% 2%
Range of acceptable error (Accuracy)
Sam
ple
Siz
e
95% Confidence99% Confidence
04/18/23 Bill Reimer 5
3 Rules of Scientific Sampling
each unit in the population has an equal chance of being chosen
we must know the chance of each member being chosen
each selection must be independent from the others
04/18/23 Bill Reimer 6
To what do you want to generalize?
People Households Trees Leaves
These are the Units of Analysis
04/18/23 Bill Reimer 7
To which units do you want to generalize?
People in Canada? Women in Québec? Citizens of Montréal? Trees in boreal forests?
This is the Population
04/18/23 Bill Reimer 8
The Population
Is the collection of units to which you want to generalize
It may be abstract It may be concrete
But it must be clear
04/18/23 Bill Reimer 9
From what will you choose your sample?
A list of units? A region or place? A drawer or computer file? A particular time?
This is your sample frame
04/18/23 Bill Reimer 10
The Sample Frame may not match the Population
Population
Sample Frame
Units in th
e SF but
not the P
Uni
ts in
the
P
but n
ot th
e SF
…but try to get them close
04/18/23 Bill Reimer 11
Sampling Requirements:
Identify the population Is the sample frame reasonable? Does it meet the 3 conditions of
scientific sampling?
We can seldom meet all 3 conditions!
04/18/23 Bill Reimer 12
Types of Samples
Probability Generalizability Efficient
Non-probability Exploratory Strategic
04/18/23 Bill Reimer 13
Simple Random Sampling
Equal chance of being chosen
We know the chance
Each selection independent
From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8.
04/18/23 Bill Reimer 14
Systematic Sampling
Determine sample rate (8/40)
Random selection of 1st case
Every nth case (n=5) Watch for
regularities in the sample frame
From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8.
Random selection
04/18/23 Bill Reimer 15
Stratified Random Sampling
Separate sample frame into strata
Take random (or systematic) sample from each strata
May be proportionate or disproportionate
From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8.
04/18/23 Bill Reimer 16
Cluster Sampling
Divide population into clusters
Randomly select clusters
Collect data on all cases in cluster
Watch cluster composition
From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8.
04/18/23 Bill Reimer 19
Convenience Sample
Choose most convenient people
Subject to multiple biases
From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8.
04/18/23 Bill Reimer 20
Volunteer Sampling
Ask for volunteers Directly Ads Selected groups
Unclear biases
From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8.
04/18/23 Bill Reimer 21
Strategic Sampling
Handpick respondents for strategic purposes
Useful for marginalized or small groups
Control biases by specifying criteria
From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8.
04/18/23 Bill Reimer 22
Snowball Sampling
Identify strategic respondent
Ask for referrals Continue the
process Use multiple starts
to avoid network biases
From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8.
04/18/23 Bill Reimer 24
Sample Size Depends On
Size of the population Accuracy desired Confidence desired Variation in the phenomenon
investigated
04/18/23 Bill Reimer 25
Assignment Specify a research question you would like to
answer. Describe a sampling procedure along with
the data you would collect to answer the question.
Identify the following: the population the sample unit the sample frame the type of sample
04/18/23 Bill Reimer 26
Samples Non-probability
Accidental Purposive
Probability Simple random Systematic Stratified:
• Proportionate• Disproportionate
Cluster Multiple
04/18/23 Bill Reimer 28
Types of Research Interviews Questionnaires
Closed and some open-ended questions Semi-structured interviews
Open and some closed-ended questions Unstructured interviews
Open-ended questions
Research Interviews are NOT what you see on TV
04/18/23 Bill Reimer 29
Advantages and Disadvantages
Questionnaires Economic Speedy Minimal
interviewer bias Anonymity Low response rates Limited exploration Comparison easy
Semi-structured High response rates Can ask
complicated questions
Can follow-up Qs Can observe Expensive Time consuming Comparison difficult
04/18/23 Bill Reimer 30
Preparing Questions Resist the temptation to start with
specific questions Clarify the RESEARCH question Identify the types of information
required to answer the research question
Within each of the types of information brainstorm for specific questions to ask
Rearrange questions
04/18/23 Bill Reimer 31
Question Order Leave sensitive issues for later - once
rapport has been established Follow from general to specific Use tables and grids where appropriate Use cards for sensitive issues if
appropriate Prepare your probes if interview Follow a natural flow
04/18/23 Bill Reimer 32
Anticipate the Analysis
Produce dummy tables Work backwards from your analysis
04/18/23 Bill Reimer 33
Format examples Contingency question Full filter question Quasi-filter question Closed question Partially open question Open question Probes
04/18/23 Bill Reimer 34
Simple, Mutually Exclusive What is your gender? (Circle the
appropriate number) Female . . . . . . . . . . . . . . . 1 Male . . . . . . . . . . . . . . . . . 2
5
04/18/23 Bill Reimer 35
Multiple Choice, not Mutally Exclusive What modes of transportation did
you used to get to school last week? (Circle all that apply)
• Metro . . . . . . . . . . . . . . . . . 1• Bus . . . . . . . . . . . . . . . . . . . 1• Car . . . . . . . . . . . . . . . . . . . 1• Bicycle . . . . . . . . . . . . . . . . . 1• Walk . . . . . . . . . . . . . . . . . . 1• Other (please specify)
_____________________ 1
6
7
8
9
10
|__|__| 11-12
04/18/23 Bill Reimer 36
Rank-order Rank order the 3 most important aspects of
your job (Place 1 beside the most important, 2 beside the next most important, and 3 beside the next most important)
• Money . . . . . . . . . . . . . . . . . . |___|• Security . . . . . . . . . . . . . . . . .|___|• New experiences . . . . . . . . . . |___|• Power . . . . . . . . . . . . . . . . . . |___|• Excitement . . . . . . . . . . . . . . |___|• Other (please specify)
________________________ |___|
13
14
15
16
17
|__|__| 18-19
04/18/23 Bill Reimer 37
Pretest, pretest, pretest Use your friends and family Discuss
Where it was clear or not clear What they thought about when
responding to each question What they thought you were getting at
Move to colleagues Check it out on people who are similar
to those you will eventually survey
Problem Questions Jargon, slang, abbreviations
What is your Internet browser? Ambiguity, confusion, vagueness
What is your income? Emotional language and prestige bias
Should we put murderous terrorists in jail? Do you support Prime Minister Chretien’s
policy on world trade? Double-barrelled questions
Should marijuana be legalized for medical or other purposes?
04/18/23 Bill Reimer 39
Problem Questions – con’t Leading questions
Do you feel that governments should have less power to interfere in business?
Beyond respondent’s competence Does your mother support extramarital
sex? False premises
How can we halt the rising crime rate?
04/18/23 Bill Reimer 40
Problem Questions – con’t Future intentions
How will you vote in the next federal election?
Overlapping or unbalanced response categories What is your opinion regarding abortion?
• Very favourable• Favourable• Neutral• Opposed
04/18/23 Bill Reimer 42
Questionnaire Construction The amount of time spent in the
preparation of the questionnaire is directly related to the value of the results.
Surveys usually cost a lot and you can't redo them.
Know what you are wanting to discover
04/18/23 Bill Reimer 43
Research Strategies Questionnaires for generalizability Exploration: well selected, in-depth
interviews Use interviews to design survey Use comparisons
04/18/23 Bill Reimer 44
Sampling and Questionnaires
Bill ReimerOctober 31, 2005
[email protected]://reimer.concordia.ca/teaching