quantitative methods in the social sciences (qmss) lugano 25-26 august 2005 increasing response...
Post on 22-Dec-2015
215 views
TRANSCRIPT
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Increasing response rates
Ineke StoopSCP
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Nonresponse research
• How to increase response rates?• How to measure and correct for
nonresponse bias?
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
SCP-survey
• Face-to-face questionnaire + long drop off for every 6+ years household member
• Use of social and cultural services, amenities, facilities (no values, opinions)
• Very low non-contact rate• High rate of refusal conversion• Final response rate 1999 66%
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Response rates AVO
0
10
20
30
40
50
60
70
1979 1983 1987 1991 1995 1999 2003
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Cumulative response rates
0
10
20
30
40
50
60
70
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
1995 1999
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Increase response ratesDescribe response process
ContactabilityReluctance
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Paradata
• Call records (nr, timing, outcome)• Characteristics interaction• Interviewer observations• Sample frame data• Low-level geographic data bases• Interviewer characteristics
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
ContactabilityTNC/NAH/CFC
Call (nr, time, mode)First
contact
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
At home, interviewer calls, contact rate
0
10
20
30
40
50
60
70
80
90
100
6 12 18 24
morning afternoon evening
0
10
20
30
40
50
60
70
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Timing contacts
• Interviewers prefer working hours• Contact rates higher in the evening• Cooperation independent of timing
• Fewer calls if evening calls only• Length fieldwork period/costs?
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Contactability
• Easy– Evening calls– Phone number
available– Detached dwelling– Child at home
• Difficult– Big city dweller– Poor maintenance
neighbourhood– Young– Single– Employed, student– High cultural
participation
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Continuum of resistance
• Extrapolate from hard to reach respondents to noncontacts?– Socio-demographics– Core variables– Two types of hard to contact (short
term/long term)
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Reluctance
Respondent has been contacted
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Reluctance
Request to cooperate
Inter-view
Cooperative respondent
Refusal
Establishfirst contact
Follow-up
Con-tact
Inter-view
Refusal
Request to cooperate
Soft convertedrefusal
Inter-view
Follow-up
Con-tact
Refusal
Refusal
Request to cooperate
Refusal
Hard converted refusal
NoYes
Yes
Yes
Yes
Yes
Yes
Yes
NoNo
No
NoNo
No
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
What happens in the AVO after a first refusal?
Refusal at first contact
Finalrefusal
Re-approached
Inter-view
Finalrefusal
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Reluctant respondents
• Many among– Big city dwellers
• Few among– Men– Singles– Higher educated
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Cooperative and reluctant respondents
0 10 20 30 40 50 60 70 80 90 100
total
couple
single male
Mid east
Big cities
immediate converted
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Refusers, reluctant and cooperative respondents
0 10 20 30 40 50 60 70 80 90 100
total
couple
single male
Mid east
Big cities
total
couple
single male
Mid east
Big cities
immediate converted refuse
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Region and refusal conversion
14
25
51
45
Amsterdam, Rotterdam, The Hague
Mid east
48
-34
7
15
-40 -20 0 20 40 60
still in the field immediate nonresponse re-issue, 2nd refusal
re-issue, no 2nd refusal converted nonrespondents
immediate cooperation
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Results refusal conversion
• Males, singles and higher educated underrepresented among converted refusals– Do interviewer strategies work for
everybody?• Are only promising cases re-
contacted? – May worsen final sample composition
• Effect refusal conversion– Works only for honest burghers, Mr.
and Mrs. Average?
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Conclusions regular AVO
• Contactability related to socio-demographics, being out of the house and fieldwork strategy
• Reluctance related to family composition, gender and education and persuasiveness
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Do nonrespondents differ?
Are final refusals similar to converted refusals
• Sample frame information• Neighbourhood information• Fieldwork information
– Late respondents (similar to non-contacts?)
– Reluctant respondents (similar to refusers?)
• Socio-demographic information
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Response and nonresponse
Frame
Observation
EasyFieldwork
Hard to contact
Reluctant
Follow-up
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Follow-up survey
• Small subsample persistent refusers
• 1 person, 20 minutes, multi-mode• Experienced, motivated, well paid
interviewers• Wide range of incentives• 235 successful interviews• 70% cooperation rate
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Why did refusers cooperate?
• High quality interviewers (and telling them they are the best)
• Extensive briefing• Trust (money for incentives) and
support (newsletter)• Importance (newspaper article)• Wide range of incentives• Better payment• Commitment sponsor and fieldwork
organisation
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Idiosyncratic choice of incentives
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Final analysis
Hard to contact respondentsReluctant respondents
Refusers who cooperated in follow-up survey
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Follow-up survey• Sample of persistent refusals• Socio-demographic differences (made
up for composition regular respondents)
• Small differences in survey variables (mostly related to socio-demographics)
• Lower participation in classical culture• Less PC ownership• Fewer sports activities• Slightly less active in many aspects• Not at all similar to converted refusals
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
refuser reluctant hard to contact
Intercept -2,42 -0,04 -0,38
Age (years) -0,01 0,00 -0,01
Age: absolute deviation mean (years)
-0,01 -0,01 -0,01
Listed phone number 0,50 -0,10 -0,26
Amsterdam, Rotterdam, The Hague -0,22 0,69 0,63
Good maintenance -0,21 -0,26 -0,26
Detached dwelling 0,00 -0,42 -0,28
Male -0,16 -0,76 -0,05
Single 0,14 -0,37 0,29
Child at home -0,30 -0,11 -0,37
Employed, student 0,17 -0,13 0,31
Higher education -0,15 -0,51 -0,03
Cultural participation
Both classical and popular -0,21 -0,03 0,13
Classical -0,56 0,23 0,17
Popular 0,17 -0,07 0,15
No sports activities 0,41 -0,03 0,00
PC in household -0,87 0,18 0,00
Use of internet, e-mail 0,94 0,07 0,14
No religious affiliation 0,38 -0,11 0,00
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
single female
single malesingle parent
couplechildren
couple
no vcr
ethnic minority
25+% ethnic
no ethnic minorities
Randstad
South
Mid east
North east
AmsterdamRotterdamThe Hague
detached
multi-unit
terraced
no phone
75+
16-24
no sports
many sports
no culture
classical
popularculturalomnivore
no pc
pc not used
pc off line
internet
EASY
RELUCTANTREFUSER
HARD TO CONTACT
poor maintenance
excellent
not employed, student
employed, student
atheist
religious
no naturenature
avid reader
no booksprimary ?education?
academic
?income?
veryhigh
income
vey low income
-1,2
-0,8
-0,4
0
0,4
0,8
1,2
1,6
2
-1,6 -0,8 0 0,8
family composition video ethnic minority ethnic neighbourhood region
dwelling telephone age sports culture
PC response maintenance employed, student religion
nature conservancy books education income
Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005
Reducing bias instead of increasing response
rates• Increasing response rates MAY not lead
to better survey estimates• Difficult respondents may not be similar
to final nonrespondents– Obtain information on the process (cause
of/reason for nonparticipation)
• Collect independent information on nonrespondents– Frame information + registers– Central question procedure– Follow-up survey among refusers