RTI International is a trade name of Research Triangle Institutewww.rti.org
Nonresponse Bias Correction in Telephone
Surveys Using Census Geocoding:
An Evaluation of Error Properties
Paul P. Biemer and Andy Peytchev
RTI International
AAPOR Annual Conference
May 13, 2010
Acknowledgments
• This material is based upon work supported by the
National Science Foundation under Grant No. SES-
0818931.
• We also thank the Survey Research Center at the
University of Michigan for their assistance in
compiling the data.
5/18/20102
3
Outline
• Census Geocoding
• Potential error in the CG approach
• Empirical evaluation
– Overall error
– Decomposition by source of error
• Conclusions and next steps in ongoing research
Correcting for Nonresponse in RDD
• Very limited information
– Frame has only telephone number
– Other identifiers can be matched to gain access to auxiliary
data
• Practitioners use this general approach to attempt to
correct for unit nonresponse
• Evaluations of such approaches compare estimates
with and without the use of the auxiliary data as
evidence for the ability to correct for nonresponse
5/18/2010 www.rti.org4
Census Geocoding (CG) Approach
1. Obtain the addresses for as many RDD sample
numbers as possible. This usually results in
addresses for about 50% to 60% of the sample.
2. For numbers matched to an address, convert
addresses to geographic coordinates (i.e., geocode).
a. Link each sample member with the census block containing
its coordinates.
b. Substitute census block aggregate characteristics.
3. Sample numbers with no (geocodable) addresses
can be handled in a number of ways. One method is
to substitute with aggregate exchange area
characteristics.
5/18/2010 www.rti.org5
Potential Problems with the CG approach
• Correctly matching to auxiliary information
– Using phone number
– Using area code and prefix
• Measurement differences with the auxiliary data
• Aggregation
5/18/2010 www.rti.org6
Potential Problems with the CG approach
• Bias:
• Decomposing error in the CG approach:
5/18/2010 www.rti.org7
)]~(*[)~( nrnrnrr yypEyBias
n
rr
y
yBiasyRelBias
)~()~(
)~(*)~(*[)~( ipipipcpcpcpr yypyypn
rnyBias
)]~(*)~(* ieieiececece yypyyp
Estimation of the CG Error Components
• Start with a “gold standard” survey that provides
responses for almost every sample member and has
collected phone numbers
• Perform matching for all available telephone
numbers, producing the four types of outcomes found
in applying the CG to RDD studies
• Simulate nonresponse under different assumptions,
such as number of call attempts or applying model
estimated from an RDD survey
• Estimate bias in CG variables and its components
5/18/2010 www.rti.org8
National Comorbidity Survey-Replication
• Area probability sample, face to face survey, n=9285
– n=8178 who provided a phone number and responses to
demographic questions
– n=4987 who also provided income
• Conducted 2001-2003
• 85.9% response rate
5/18/2010 www.rti.org9
Absolute Relative Bias
5/18/2010 www.rti.org10
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
Proportion of Bias in CG Process by Source
5/18/2010 www.rti.org11
-100%
-80%
-60%
-40%
-20%
0%
20%
40%
60%
80%
100%
Non
hisp
anic
Whi
te
Non
hisp
anic
Bla
ck
Non
hisp
anic
Asia
n
Hisp
anic
Mal
e
One
-per
son
Hou
seho
ld
Child
ren
Und
er 1
8
Age
18
-24
Age
25
-29
Age
30
-34
Age
35
-39
Age
40
-44
Age
45
-49
Age
50
-54
Age
55
-59
Age
60
-64
Age
65
-69
Age
70
-79
Age
80
and
over
Inco
me
0-9
99
9
Inco
me
1000
0-1
49
99
Inco
me
1500
0-1
99
99
Inco
me
2000
0-2
49
99
Inco
me
2500
0-2
99
99
Inco
me
3000
0-3
49
99
Inco
me
3500
0-3
99
99
Inco
me
4000
0-4
49
99
Inco
me
4500
0-4
99
99
Inco
me
5000
0-5
99
99
Inco
me
6000
0-7
49
99
Inco
me
7500
0-9
99
99
Inco
me
1000
00-1
24
99
9
Inco
me
1250
00-1
49
99
9
Inco
me
1500
00-1
99
99
9
Inco
me
2000
00 an
d up
p*y -tilde phone - y-bar survey, correct matches p*y -tilde phone - y-bar survey, incorrect matches
p*y -tilde exchange - y-bar survey, correct matches p*y -tilde exchange - y-bar survey, incorrect matches
Conclusions
• Substantial error in the census geocoding approach
was found for some demographic estimates and it
varied by source
– Error from cases erroneously mismatched at the exchange
level was a major source
– A large component could also be attributed to cases
correctly matched at the census blockgroup level
• Of critical importance is how the CG approach may
still be informative of nonresponse bias in survey
estimates and can be used to correct for it
5/18/2010 www.rti.org12
Paul Biemer: [email protected]
Andy Peytchev: [email protected]
www.rti.org/aapor
5/18/2010 www.rti.org13
