address frames and mail surveys as complements (or alternatives) to rdd surveys michael w. link,...
Post on 27-Mar-2015
213 Views
Preview:
TRANSCRIPT
Address Frames and Mail Surveysas Complements (or Alternatives)
to RDD Surveys
Michael W. Link, Michael P. Battaglia, Martin R. Frankel,
Larry Osborn, and Ali H. Mokdad
Second International Conference on Telephone Survey Methodology, Miami, FL
Problems Facing RDD SurveysGrowing NonresponseFrame coverage issues:
Households with no telephones (1.6%) Cell phone only households (3.7%) Households in zero-banks (3-4%)
Frame efficiency issues: Proliferation of telephone numbers Cell phone numbers in mixed-use exchanges
Other issues: Erosion of geographic specificity at state and
substate levels
Behavioral Risk Factor Surveillance System (BRFSS)
Monthly state-based RDD survey of health issues
50 states, District of Columbia, Puerto Rico, Guam, and Virgin Islands
300,000 adult interviews conducted in 2005
Faced with declining response rates
Need to identify best future design (frame & mode)
USPS Delivery Sequence File (DSF) as an alternative sampling frame
File contains All delivery point addresses serviced by USPS Identifies address type
Residential vs business City style vs PO box vs other types
Format conforms to USPS addressing standards
Initial assessments for survey use: Highest coverage in urban areas Potential for coverage to improve in rural areas
Potential Drawbacks of DSFUnknown level of coverage:
Excludes households with no USPS mail delivery Must purchase through list vendor (not USPS)
Updates/list maintenance may vary Some exclude addresses on request
Includes simplified addresses: City, state, zip code only
Other potential problems: Seasonal units, PO Boxes and multi-drop addresses
Key questions to addressHow do RDD and DSF-based mail surveys
compare in terms of: frame efficiency response rates respondent demographics Estimates on key health issues
Can DSF-based mail surveys reach households without telephones and cell phone-only households?
BRFSS 2005 DSF mail survey pilot
Six states: CA, IL, NJ, NC, TX, WA
Sampling frame: access to Delivery Sequence File (DSF) provided by Marketing Systems Group
Mode: mail survey with telephone verification for respondent selection
Mail survey fielded March 15-May 15, 2005
Compared to monthly RDD surveys from March-May, 2005
BRFSS 2005 DSF pilot: sample designProbability sample from DSF household frames in each
state
Excluded business addresses identified by USPS or Marketing Systems Group
Included seasonal units, vacant units, PO Boxes, throwback units, and drop point units
Stratified each state sample by county and address type
Drew 1,680 addresses per state using systematic random sampling
Split sample treatment groups
Postcard (after 7 days)
Second questionnaire mailing (after 2 weeks)
Surname on address label
Alternative within household selection methods: any adult (non-probability) next birthday all adults
Frame coverage assessment and characteristics
Percentage of Counties with >10% Under-coverage by State
0
10
20
30
40
50
60
NJ CA IL WA TX NC
% counties
with
>10% under-
coverage
Percentage of Counties with >10% Under-coverage by Pct. Urban
0
10
20
30
40
50
60
70
80
90
% counties
with
>10% under-
coverage
< 25% 25-49%
50-74%
75+%
% of adults in county living in urban area
BRFSS DSF Pilot:Types of Addresses
State
Address Type
City Style(%)
PO Box(%)
Seasonal Unit(%)
Vacant Unit(%)
Throw-back Unit(%)
DropPoint Unit(%)
California 91 8 <1 1 <1 <1
Illinois 87 5 0 3 <1 5
New Jersey 86 6 <1 1 <1 5
North Carolina
89 7 <1 2 <1 <1
Texas 89 8 <1 2 <1 <1
Washington 91 6 0 2 <1 <1
Response rates
Design factors and probability of completed interview from total cases
(adjusted odds ratios)
AOR (95% CI)
Address type Other type 1.00
City style 2.27 (1.74-2.95)
PO Box 1.83 (1.30-2.58)
Postcard sent No 1.00
Yes 1.12 (1.02-1.22)
Second Questionnaire No 1.00
Yes 1.58 (1.44-1.73)
Surname on mailing No name available 1.00
Name not used 2.01 (1.77-2.29)
Name used 1.84 (1.62-2.09)
Respondent selection Any adult 1.00
Next birthday 0.91 (0.81-1.02)
All adults 0.91 (0.81-1.01)
(n) (10,080)
Comparison of RDD telephone and DSF mail survey response rates
State
Response Rates
RDD telephone survey
%(n)
DSF mail survey:All cases
%(n)
DSF mail survey:Cases with 2nd
Mailing(n)
California 39.2(4,318)
31.8(1,266)
39.2(597)
Illinois 38.7(4,462)
36.2(1,356)
42.8(671)
New Jersey 33.8(9,976)
23.2(1,250)
30.5(614)
North Carolina 56.0(7,992)
36.3(1,200)
42.5(602)
Texas 43.6(4,920)
35.5(1,122)
44.4(543)
Washington 45.7(12,910)
39.9(1,334)
44.9(626)
Within household selection
Comparison of “Equalized” Weighted Gender Distributions: %
Female
Population 51.4%
Any Adult 61.5%
Next Birthday 61.5%
All Adults 50.8%
Other demographics of DSF mail survey respondents
Percent some college or more
0
10
20
30
40
50
60
70
80
90
100
% Some college or more
CPS RDD DSF Mail
58.471.8
53.8
Percent white
0
10
20
30
40
50
60
7080
90
100
% White
CPS RDD DSF Mail
67.7 69.564.9
Percent household income > $50,000
010
20
30
4050
6070
8090
100
% Household income > $50,000
CPS RDD DSF Mail
43.848.646.4
Comparison of Survey Estimates
Comparison of Survey Estimates
Health conditions / risk behaviors
Unadjusted prevalence Adjusted odds ratio
Telephone Mail Telephone Mail
Asthma 12.2 14.2 1.00 1.18
Diabetes 8.4 7.6 1.00 0.94
High blood pressure 26.5 27.4 1.00 1.09
Obese (BMI > 30) 29.0 22.4** 1.00 0.83*
Current smoker 18.6 17.0 1.00 1.04
Binge drinking 12.5 20.4*** 1.00 1.77***
Tested for HIV3 42.3 40.5 1.00 0.88
HIV risk behaviors3 4.1 6.9** 1.00 1.74**Significance: * p<.05, ** p<.01, *** p < .001Note: Data weighted for sample design and post-stratified to sex-age totals for each state. Final weights were ratio adjusted to equalize the number of cases across states. Logistic regression models adjusted for state of residence, sex, race, age, education, and having health care coverage.
Reaching cell-only andnon-telephone households
Type of householdtelephone access
1 Based on interviews NHIS conducted July – December, 2004. Source: Stephen J.Blumberg, Julian V. Luke, and Marcie L. Cynamon (2005). “The Prevalence and Impact of Wireless Substitution: Updated Data from the 2004 National Health Interview Survey.” Presented at the 2005 American Association for Public Opinion Research Annual Conference, Miami Beach, FL.
Household telephone access National Health Interview Survey1
(%)
BRFSS mail survey
(%)
Land line 92.1 91.4
-- Landline only --- 13.7
-- Landline and cellular phone --- 79.4
Cellular phone only 5.5 6.0
No telephone 2.4 0.9
Effect of household telephone access on mail survey estimates
Health condition /risk factor
Type of household telephone access
Landline only
Landline and cell phone
Cell phone only
Asthma 1.00 0.84 1.08
Diabetes 1.00 0.86 1.59
High blood pressure 1.00 0.95 0.99
Obese (BMI > 30) 1.00 0.95 1.05
Current smoker 1.00 0.70* 1.06
Binge drinking 1.00 1.56* 1.90*
Tested for HIV 1.00 1.02 1.35
HIV risk behaviors 1.00 1.22 3.95**
Figures are adjusted odds ratios. Significance: * p<.05, ** p<.01, *** p < .001Note: Data weighted for sample design and post-stratified to sex-age totals for each state. Final weights were ratio adjusted to equalize the number of cases across states. Logistic regression models adjusted for state of residence, sex, race, age, education, and having health care coverage.
Advantagesof address-based design
In low response rate states the address-based mail survey approach can yield response rates similar to RDD rates Telephone follow-up of non-respondents should raise
rates
Approach reaches households without land-line telephones
Weighted prevalence estimates were similar for 5 of 8 risk factors
Facilitates geocoding and mapping
Disadvantagesof address-based design
Coverage in rural areas is a potential problem
Mail survey limits number of questions and complexity of survey
Mail survey alone does not yield higher response rates than RDD
Less control over within household selection
Mail survey respondents tend to have higher SES
Next Steps2006 “production level” pilot study in 6 statesTest alternative sampling approaches:
RDD sample reverse-matched for addresses Address-based sample matched for
telephone numbersTest mixed-mode design:
If address available: mail survey with telephone follow-up of nonrespondents
If no address available: telephone survey
Contact:
Michael LinkMLink@cdc.gov
www.cdc.gov/brfss
top related