1

Sampling Telephone Numbers and Adults, and Interview Length, and Weighting in the California Health

Survey Cell Phone Pilot Study

J. Michael Brick, WestatW. Sherman Edwards, WestatSunghee Lee, UCLA Center for Health Policy Research

2

Goals of 2005 CHIS Pilot Study

• Main goal was to study methods for interviewing adults on cell-phones, including – Screening for cell-only households.– Feasibility of conducting a 30-minute health

interview on a cell phone.– Within-household adult sampling.– Weighting issues.

• Subsequent research on these issues.

3

Sampling Numbers and Households

• Numbers sampled from cells in Telecordia frame and from RDD pre-identified as cells. – 4,653 from the Telecordia cell frame

– 605 from the RDD sample (0.1% of the sampled numbers).

• Households with a landline were ineligible

– Screen out or non-overlapping frames.

4

Data Collection Procedures

• Modeled on RDD procedures. Same adult CATI interview (averaged = 35 minutes).

• Some key differences:– Lower level of effort (e.g., call attempts,

English-only)– Revised screener introduction (e.g., safety

issues, telephone service items)– Reimbursement ($5/screener; $25/adult)

5

Sampling Adults in Cell-Only Households

• Previous surveys interviewed person who answered the telephone.– Cell phone supplement to 2004 Current

Population Survey reveal sharing of phones, even in cell-only households.

• The CHIS pilot is first to sample 1 adult from a household in a cell phone survey.

6

Adult Sampling Procedure

• When >1 adult in household, screener respondent (SR) was asked if this cell phone was shared: – If not shared then SR was selected.– If shared, then Rizzo method was used.

7

Adult Sampling Procedure

• When >1 adult in household, screener respondent (SR) was asked if this cell phone was shared: – If not shared then SR was selected.– If shared, then Rizzo method was used.

• Potential bias if every adult in hh does not have a cell phone when this cell is not shared. – 8% of SRs reported fewer cells than the

adults in household.

8

Adult Sampling Outcome

• In 1 adult / household - 53 (30%)

• In 1+ adult / household - 123 (70%)

Cell not shared - 109 (89% of 123) Cell shared - 14 (11% of 123)

( 7 SRs and 7 nonSRs sampled)

9

Response by Sampling Procedure

• 1 adult sampled in 176 households (2 ineligible) and 99 completed the adult interview.– 99 of 168 SRs responded (59%)– 0 of 6 non-SRs responded

• Should we ignore sampling other household members and take whoever answers the phone?

10

Sampling NonSRs

• A sample size of 6 is too small a number to draw firm conclusions.

• 4 of 6 nonSRs did not speak English and were not be interviewed for this reason

• Proposed sampling procedure converts a coverage problem into a nonresponse one, and this may be easier to adjust for by weighting.

11

Discussion

• The study shows that conducting a full interview on the cell phone is feasible, but lower screener response rates expected.

• The sampling procedure worked reasonably and selected persons not covered otherwise.

• The ability to get the nonSRs to respond needs to be assessed in a larger study.

12

2008 UpdateSampling and Weighting Cell Phone Surveys to Supplement

RDD SurveysJ. Michael Brick, WestatW. Sherman Edwards, WestatIsmael Flores Cervantes, WestatSunghee Lee, UCLA Center for Health Policy Research

13

Response by Sampling Procedure

• CHIS 2005– 59% SRs responded– 0% non-SRs responded (0 of 6)

• CHIS 2007– 67% of SRs responded– 25% non-SRs responded

14

2007 Adult Response Rates

• 1 adult households – 54% • 2 or more adults in household – 65%

– Not shared – 68% – Shared cells – 36%

• SR sampled – 46%• Not SR sampled – 25%

15

Weighting Cell Samples

• In every cell phone survey, cell-only households have a higher probability of responding.

• Result is over-estimating percentage of frequent cell users (cell-only especially).– Size of bias depends on the differences in

characteristics between frequent and non-frequent cell users.

16

Nonresponse Problem

• CHIS 2007 unweighted distribution of households with both cells and landlines.

Of all calls, how many on cell? RDD sample Cell sample

All or almost all 23% 33%

Some on both 41% 48%

Very few or none 36% 19%

17

Example of Potential Bias

• Assume households with cells have the following distributions.

Cell usage Percent Response rate

Cell-only 18% 41%

All or almost all 18% 37%

Some 35% 27%

Very few 29% 13%

18

Example in Screening Sample

• With standard nonresponse adjustment the effect would be to estimate.

Homes with cells

Cell usage Actual Estimated

Cell-only 18% 36%

19

Example in Dual Frame Sample

• Standard NR adjustment estimates

Homes with cells

Cell usage Actual Estimated

Cell-only 18% 36%

All or almost all 18% 33%

Some 35% 48%

Very few 29% 19%

20

Implications for Weighting Cell Samples

• In screening samples, controlling to cell-only population may reduce nonresponse bias.

• In dual frame survey, controlling to cell-only or total cell population are not sufficient; need to control for usage.– More comprehensive statistical evaluation

needed.

21

Discussion

• Sampling cell phones has revealed a real weakness in our dual frame theory in that it does not handle other survey errors well.

• We are making progress and learning as we go.