cell-phone-only voters in the 2008 exit …web.utk.edu/~dfolz/poq cell phones.pdfpublic opinion...

Public Opinion Quarterly, Vol. 73, No. 5 2009, pp. 845–865

CELL-PHONE-ONLY VOTERS IN THE 2008 EXIT POLLAND IMPLICATIONS FOR FUTURE NONCOVERAGEBIAS

MICHAEL MOKRZYCKI∗

SCOTT KEETERCOURTNEY KENNEDY

Abstract Amid growing concern about potential noncoverage bias inrandom digit dial survey samples that exclude cell phones, a nationalface-to-face exit poll provided an opportunity to reach November 2008Election Day voters regardless of telephone status and to evaluate howthe cell-only subgroup has changed since the 2004 election. The NationalElection Pool’s survey found a sharp increase in cell-only incidence,comparable to trends for the general public in government surveys, withcell-only status approaching the norm for voters under age 30. But vot-ers age 30 and older actually abandoned landlines at a faster rate, andthe difference in presidential vote preference between the cell-only andlandline-accessible voters in this age group was even greater than foryounger voters. This suggests that typical poststratification weightingadjustments for age may be less likely to mitigate noncoverage bias infuture landline-only RDD surveys.

In the past five years the survey research profession has devoted extensivestudy to the question of whether landline random digit dial (RDD) samplesmay suffer noncoverage bias by omitting people who only have cellular phones(see e.g., American Association for Public Opinion Research Cell Phone TaskForce 2008).1 The potential problem attracted considerable attention fromnews reporters and commentators in the 2004 presidential election and this

MICHAEL MOKRZYCKI is an independent consultant, 168 Middle Street, West Newbury, MA 01985–1926, USA. SCOTT KEETER is with Pew Research Center, 1615 L Street NW, Suite 700, Washington,DC 20036, USA. COURTNEY KENNEDY is with the University of Michigan, Program in SurveyMethodology, 426 Thompson Street, Room 4050, Ann Arbor, MI 48104, USA. Address corre-spondence to Michael Mokrzycki; e-mail: [email protected].

1. At least twenty-five papers presented at the 2009 annual AAPOR meeting focused entirely orlargely on aspects of cell phones in survey research, according to the conference program. And aspecial 2007 issue of Public Opinion Quarterly 71(5) was devoted to the subject.

doi:10.1093/poq/nfp081C© The Author 2009. Published by Oxford University Press on behalf of the American Association for Public Opinion Research.All rights reserved. For permissions, please e-mail: [email protected]

846 Mokrzycki, Keeter, and Kennedy

continued in 2008.2 However, until 2008, empirical evidence largely reassuredresearchers the impact of cell phones on survey quality was minimal for mosttopics. While the number of cell-phone-only Americans had steadily increased,it remained a small fraction of the general population. Certain subgroups weremuch more likely to be cell-only (most notably younger people) but differenceson demographic and attitudinal measures were small between them and theircohorts who still had landlines. A 2006 study found that typical poststratifica-tion weighting techniques in landline RDD samples produced survey estimatesthat hardly differed nominally, much less to a statistically significant degree,from blended landline and cell phone samples (Keeter et al. 2007). And analysisof the 2004 Election Day exit poll found that cell-only and landline-reachablevoters differed little within age cohorts (Keeter 2006).

Yet, substitution of wireless for landline telephone service has increasedunrelentingly since those studies were conducted. By the second half of 2008,fully one in five U.S. households was cell-only according to the in-personNational Health Interview Survey (NHIS) (Blumberg and Luke 2009). Concernabout potential noncoverage bias prompted a number of national pollstersto include cell phone samples in their pre-election surveys during the 2008campaign. And these more recent dual-frame telephone studies began to suggestthese concerns may have been warranted. For example, in September 2008 thePew Research Center discerned a pattern in its national dual-frame telephonepolls on preference in the presidential race: including a sample of cell phoneinterviews produced a 2- to 3-point shift in the margin toward Democrat BarackObama and away from Republican John McCain in all three surveys (Keeter,Dimock and Christian 2008a). Other survey firms found directionally similarresults (Benford et al. 2009, Langer et al. 2009) although the differences werejust shy of statistical significance. A later Pew analysis of combined datafrom its six pre-election polls from September through the weekend before theelection found a statistically significant but small 2.4-percentage point decreasein Obama’s lead when cell phone interviews were excluded (Keeter, Dimockand Christian 2008b).

Fortunately, we find that final pre-election polling in 2008 was generally ac-curate in the end, with surveys that excluded cell phones and those that includedthem performing equally well compared to actual vote for president. Despitethe sizeable coverage gap, weighting adjustments appeared to have correctedfor the growing bias. But new evidence presented in this study indicates this

2. A search of the keyword combination of “cell phones,” “pollsters” and “polls” in the Nexisdatabase for the four weeks before the 2008 election yielded 160 stories in major newspapers,news magazines or broadcast media; the same search of Factiva for the comparable time periodin 2004 found 150 stories (Keeter 2006). In both cases very tight search criteria likely producedconservative estimates, and the search parameters excluded blogs. In 2008 some media referencesmay have been reports of results of surveys that began including a cell phone frame, but manyreports questioned whether the omission of cell phones from RDD samples might bias surveyestimates.

Cell-Phone-Only Voters in the 2008 Exit Poll 847

situation is not likely to last indefinitely. The seeming discrepancy between thesmall noncoverage bias identified in recent dual-frame studies and their gener-ally good performance in comparison with the single-frame final pre-electionpolls is a puzzle with many possible solutions, but not a cause for future com-placency. And aside from how accurately they predict vote preference, surveyssuffer a growing validity challenge on their face when they exclude one-fifthor more of the population of interest.

The national exit poll—conducted mostly in person outside voting places onNovember 4, 2008—offered an opportunity to reach a sample of all ElectionDay voters regardless of their telephone status. As in 2004, the news mediaconsortium that commissions the exit polls included a question on the 2008national exit poll that allows analysis of attitudinal and demographic differencesbetween voters who only have cell phones and those who have landline service.The sharp increase in the incidence of cell-only Election Day voters—from 7percent in 2004 to 20 percent in 2008—enhances the statistical power of thelatest subgroup analysis (with nearly 1,500 cell-only respondents). These exitpoll data allow us to examine how the relationship between telephone serviceand voting behavior has changed since 2004, as well as whether demographicweighting to adjust for noncoverage of the cell-only voters remains effective.

We proceed by first assessing the impact of cell-phone-only voters in the pre-election polls, comparing the performance of dual-frame telephone polls withsingle-frame polls sampling only landline numbers. We then present findingsfrom respondent-level analysis of our primary data source, the Election Daynational exit poll. We find evidence that the risk of noncoverage error in landlinesurveys has increased since the 2004 election because the voting behaviorof landline and cell-only respondents within age groups is less similar thanit was four years ago. Somewhat surprisingly, the difference in presidentialvote preference between cell-only and landline-accessible voters was greateramong older voters than among younger voters in 2008. We close by drawingimplications for future telephone surveys.

The Predictive Accuracy of Dual- versus Single-frame 2008Pre-election Polls

To begin to understand the consequences of cell phone noncoverage in 2008,we reviewed fifteen national telephone polls with RDD sampling conductedduring the final week of the presidential campaign. More than half of thesepolls were conducted for news media outlets; the rest were conducted byprivate or academic survey organizations for marketing or analytic purposes.Results for these polls were released to the public and summarized on politicalpolling websites including Pollster.com and RealClearPolitics.com. Six useda dual-frame design and nine sampled landlines only. All the polls projectedthe correct winner, but given Obama’s 7-point victory margin, this is perhaps


Table 1. Accuracy of Pre-Election Polls by Sample Design

Obama McCain A s.e.(A)

Election result 52.9 45.7Landline telephone RDD

FOX News/Opinion Dynamics 50 43 0.00 0.07CNN/Opinion Research Corp. 53 46 0.00 0.08American Research Group 53 45 0.02 0.06IBD/TIPP 52 44 0.02 0.07Rasmussen 52 46 −0.02 0.04George Wash. U. (Lake/Tarrance) 49 44 −0.04 0.08Diageo/Hotline 50 45 −0.04 0.07Research 2000 51 46 −0.04 0.06Marist College 52 43 0.04 0.07

Mean A for landline RDD 0.007Mean absolute value (A) for landline RDD 0.026

Landline and cell telephone RDDIpsos/McClatchy 53 46 0.00 0.07Pew Research Center 52 46 −0.02 0.04NBC News/Wall Street Journal 51 43 0.02 0.07ABC News/Washington Post 53 44 0.04 0.04CBS News/New York Times 51 42 0.05 0.08Gallup 55 44 0.08 0.04

Mean A for dual-frame RDD 0.027Mean absolute value (A) for dual-frame RDD 0.036

NOTE.—Means are based on A values prior to rounding.SOURCE.—Pre-election poll estimates compiled by Pollster.com and RealClearPolitics.com.

to be expected. To gauge the relative performance of the polls, we computedthe A value for each (Martin, Traugott and Kennedy 2005). The measure Asummarizes the degree to which the poll estimate of the relative vote share ofthe top two candidates deviated from the election result. A values closer to zeroreflect greater accuracy than those farther from zero in either direction. Valueswith a positive sign reflect overestimation of support for Obama (and under-estimation of support for McCain) relative to the outcome. Negative valuesreflect overestimation of support for McCain (and underestimation of supportfor Obama). The results are presented in table 1. Several of the polls allocatedthe percentage of undecided voters to the candidates in their final estimates.A is not altered if the undecided voters are dropped, allocated proportionally,or kept as undecided. Of course, there are other differences among the pollsthat could affect their accuracy, such as how the data are weighted or how theyscreen for likely voters. We cannot account for these differences.

As seen in table 1, landline polls and dual-frame polls differed relatively littlein accuracy by this measure. The mean of the absolute values of A was 0.026 for


the landline RDD polls versus 0.036 for the dual-frame polls. At the same time,the observed pattern taking account of the sign of A is consistent with the findingin pre-election polls that dual-frame surveys have a higher vote share for Obama.Of the nine landline-only polls, four had negative values for A (signifying anunderestimate of Obama’s margin). By contrast only one of the dual-framesurveys had a negative value for A. Nonetheless, judged by predictive accuracythe dual-frame surveys did not outperform single-frame surveys.

Design of the 2008 National Exit Poll

Although dual-frame pre-election surveys provide a way to gain insights intothe thinking and demographics of cell-only voters, the national exit poll isa unique source of data because it doesn’t rely on telephones to reach re-spondents and it only interviews those who actually have voted. The exit pollalso asks a wider range of questions about political attitudes and behaviorthan in-person government surveys that provide benchmarks of the size of thecell-only population. In the U.S. general elections of 2004 and 2008, the Na-tional Election Pool (NEP)—ABC, The Associated Press, CBS, CNN, FOXand NBC—commissioned surveys of voters in all fifty states and the Districtof Columbia from Edison Media Research and Mitofsky International, as wellas a national sample. Interviews predominantly were conducted in person onElection Day in a probability sample stratified by geography and past vote.(See Appendix for methodological details.)

In recent years, a growing number of voters in many states have cast ballotsearly or absentee before Election Day. By November 2008 nearly a third of allvotes (32.7 percent) were cast before Election Day, up from less than a quarter(22.5 percent) in 2004 (Associated Press 2009). To cover early voters, the in-precinct “exit” polls—which interview voters as they exit the voting booth—have been supplemented in numerous states and nationally by telephone surveysconducted the week before the election. Note that all analysis herein is basedsolely on voters who cast ballots on Election Day in the national sample of300 precincts, as the 2008 NEP telephone surveys—in eighteen states plus anational phone sample—only covered households with landline phones. Ananalysis of possible effects of excluding cell-only voters from the NEP earlyvoter telephone samples is beyond the scope of this paper; that noncoveragehas no bearing on the population being studied here, Election Day voters.

Election Day respondents complete the exit poll by filling out a paper ques-tionnaire. There were four versions of the national exit poll instrument, two ofwhich included the question shown in figure 1.3

3. There were three minor differences in the presentation of the question in 2004: The words “inyour home” were underlined; the parenthetical “Check only one” instruction was not capitalized;and in the second response code there was a comma after the word “regular.”


Figure 1. Telephone-Status Question as It Appeared on the 2008 National ExitPoll Questionnaire.Source.—Election Day national exit poll conducted by National Election Pool,November 4, 2008.

Although this survey offers a unique opportunity to examine the cell-onlyphenomenon, it is far from perfect. For one thing, the wording of the tele-phone status question does not permit identification of “cell-mostly” voters—individuals who have landline phones but mostly rely on their cell phones andthus may be difficult or impossible to reach in landline samples.4 Also, dataare not available on many substantive and demographic variables in the exitpoll, such as marital status or religious affiliation, which were not includedon the same forms as the telephone status question. And with limited ques-tionnaire space, the exit poll did not include measures of some demographiccharacteristics that government studies consistently find to be strong predictorsof cell-only status, such as living in a household with only unrelated adults orrenting rather than owning one’s home.

Results

INCIDENCE OF CELL-PHONE-ONLY VOTERS OVERALL

We first look at the basic incidence of cell-phone-only voters in the exit pollsample in 2008 compared to 2004. The proportion of Election Day voters wholive in cell-only households nearly tripled over four years, to 19.9 percentin 2008 (table 2). This is similar to the finding of the general populationNational Health Interview Survey, which found 20.2 percent of householdshad no landline but at least one wireless telephone in the second half of 2008.Another 4.1 percent of Election Day voters reported in the exit poll that theirhousehold had no telephone service at all, indicating that pre-election polls

4. “Cell-mostly” or “cell-mainly” voters are of increasing interest to survey researchers. Theevidence presented thus far is that the potential for bias from omitting “cell-mostly” respondentsfrom samples is more modest than from noncoverage of those who are “cell-only.” See, for example,Keeter et al. 2009, one of at least six papers presented at the 2009 AAPOR annual conference thataddressed this topic. NEP considered modifying the wording of the telephone status question onthe exit poll to capture “cell-mostly” data but opted instead to keep the 2004 wording in 2008 fortrend analysis with the largest possible sample size.


Table 2. Household Telephone Status in Election Day National Exit Polls

2008 2004

Weighted Unweighted Weighted Unweighted(%) N (%) N

Both regular landlineand cell phone service

62.3 4,576 70.1 3,962

Only regular landlinephone service

13.7 952 20.7 1,087

Only cell phone service 19.9 1,496 7.1 446No telephone service at

home4.1 317 2.1 124

Total 100.0 7,341 100.0 5,619

SOURCE.—Election Day national exit polls conducted by National Election Pool, November 4,2008, and November 2, 2004.

using only landline samples failed to cover about 24 percent of the ElectionDay electorate.5

VOTE FOR PRESIDENT BY TELEPHONE STATUS AMONG ELECTION

DAY VOTERS

We now turn to analyzing how the growing population of cell-only votersdiffers from those reachable by landline. Table 3 shows how Election Dayvoters cast their ballots for president in 2004 and 2008 by telephone status.The 2008 results reinforce the findings from four years earlier: cell-only votersremained a source of outsized support for the Democratic candidate, while theRepublican continued to fare best among Election Day voters with both regularlandline service and cell phones in their household. This is not surprising, giventhat vote for Republicans tends to grow with higher income, and those withhigher income can better afford to have both landline and cell phones.

THE GROWING CELL-ONLY VOTER POPULATION

While much of the focus on cell-only voters remains on the youngest agegroups6, where the incidence rates are very high, the majority of cell-only

5. These estimates are subject to debate. Recent research on list-assisted RDD designs suggestthat actual noncoverage rates for both landline RDD and dual-frame RDD are even higher (Fahimi,Kulp, Brick 2008). Meanwhile, the estimated 4.1 percent incidence of households with no telephoneservice is higher than the NHIS estimate of 1.9 percent in the second half of 2008. Reasons forthis difference may include sampling error, the fact that the sample of Election Day voters coversa smaller portion of the U.S. public than does the NHIS sample of all U.S. households, anddifferences in NEP and NHIS question wording.6. The 2008 NHIS study finds a smaller proportion of all adults age 18–24 (33.1 percent) are inwireless-only households compared to 18–24 voters in the Election Day exit poll (42.2 percent)


Table 3. Presidential Vote by Household Telephone Status in Election DayNational Exit Polls

2008 2004

Obama McCain Kerry Bush(%) (%) (%) (%)

ACTUAL POPULAR VOTE(Election Day + early)

52.9 45.7 48.3 50.7

Estimate among all ElectionDay voters

52.4 46.2 48.4 50.8

Both regular landline and cellphone service

49.2 49.2 46.5 52.6

Only regular landline phoneservice

52.1 46.6 51.6 47.5

Only cell phone service 60.5 37.8 53.5 44.4No telephone service at home 61.1 36.2 58.8 39.5

SOURCE.—Election Day national exit polls conducted by National Election Pool, November 4,2008, November 2, 2004; Federal Elections Commission.

voters are thirty and older. This was true even in 2004, when 51.7 percent ofcell-only voters in the NEP exit poll were at least thirty years old. Of course,part of this is simply the age composition of the voting public: the vast majorityof voters are at least thirty years old—more than 80 percent in both 2004 and2008. The 2008 NHIS study indicates that 21.6 percent of the 30–44 age groupwas cell-only in the last half of 2008 (up from 19.1 percent in the first halfof the year), very close to NEP’s estimate for this age group among voters(20.2 percent cell-only). Table 4 shows these differences in telephone servicefrom 2004 to 2008 by age groups in the Election Day national exit poll. Notethat in this and following tables, the results for “Landline” voters collapse twocategories in the exit poll question: “Both regular land-line and cell phoneservice” and “Only regular land-line phone service.”

The percentage of voters ages 30–39 who were cell-only more than tripledbetween 2004 and 2008, from 7.3 to 23.2 percent. Interestingly, as a percentageincrease over the 2004 baseline this growth rate was even greater than for voters

Blumberg and Luke 2009. While these datasets lack all the necessary variables to draw firmconclusions, one possible explanation for this finding would be that the youngest cell-phone-onlyadults are more likely to vote than their cohorts with landlines. This may be related to a differencebetween young adults away at college and thus more likely to be wireless-only compared to youngcohorts who may still be living at home with parents and thus more likely to have landlines inthe household; in the exit poll, cell-only 18- to 24-year-olds were more likely than their landline-reachable cohorts to be college graduates or have some college education. Another factor in thedifference between the NHIS and NEP Election Day estimates may be differential distributionof cell-only 18- to 24-year-olds between the early voting and Election Day electorates. Questionwording also may contribute to the difference.


Table 4. Telephone Service by Age in Election Day National Exit Polls

18–24 25–29 30–39 40–49 50–64 65+ Total

2008Landline 51.7 51.8 72.2 81.4 87.4 91.1 76.0Cell only 42.2 40.8 23.2 14.8 9.7 7.4 19.9No phone 6.0 7.4 4.7 3.7 2.9 1.5 4.1

100.0 100.0 100.0 100.0 100.0 100.0 100.02004

Landline 77.6 74.5 89.8 94.6 95.6 96.7 91.0Cell only 18.5 20.0 7.3 4.2 3.7 2.3 7.1No phone 3.8 5.5 2.9 1.2 0.7 1.1 2.0

100.0 100.0 100.0 100.0 100.0 100.0 100.0

Cell-only 2008–2004 +23.7∗ +20.8∗ +15.9∗ +10.6∗ +6.0∗ +5.1∗ +12.8∗

Pct increase on base 128% 104% 218% 253% 165% 226% 182%

SOURCE.—Election Day national exit polls conducted by National Election Pool, November 4,2008 November 2, 2004.

∗p < .05.

under age 30. This can be explained in part by aging over four years of thecohort that in 2004 was nominally most likely to be cell-only—those who thenwere twenty to twenty-nine years old. In 2008 the cell-only growth rate inthe survey among the 40–49 age group was even greater than for those aged30–39, even though the absolute change was smaller. Statistically significantincreases also occurred among those 50–64 and 65 and older. With wirelesssubstitution growing rapidly across all age groups, differences in the growthrates by age can have consequences for poststratification adjustments by age inlandline-only RDD surveys.

TELEPHONE STATUS BY OTHER DEMOGRAPHIC AND ATTITUDINAL

VARIABLES

Cell-phone-only status also is related to other key demographic correlates ofelectoral behavior. These relationships are important because of the implica-tions for demographic weighting used to adjust for noncoverage. Keeter (2006)reported how the incidence of cell-only status varied by race, education, incomeand other factors in the 2004 national exit poll. We compared those results tothe 2008 figures to determine how the rate of transitioning to cell-only statuschanged along these dimensions. The results are presented in figure 2. Themost sizable increases in cell-phone-only status occurred for the lowest incomeand least educated voters, and for Latinos. In 2004 some 13 percent of ElectionDay voters with an annual income under $15,000 were cell-only; by 2008 that


Figure 2. Proportion of Election Day Voters Who Are Cell-Only, by Demo-graphics.Source.—Election Day national exit poll conducted by National Election Pool,November 4, 2008 and November 2, 2004. The 2008 standard errors rangefrom 2.0 for the estimate for whites to 8.2 for the estimate for non-high schoolgraduates.

figure nearly tripled to 37 percent. Similarly, 9 percent of Latino or Hispanicvoters in 2004 were cell-only, but this rose to 28 percent in 2008.

Among most Election Day voters, the likelihood of being cell-only declinedas income rose, but in both 2004 and 2008 the trend was not monotonic. Curi-ously, the trend reversed slightly for those with the highest incomes. In 2008 thiscurvilinear effect of income is highly significant in logistic regression modelspredicting cell-only status, with and without controls for other demographics.The mechanism behind this pattern is unclear because the ability to afford land-line service is clearly not a factor for those at the highest income levels. Onepotential consequence of this phenomenon may be to diminish the reliabilityof landline-only surveys that target high-income population subgroups.


Table 5. Presidential Vote by Telephone Service in Election Day National ExitPolls

2008 Landline Cell only Diff. 2004 Landline Cell only Difference

18–24 Obama 61.2 68.6 −7.4 Kerry 59.5 62.5 −3.0McCain 36.8 29.3 7.5 Bush 39.3 36.5 2.8Other 2.1 2.1 Other 1.2 1.0

100.0 100.0 100.0 100.0

25–29 Obama 63.3 65.4 −2.1 Kerry 51.5 53.4 −1.9McCain 35.6 33.6 1.9 Bush 46.6 45.5 1.2Other 1.1 0.9 Other 1.8 1.1

100.0 100.0 100.0 100.0

30–39 Obama 51.3 62.6 −11.3∗ Kerry 46.9 49.3 −2.5McCain 47.1 36.2 10.9∗ Bush 52.3 48.0 4.3Other 1.6 1.2 Other 0.9 2.7

100.0 100.0 100.0 100.0

40–49 Obama 46.6 54.5 −7.9McCain 51.2 44.3 6.9Other 2.2 1.1

100.0 100.0

50+ Obama 46.9 46.8 0.1McCain 52.2 52.1 0.1Other 0.9 1.1

100.0 100.0

SOURCE.—Election Day national exit polls conducted by National Election Pool, November 4,2008 and November 2, 2004.

∗p <.05 in difference of proportions test.

PRESIDENTIAL VOTING BY AGE AND PHONE STATUS

Perhaps the most striking finding of the 2008 exit poll question on telephonestatus was how differences in voting patterns between cell-phone-only andlandline-reachable respondents played out among age groups. Differences inpresidential vote between the landline and cell-only groups in 2008 were largestamong voters ages 30–39 (table 5). Obama was the choice of 62.6 percent ofcell-only voters ages 30–39 but only 51.3 percent among those reachable bylandline, a difference of 11.3 percentage points that was significant at the .05level. The difference was smaller and not significant among all other age groupsin the survey: 7.9 points among those 40–49, 7.4 points among those 18–24, andless among those 25–29 and 50+. In 2004 older cell-only voters were such asmall part of the voting population that their voting patterns, however different,


posed little risk of biasing election surveys based on landline samples. This hadclearly changed by 2008.

Among all Election Day voters thirty and older, Obama was the choice of55.5 percent of those who were cell-only and 47.8 percent of the landline-accessible, a significant difference of 7.7 percentage points. Among ElectionDay voters 18–29, the difference was 4.9 points, though not statisticallysignificant. Four years earlier, the difference in vote for Democratic presidentialnominee John Kerry among the 30+ age group was 2.8 points; among those18–29 it was 2.2 points. Although the sample sizes in the poll cannot supporta definitive conclusion, the disjuncture between landline and cell-only votersappears to have grown more among older voters than among younger ones.Because age is typically a poststratification weighting variable, this resultsuggests adjustments by age may be insufficient to overcome noncoverage inlandline-only RDD surveys.

Generally, voter attitudes other than presidential vote choice differed some-what less by telephone status. Table 6 shows cell phone status by party identifi-cation and ideology, within age groups. The differences across telephone statuswithin age groups indicate how well cell-phone-only voters are represented bytheir landline-accessible peers. Several differences are apparent. Among olderElection Day voters (age 30–39 and 40+), those living in landline householdsare significantly more likely to consider themselves conservative than theircell-only peers. Among younger voters, by contrast, telephone status appearsto be more closely related to party identification, though these differences werenot statistically significant.

Although not shown, we analyzed a number of other measures asked on theexit poll questionnaire. Significant differences by telephone status appeared invariables related to partisanship and ideology; for example, cell-only respon-dents were 10 points more likely than landline-reachable voters to stronglydisapprove of George W. Bush’s performance as president. On the other hand,there were no significant differences by telephone status on whether the can-didates’ race or age were factors in voting for president. And measures of theeconomy produced mixed results: there was no significant difference in like-lihood of calling the economy the nation’s most important issue nor in worryabout the economy’s direction over the next year, but cell-only Election Dayvoters were a little more likely than the landline-reachable to say the economycurrently was poor—yet slightly less likely to call it “not so good.” Theseresults suggest it may not be easy to predict how cell phone noncoverage biasmay affect estimates on individual attitudinal items within a survey.

There also were interesting differences by phone status in voter mobiliza-tion, as shown in table 7. Cell-only Election Day voters were far more likelythan those with landlines to have voted for the first time in 2008, though nat-urally this correlates strongly with age (the youngest voters are most likely tobe cell-only and also most likely to be new voters). But this same group alsowas far less likely to have been contacted by either campaign. Even Obama’s


Table 6. Ideology and Party Identification by Telephone Status and Age inElection Day National Exit Poll

Landline phone Cell phone only Difference

Age 18–29Party identification

Democrat 48.4 43.0 5.4Republican 27.9 26.5 1.5Independent 18.6 23.8 −5.2Other 5.1 6.7 −1.7

IdeologyLiberal 32.7 32.2 0.6Moderate 43.6 43.6 −0.1Conservative 23.7 24.2 −0.5

Age 30–39Party identification

Democrat 39.2 45.0 −5.8Republican 36.0 28.3 7.7Independent 20.6 21.9 −1.4Other 4.2 4.8 −0.6

IdeologyLiberal 21.9 32.3 −10.4∗

Moderate 44.3 43.5 0.7Conservative 33.8 24.2 9.6∗

Age 40+Party identification

Democrat 39.9 45.5 −5.6Republican 36.4 32.0 4.4Independent 21.3 17.9 3.4Other 2.4 4.6 −2.2

IdeologyLiberal 18.6 17.3 1.3Moderate 45.2 53.6 −8.3∗

Conservative 36.1 29.1 7.0∗

SOURCE.—Election Day exit poll conducted by National Election Pool, November 4, 2008.∗p < .05 in difference of proportions test.

campaign—renowned for its technological innovation, including its plan touse text messaging to notify 3 million supporters of Obama’s vice-presidentialpick—was more likely to contact landline-reachable than cell-only voters, ac-cording to the Election Day exit poll. (So it’s not only pollsters who have troublereaching those who are wireless-only.) These differences remain statisticallysignificant even within age groups.

Given extensive research showing that mobilization efforts are effective atstimulating voter turnout, these findings have important normative implications.


Table 7. Voter Mobilization by Telephone Status in Election Day National ExitPoll

Landline phone Cell phone only Difference(%) (%) (%)

Is this the first year you have evervoted?Yes 9.8 22.3 −12.5∗

Did anyone call you or talk toyou in person on behalf ofeither major presidentialcampaign about coming out tovote?Contacted by both 18.0 7.1 +10.9∗

Contacted by Obama only 16.4 15.8 +0.7Contacted by McCain only 8.4 2.7 +5.7∗

Contacted by neither 57.2 74.4 −17.2∗

SOURCE.—Election Day exit poll conducted by National Election Pool, November 4, 2008.∗p < .05 in difference of proportions test.

Young people, not surprisingly, are much less likely than older people to havedeveloped habits of political engagement and are in greater need of mobilizationto vote in elections. Yet, because of their telephone status, they are far less likelyto be reached by campaigns, even if the campaigns intend to reach them. Asimilar logic may apply to lower income and minority individuals. This patternalmost certainly will exacerbate political inequality.

POSSIBLE REASONS FOR POLITICAL DIFFERENCES BETWEEN CELL-ONLY

AND LANDLINE-ACCESSIBLE VOTERS

The observed political differences between the cell-only and landline-accessible voters likely stem from the outlined life-cycle and socio-economicdifferences between these two groups, which correlate with political views.Pre-election polls typically collect some indicators of these mechanisms. Age,marital status, and presence of children in the household are commonly usedindicators of life-cycle stage. Income, education, and homeownership are com-monly used indicators of socio-economic status. Many but not all of thesevariables were measured alongside telephone service in the 2008 national exitpoll.

We used logistic regression to better understand how these factors relatein a multivariate setting. The outcome we modeled is vote for Obama amongthose voting for either of the major party candidates. The model was estimatedseparately for those aged 18–29, 30–39, and 40 and older, because we expect


Table 8. Logistic Regression Models Predicting Vote for Obama, by AgeGroupˆ

Age 18–29 Age 30–39 Age 40+Parameter B s.e. B s.e. B s.e.

Telephone serviceNo telephone 0.10 0.26 0.18 0.29 0.30 0.22Cell-only 0.36∗ 0.13 0.58∗ 0.15 0.14 0.12

Male −0.26∗ 0.13 −0.25∗ 0.12 −0.32∗ 0.07African-American 3.44∗ 0.46 2.89∗ 0.32 3.84∗ 0.32Hispanic 0.83∗ 0.20 0.54∗ 0.21 0.64∗ 0.15Education 0.11 0.06 0.26∗ 0.06 0.25∗ 0.04Income −0.26 0.14 −0.29 0.16 −0.38∗ 0.09Incomeˆ2 0.02 0.02 0.02 0.02 0.03∗ 0.01Intercept 0.92∗ 0.32 0.03 0.42 0.34 0.22Model sample size (1,514) (1,428) (3,727)% Predicted correctly 73.5% 64.0% 58.3%

NOTE.—The results do not change substantively in any of the three models when weighting isused.

SOURCE.—Election Day exit poll conducted by National Election Pool, November 4, 2008.∗p < .05.ˆReference groups are “has landline,” female, not African-American, and not Hispanic. Models

are based on Election Day voters voting for either Barack Obama or John McCain. Model estimatesare unweighted so as not to distort the effect of demographic predictors.

the predictors to interrelate differently depending on the voter’s age. We alsowant to maintain our focus on how noncoverage error might arise in landlinetelephone polls if cell-only voters differ from landline-accessible voters withintheir age group, since age is a common variable used in weighting. Presidentialvote was regressed on telephone status (cell-only, no phone, with landline asthe reference group), gender, race, Hispanic ethnicity, education, income, and acurvilinear effect for income. In a preliminary model, we tested for interactioneffects between telephone service and the demographic variables, but none wassignificant. The final model estimates are reported in table 8.

Cell-only status is a highly significant predictor of voting for Obama amongthose 30–39 years old, even when controlling for common weighting variables.It also remains significant for those 18–29, though to a slightly lesser degree.Telephone service is not, however, predictive of an Obama vote among thoseaged 40 and older.

The regression analysis predicting the vote finds that including the avail-able demographic variables—age, sex, education, income and race—reducesthe impact of telephone status somewhat (compared with a model in whichthe demographic factors are not included) but leaves much of the variabilityunexplained. The gap in predicted probability of a vote for Obama between


cell-only and landline-accessible voters, without controlling for demographiccharacteristics, is 10.7 percentage points. Including the demographics in themodel reduces this difference to 6.7 points.

These results suggest that noncoverage bias can persist in landline surveyestimates even when standard demographic weighting is employed. Unfortu-nately, other important variables related to phone status and the vote were notasked on the same forms of the questionnaire that included phone status, in-cluding marital status and presence of children in the home. Religion, whichis associated with social integration and is correlated with phone status, alsowas not asked on the same forms. Homeownership—with renting known tobe a strong predictor of cell-only status—was not asked in the exit poll at all.A similar multivariate analysis using a September 2008 Pew Research Centerpoll found that the independent impact of cell-only status on vote choice dimin-ished to insignificance when marital status and homeownership were includedalong with standard weighting demographics, suggesting that more elaborateweighting protocols might help to reduce noncoverage bias (Keeter et al. 2009).

Conclusions

Considering the trend in telephone usage and the evidence for the potentialgrowth of coverage error, researchers should ensure that the cell-only popula-tion is covered in future pre-election surveys, including the National ElectionPool’s own surveys of early and absentee voters. It may seem odd to make sucha recommendation in the absence of strong evidence of bias in vote preferenceestimates in pre-election polls to date. But the risk of error posed by noncover-age is no longer trivial. And there are other reasons beyond forecast accuracyto improve the coverage of polls.

The percentage of Election Day voters who were cell-only nearly tripledbetween 2004 and 2008, mirroring trends in government surveys of the gen-eral population. There is no evidence that this rapid rate of growth is likelyto diminish in the near future. Because of the sizeable differences betweenlandline-accessible and cell-only voters, the potential for noncoverage bias infuture landline-only surveys may be substantial.

The good news is that it appears that noncoverage bias did not present alarge problem for national pre-election polls in 2008. Surveys that includedcell phones were no more accurate in forecasting the presidential race thanwere those that relied only on landline samples. But warning signs appeared.Several dual-frame pre-election polls in 2008 found that estimates based oncombined landline and cell phone samples tended to increase the estimatesof preference for Barack Obama by 2–3 percentage points, compared with thelandline sample alone. In a race tighter than the 2008 presidential election, evena bias this small could cause some surveys to estimate—at least nominally, if


not necessarily with high statistical confidence—that the “wrong” candidate isahead.

Additionally, to the surprise of many analysts, the potential noncoveragebias is not concentrated solely among the younger voters. Among Election Dayvoters in the national exit poll, the majority of the cell-only population was atleast 30 years old, and larger differences in voter preference by telephone statuswere observed for those 30–39 than for those 18–29—a pattern also seen inPew Research Center pre-election surveys during the fall campaign. The votepreference differences by phone status in the exit poll remained significant ina multivariate analysis that controlled for most of the factors normally usedin weighting pre-election polls. Although this bias was modest in 2008, thepotential for bias may continue to grow as cell-only status becomes morecommon among older age groups.

Another issue is the declining ability of landline-based polls to describeyoung voters with an acceptable level of reliability. As the cell-only populationhas grown, landline samples are obtaining fewer younger adults, leaving manypolls with insufficient samples for such analyses. For example, only 9 percentof respondents in Pew Research’s landline samples were ages 18–29 in 2008,compared with a population parameter of 22 percent (Keeter 2009). Youngvoters were the subject of intense interest and study in the 2008 cycle, giventheir comparatively high levels of engagement in the campaign and the factthat a higher percentage of them supported Obama than in any other age group.Pre-election polls needed to be able to describe these young voters and explaintheir opinions and motivation, but many were limited in their ability to dothis because of the coverage issue. The National Election Pool’s landline-only pre-election polls were similarly constrained from describing how therapidly growing number of young—and older—people who voted early orabsentee compared with those interviewed in the Election Day national exitpoll. Including cell phone samples is an effective remedy for these problems,although admittedly more costly than landline-only samples.

Although not addressed in the current analysis, future survey research mustalso contend with the “cell-mostly” population. There is, as yet, no evidenceof serious coverage bias related to voters who have a landline but rely mostlyon their cell phones. But a better understanding of this population is needed,and the size and characteristics of this group should be monitored closely. Ifnecessary, steps should be taken to ensure such individuals are covered andmeasured appropriately in future pre-election polling and surveys of actualvoters. The inclusion of cell phone samples can address this problem as well.

Correctly forecasting the margin in an election is important for polling, butit is not the only—or even the most important—contribution polls make tothe electoral process. Polls help citizens, the news media and policy makersunderstand how voters are making their judgments and what considerationsanimate them. Polls are critical for informing an understanding of why electionsturn out as they do, and thus have implications for how candidates, once elected,


may govern. For these reasons it is critical that polling fully represent the votingpublic. Ultimately, polls that omit one-fifth or more of voters—voters who aredemonstrably different from those included in standard polling—are not likelyto have the credibility they need, even if they correctly predict the final electiontally.

Appendix: Methodology and Question Wording

The sample for the Election Day national exit poll used in this analysis wasdrawn from the population of all people who voted in person on November 4,2008, in the U.S. general elections. A probability sample of precincts wasdrawn in each of the fifty states and District of Columbia from among allvoting locations in the state; a subsample of the state precincts was drawnto create the national sample. Within each precinct, voters were selected at asystematic interval throughout Election Day so all voters in that precinct hadan equal chance of being interviewed.

Each state sample was stratified by geography and partisan voting history.Precincts were selected with a probability proportionate to the number ofvoters in each precinct, except that in some states, precincts with large minoritypopulations were sampled at a higher rate, and in New Hampshire several verylarge precincts were sampled with certainty. In those cases, sample weightingadjusted representation of precincts to their correct share of the total vote.

Other post hoc weighting of the national exit poll included: weighting totake into account the probability of selecting a precinct; adjustments based onthe number of respondents and interviewers’ tallies of the gender, estimatedage (into three bands) and race (black or nonblack) of voters selected in therandom sampling interval but who were missed or refused to cooperate; forcingof survey estimates of each candidate’s vote share to actual vote received;weighting to targets for age, race and gender; and adjustments based on sizeof vote by region. (For data about the entire electorate, the national ElectionDay exit poll was combined with National Election Pool telephone pollingconducted the week before the election to sample early and absentee voters;this analysis focused only on those who voted in person on Election Day.)

See National Election Pool (2005) for additional methodological detail. Thesampling and interviewing procedures did not substantively change in 2008.Documentation for the 2008 study will be available at the Interuniversity Con-sortium for Political and Social Research (ICPSR) and Roper Center for PublicOpinion Research.

The completion rate for the 2008 Election Day national exit poll was 43.1percent. This rate is the number of voters who completed the survey out of thetotal number selected to take the survey, as tallied by the interviewers. AAPORStandard Definitions do not define how to calculate response rates for in-personexit polling of individuals, only for in-person surveys of households.


Following is the exact wording of the main questions used in this analy-sis. The complete 2008 national exit poll questionnaires are available in thesupplementary data online at http://poq.oxfordjournals.org/.

What type of telephone service is there in your home that you could use orbe reached on? (CHECK ONLY ONE)

1. Both regular land-line and cell phone service2. Only regular land-line phone service3. Only cell phone service4. No telephone service at home

In today’s election for president, did you just vote for:

1. Barack Obama (Dem)2. John McCain (Rep)9. Did not vote0. Other: Who?

To which age group do you belong?

1. 18–24 4. 40–44 7. 60–642. 25–29 5. 45–49 8. 65–743. 30–39 6. 50–59 9. 75 or over

Are you:

1. White 4. Asian2. Black 5. American Indian3. Hispanic/Latino 6. Other

Are you of Hispanic or Latino descent?

1. Yes2. No

(Note: Those who answer “Yes” to the Hispanic/Latino ethnicity questionare counted as Hispanic/Latino in the race demographic even if they selectedsomething else there.)

Are you:

1. Male 2. Female

2007 total family income:

1. Under $15,000 5. $75,000–$99,9992. $15,000–$29,999 6. $100,000–$149,9993. $30,000–$49,999 7. $150,000–$199,9994. $50,000–$74,999 8. $200,000 or more


What was the last grade of school you completed?

1. Did not complete high school2. High school graduate3. Some college or associate degree4. College graduate5. Postgraduate study

No matter how you voted today, do you usually think of yourself as a:

1. Democrat2. Republican3. Independent4. Something else

On most political matters, do you consider yourself:

1. Liberal2. Moderate3. Conservative

Is this the first year you have ever voted?

1. Yes2. No

Did anyone call you or talk to you in person on behalf of either majorpresidential campaign about coming out to vote?

1. Yes, for Barack Obama2. Yes, for John McCain3. Yes, for both Obama and McCain4. No, I was not contacted

Acknowledgements

The 2004 exit poll data are available from a dataset that is archived at the RoperCenter for Public Opinion Research, University of Connecticut. The 2008 dataare from an NEP dataset that at this date remains unpublished; we obtained thedataset as all three authors worked or consulted for NEP members during the2008 elections.

Supplementary Data

Supplementary data are available online at http://poq.oxfordjournals.org/.


References

American Association for Public Opinion Research. 2008. “Guidelines and Considerations forSurvey Researchers When Planning and Conducting RDD and Other Telephone Surveys in theU.S. with Respondents Reached via Cell Phone Numbers.” Available at http://www.aapor.org/uploads/Final_AAPOR_Cell_Phone_TF_report_041208.pdf. Accessed October 7, 2009.

Associated Press Election Research. 2009. “2008 Turnout and Advance Voting.” Unpublishedreport dated April 21, 2009. Cited by permission.

Benford, Robert, Trevor Tompson, Barry Feinberg, Geoff Feinberg, Annie Weber, Nicole Speuldaand Christopher Fleury. 2009. “Wireless and Wireline: Dual-frame Implications for SampleDesign Decisions on Estimates, Weighting and Costs.” Paper presented at the 64th AnnualConference of the American Association for Public Opinion Research, Hollywood, FL, USA,May 2009.

Blumberg, Stephen J., and Julian V. Luke. 2009. “Wireless Substitution: Early Release of Estimatesfrom the National Health Interview Survey, July–December 2008.” National Center for HealthStatistics. Available at http://www.cdc.gov/nchs/nhis.htm. Accessed October 7, 2009.

Fahimi, Mansour, Dale Kulp and J. Michael Brick. 2008. “Bias in List-Assisted 100-Series RDDSampling.” Survey Practice September 28, 2008.

Keeter, Scott. 2006. “The Impact of Cell Phone Noncoverage on Polling in the 2004 PresidentialElection.” Public Opinion Quarterly 70:88–98.

———. 2009. “New Tricks for Old—and New—Dogs: Challenges and Opportunities FacingCommunications Research.” March 3, 2009. Available at http://pewresearch.org/pubs/1138/survey-problems-new-techniques-communications-research. Accessed October 7, 2009.

Keeter, Scott, Michael Dimock, and Leah Christian. 2008a. “Cell Phones and the 2008 Vote:An Update.” Pew Research Center for the People & the Press. Sept. 23, 2008. Available athttp://pewresearch.org/pubs/964/. Accessed October 7, 2009.

———. 2008b. “Calling Cell Phones In ‘08 Pre-Election Polls.” Pew Research Center for thePeople & the Press. Dec. 18, 2008. Available at http://people-press.org/reports/pdf/cell-phone-commentary.pdf. Accessed October 7, 2009.

Keeter, Scott, Courtney Kennedy, April Clark, Trevor Tompson, and Mike Mokrzycki. 2007.“What’s Missing from National Landline RDD Surveys? The Impact of the Growing Cell-OnlyPopulation,” Public Opinion Quarterly 71:772–92.

Keeter, Scott, Jocelyn Kiley, Leah Christian, and Michael Dimock. 2009. “Perils of Polling inElection ‘08.” Paper presented at the 64th Annual Conference of the American Association forPublic Opinion Research, Hollywood, FL, USA, May 2009. Available at http://pewresearch.org/pubs/1266/polling-challenges-election-08-success-in-dealing-with. Accessed October 7, 2009.

Langer, Gary, Peyton Craighill, Patrick Moynihan, Jon Cohen, Jennifer Agiesta, and Dave Lambert.2009. “‘These Nutty Pollsters’: Methodological Issues in ABC News/Washington Post 2008 Pre-Election Polling.” Paper presented at the 64th Annual Conference of the American Associationfor Public Opinion Research, Hollywood, FL, USA, May 2009.

Martin, Elizabeth A., Michael W. Traugott, and Courtney Kennedy. 2005. “A Review and Proposalfor a New Measure of Poll Accuracy.” Public Opinion Quarterly 69:342–67.

National Election Pool, Edison Media Research, and Mitosky International. 2005. National ElectionPool Poll # 2004-NATELEC:National Election Day Exit Poll. Roper Center version. Somerville,NJ: Edison Media Research/New York, NY: Mitofsky International [producers], 2004. Storrs,CT: Roper Center for Public Opinion Research, University of Connecticut [distributor]. Availableat http://www.ropercenter.uconn.edu/ipoll.html.

cell-phone-only voters in the 2008 exit …web.utk.edu/~dfolz/poq cell phones.pdfpublic opinion...

Documents