Two important reports were released over the last few days on the growing number of Americans with a mobile phone but no “wired” phone service and the implications of this trend for the accuracy of telephone surveys. Last Friday, the National Center for Health Statistics (NCHS) released an update of their respected estimates of the size of the population without landline telephones. Yesterday, the Pew Research Center released the results of an innovative study based on two samples: one of Americans with landline telephones and another with only cell phones.
The growing number of cell-phone only households has the potential to affect telephone surveys because most survey organizations do not include wireless phones in their samples. I discussed this issue in detail in two posts (here and here) during the 2004 campaign and a follow-up based on exit poll results.
So what does a government health survey have to do with cell phones? Well, first, lets get the acronyms out of the way. The National Center for Health Statistics (NCHS), an agency within the Centers for Disease Control and Prevention (CDC), regularly conducts the National Health Interview Survey (NHIS) to gather data on the health status of Americans. The connection to mobile phones is that NCHS conducts the NHIS survey in person using the most rigorous and reliable sampling design to regularly interview randomly selected Americans in over 3,000 households per month. Since NHIS interviews in person, the sample represents Americans with and without telephone service. Since 2003, NHIS has asked questions that identify the type of telephone service in the household. Their huge sample size allows for some of the most precise data available on the mobile-phone-only phenomenon.
The latest data from NCHS, gathered from July to December 2005, show that approximately 8.4% of American households, or approximately 7.8% of all adults, do not have a traditional landline but do have a mobile phone. As the chart below shows, the size of the mobile-phone-only population has doubled since late 2003. The linear upward trend shows no signs of slowing.
The characteristics of those living in wireless-only households reported in this update are similar to what NCHS showed in previous releases. Americans with wireless-only phone service are more likely to rent their homes, live with unrelated roommates, be 18-25 years of age, to be male, Hispanic or live in poverty.
How does the rapidly growing “coverage” problem affect the accuracy of conventional telephone surveys? To try to get a better answer, the Pew Research Center conducted a survey in partnership with the Associated Press and America Online that conducted two parallel studies that asked identical questions on politics and policy: One was a standard telephone survey with a sample that did not include cell phones. The second was an innovative sample of cell phone users that included a “cell phone number frame” that allowed the Pew Center to conduct 750 interviews over the respondents mobile phone. Roughly a quarter of those interviewed by mobile phone (200 respondents) said their cell phone was their only phone.
Here is Pew’s bottom line:
[Cell only Americans] are younger, less affluent, less likely to be married or to own their home, and more liberal on many political questions.
Yet despite these differences, the absence of this group from traditional telephone surveys has only a minimal impact on the results. Specifically, the study shows that including cell-only respondents with those interviewed from a standard landline sample, and weighting the resulting combined sample to the full U.S. public demographically, changes the overall results of the poll by no more than one percentage point on any of nine key political questions included in the study.
Estimates of the respondents’ likely congressional vote this fall, approval of President Bush, opinion about the decision to go to war in Iraq, and other important social and political measures are unaffected when cell-only respondents are blended into the sample. The relatively small size of the cell-only group, along with the demographic weighting performed when it is combined with the landline sample, accounts for the minimal change in the overall findings.
The full Pew report (available in either HTML or PDF format) is definitely worth reading for those concerned about this critical issue affecting political surveys.
PS: The AP’s Will Lester also has coverage of this study.
PPS: Both of these studies were released in conjunction with the annual conference of the American Association for Public Opinion Research (AAPOR), which will be held in Montreal later this week. MP is an active member of AAPOR and will be on hand for the conference, which will mean an altered posting schedule this week. More details on that tomorrow.
Another reason this may have a minimal impact on the results is that the demographic that describes the cell-phone-no-landline respondent is also one of the demographics that is least likely to vote.
How to conduct a phone survey
Conducting a survey is difficult. People give you nonsensical answers, they hang up on you, and sometimes they just are away from the phone. To the extent that these people you miss are different from the ones that do respond,…
This is a very interesting article. Thanks for the post. I look forward to your future work. Thanks.
JackR beat me to it. The number of people in that demographic who vote is probably vanishingly small.
Although currently it has little impact, the future impact may increase to a statistically significant point as this population (1) ages, and (2) comprises a large percentage of the population.
One thing I don’t get: how do pollsters really know they aren’t calling wireless phones? Doesn’t number portability — the ability to move your number from a wireline phone to wireless (and vice versa) — make it increasingly difficult to predict with confidence, let alone know with certainty, what kind of service is associated with that number?
Handicapping the House: Part I
With the addition of House race data to Pollster.com, it is a good time to talk about the difficulty of measuring the status of the race to control Congress at the district level. Political polling is always subject to a…