Likely Voters IV – The Gallup Model

October 27, 2004October 1, 2019 Mark Blumenthal16 Comments

So how do pollsters select likely voters?

The best place to start is the Gallup likely voter model, the granddaddy of them all. Gallup is also worthy of special scrutiny for other reasons: It is easily the best-known brand name in survey research. Its campaign polls, conducted in partnership with CNN and USA Today, receive more attention and arguably have greater influence than other polls over campaign coverage. Finally, Gallup’s methodology has also been the object of far more criticism this year than any of the others.

Before reviewing the Gallup model and its shortcomings, I want to strongly emphasize one point: We are able to nitpick their model largely because Gallup has been extraordinarily open about their internal procedures, more so than other pollsters. They have patiently answered questions from the most critical of outsiders. They routinely turn their raw data over to the Roper Center after each election, where academics can scrutinize their methods and search for flaws. That Gallup has been punished, in effect, for its openness has not been lost on competitors who remain considerably less forthcoming. So while it is appropriate to question Gallup’s model, we ought to give them credit for their transparency. By opening themselves up to criticism this way, they are advancing the art and science of survey research.

Gallup has been open about its methods from the start. In 1960, Paul Perry, Gallup’s president and research director, published an article in Public Opinion Quarterly detailing their election poll methodology (“Election Survey Procedures of the Gallup Poll,” vol. 24, pp. 531-542). Then as now, respondents tended to over-report their true voting intentions, so selecting likely voters was not a matter of simply asking, “will you vote?” To identify the true “proportion of the population old enough to vote who will vote,” Perry used internal validation studies that compared respondents’ answers to their actual vote history. During the 1950s, Gallup sent its interviewers to vote registrar offices after each election to check whether their respondents had actually voted.

While no single question perfectly predicted whether a respondent would vote, Perry combined a series of questions “related to voting participation” into a 1-7 point scale that was highly predictive of actual turnout: “The system is such that the greater their likelihood of voting, the higher their score. Respondents are then ranked on the basis of their scores.” Perry first set aside those who said they were not registered because their studies had shown that only “a negligible percentage of them vote, something on the order of between 1 and 5 percent.” Then he used the index to select a subgroup of the highest scoring respondents whose size matched the proportion of adults that typically voted in each election. In presidential and congressional elections from 1950 to 1958, the model reduced the average “deviation” from reality on Gallup’s polls from 2.8 among registered voters to 1.1 percentage points among likely voters.

Although Gallup has made minor modifications, the questions and procedures that Perry described 44 years ago remain in use by the Gallup Poll today. Among those who say they are registered to vote (or who plan to do so before the election), Gallup uses the following questions to create a scale that varies from 0 to 7:

2) Do you happen to know where people who live in your neighborhood go to vote? (Yes = 1 point)

3) Have you ever voted in your precinct or election district? (Yes = 1 point)

4) How often would you say you vote, always, nearly always, part of the time or seldom (Always or nearly always = 1 point)

5) Do you plan to vote in the presidential election this November? (Yes = 1 point)

6) In the last presidential election, did you vote for Al Gore or George Bush, or did things come up to keep you from voting?” (Voted = 1 point)

7) If “1” represents someone who will definitely not vote and “10” represents someone who definitely will vote, where on this scale would you place yourself? (Currently 7-10 = 1, according to this “quiz” on USA Today)

A few additional notes: They automatically exclude from the likely voter pool anyone who says they do not plan to vote (on #5). They also give anyone 18-24 an extra point, to help make up for having said they did not vote in the last election (perceptive readers will immediately sense a problem here — I’ll take that up in the next post).

According to Gallup’s David Moore, they aim this year to select a pool of likely voters equal to 55% of their adult sample – their estimate of the appropriate “turnout ratio” likely in this election. In practice, the percentage that scores a perfect 7 out of 7 typically comes very close to 55%. If it ever goes over, they will tighten the scoring of the last question about likelihood to vote (giving a point to those who answer 8-10, for example, instead of 7-10), so that likely voters will always be some combination of sixes and sevens this year.

The one hitch is that they usually have more than enough sixes to bring the total size of the likely voter pool to 55%. So Gallup weights down the sixes to make the weighted value of the likely voters equal to 55%. An example makes this easier to follow: (although the following numbers are totally hypothetical – I made them up): Suppose the pool of those scoring 7 out of 7 is 50%, and the sixes are 10%. They would then weight down the value of the sixes by half (multiply times 0.5): 50% + (10% *0.5) = 55%.

What if the sevens are 50% and the sixes are 15%? They would weight the sixes by 0.33: 50% + (15%*0.33)=55%. Make any sense?

Important concept: Gallup does not claim that this model perfectly predicts who will vote, only that the pool of likely voters consists of those most likely to vote. They also designate some voters as likely and others as not likely. In these two respects, their model is consistent with virtually other pollster. From there, however the way pollsters pick likely voters diverges in a big way.

In the next post, the shortcomings and critiques of the Gallup model..

[See other discussions of Gallup’s seven question model by the Wall Street Journal, Salon.com and Ruy Teixeira]

Mark Blumenthal

Mark Blumenthal is the principal at MysteryPollster, LLC. With decades of experience in polling using traditional and innovative online methods, he is uniquely positioned to advise survey researchers, progressive organizations and candidates and the public at-large on how to adapt to polling’s ongoing reinvention. He was previously head of election polling at SurveyMonkey, senior polling editor for The Huffington Post, co-founder of Pollster.com and a long-time campaign consultant who conducted and analyzed political polls and focus groups for Democratic party candidates.