Poll of Polls

Divergent Polls Interpreting Polls Legacy blog posts The 2004 Race
Mark Blumenthal11 Comments on Poll of Polls

Alert reader JM tipped me off to the fact that Bill Schneider has reprised his “poll of polls” summary on CNN’s Inside Politics. According to the transcript, Schneider combined the results of five surveys released this week by Gallup, ABC News, Investor’s Business Daily/TIPP, Time and Newsweek and showed George Bush ahead of John Kerry by an average of five points (50% to 45%). Fair enough. But then Schneider added this: “Last week’s poll of polls showed Bush just one point ahead.” The contrast certainly implies a big shift in Bush’s favor.

Now, I’m glad that CNN is willing to look at a broad cross-section of surveys rather than focusing exclusively on the Gallup survey they sponsor with USA Today. I have certainly urged readers to average polls as a way of reducing random variation, although I have emphasized averages that make apples-to-apples comparisons over time. Unfortunately, Schneider’s two weekly averages were not comparable. I do not know which surveys he used last week (and I can find no reference to a “poll-of-polls” in the Inside Politics transcripts from last week), but they were certainly different since neither Time nor Newsweek released surveys last week.

Has there been a sea change in the race over the last week? The Gallup/CNN/USA Today poll certainly suggests as much, having flipped in just one week from a one-point Kerry lead (49% to 48%) to an eight-point Bush blowout (52% to 44%). This Gallup result gets support from the latest Newsweek survey, which shows Bush with a similar six-point lead among likely voters (50% to 44%; although the last Newsweek survey, conducted just after the first debate, did not report results for likely voters).

However, consider the following survey results for likely voters released in the last few days compared to comparable surveys from the previous week (except Time, which was conducted 10/6-7; all are based on vote questions that include Ralph Nader).

Only the IBD/TIPP survey shows a slight, but statistically insignificant increase in Bush’s lead, from one point (45% to 44%) to three points (47% to 44%) over the last week. [These results are for the two-way vote question without Nader — see note below]

Others show no change or statistically insignificant movement to Kerry:

  • ABC News – Bush lead goes from four points (50% to 46%) last week to three points (50% to 47%) today.
  • CBS News – goes from a three-point Bush lead (48% to 45%) to a two-point Bush lead (47% to 45%)
  • Time – Bush’s retains a one one-point (46% to 45%) although both candidates gained (48% to 47% reported 10/6-7)
  • Zogby – reported a 45%-45% dead heat on October 13 and the same 45%-45% margin yesterday.
  • The Rasmussen automated surveys have gone from 47%-46% Bush lead reported October 12 to a 47%-47% dead heat yesterday.
  • The Democratic polling group Democracy Corps shows Kerry gaining, going from a 48%-48% dead heat to a 50% to 47% Kerry advantage.

(All of the numbers are available at the Polling Report and Rasmussen Reports).

So what about the Gallup and Newsweek surveys? As I will describe in much more detail in a few days, both appear to use a very similar method to identify “likely voters,” a method that has been the object of much criticism and debate. As Ruy Teixeira and Alan Abramowitz have pointed out (and I agree with them on this point), among registered voters, both surveys show much less change:

  • The Gallup survey went from showing an even race last week (48% to 48%) among registered voters to a three-point Bush lead (49% to 46%) now
  • The Newsweek survey went from a two-point Kerry advantage over the weekend following the first debate (49% to 46%) to a one point Bush advantage now (48% to 47%)

So consider a different set of averages that makes an apples-to-apples comparison using all nine of the surveys mentioned above, substituting results among registered voters for likely voters for Gallup and Newsweek. Before last week’s debate, Bush led by one point (47% to 46%). Now he leads by two (48% to 46%). That’s a gain for the President, but a very small one.

[Update: In citing results from the IBD/TIPP poll above, I inadvertently used the Bush-Kerry two-way vote question above. The change on the question that includes Ralph Nader was even smaller over the last week, Bush’s lead grew from three points (47% to 44%) to four points (49% to 45%) over the equivalent days. Note that this morning’s release from TIPP has the margin closing slightly: Bush ahead by two on the 3-way choice (48% to 46%, 2% for Nader) and dead even (45% to 45%) on the 2-way choice)]

Mark Blumenthal

Mark Blumenthal is the principal at MysteryPollster, LLC. With decades of experience in polling using traditional and innovative online methods, he is uniquely positioned to advise survey researchers, progressive organizations and candidates and the public at-large on how to adapt to polling’s ongoing reinvention. He was previously head of election polling at SurveyMonkey, senior polling editor for The Huffington Post, co-founder of Pollster.com and a long-time campaign consultant who conducted and analyzed political polls and focus groups for Democratic party candidates.

11 thoughts on “Poll of Polls”

  1. Mark:
    The latest Gallup poll had a LV sample that self-identified with a 3% GOP edge, and a RV sample that self-identified with a 2% GOP edge, neither one of them representative of RVs or LVs now. Yet Gallup’s lapdogs CNN and USAT reported the results as if there was a sea change in the race this week. In fact, if you reweight the LV sample along the lines of the 2000 exit polls, then Kerry is ahead.

  2. I don’t think calling it a ‘sea change’ is accurate. I’ve been watching the RCP poll averages and there seems to have been a move of less than 3 points toward Bush. Meaning less than a 1.5% move toward Bush and against Kerry.
    How much of it was due to skewed samples I don’t know. Well within the margin of error of the individual polls for sure. It’s still a horserace….

  4. Actually Don, it is misleading. Look at where the line indicating a trend for Kerry starts right at the beginning of the polling period. Look at where the line for Bush starts, where he is at his lowest. In other words, the line indicating a trend starts at the point best suited to show a Bush upward trend.

  5. The Gallup Poll was way off the mark during this time in the 2000 election. In the October 27, 2000 poll of likely voters, Bush was leading Gore 52% to 39%, while at the same time Zogby had the race 45%-43% for Gore. During this election, Gallup and Newsweek have almost always been the only polls to show Bush with a statistically significant lead. Zogby’s polls have had the race tied at 45% for the last two days, while the NYT/CBS poll is showing Bush with an approval rating of 44% and the race essentially tied. Given Gallup’s history in this election (remember Bush’s 13 point lead a month ago) and the last to be so far off the mark, I would assume that the Zogby poll is one of the most reliable indicators of the status of the horserace.

  6. Sir, Mark,
    Ali Karim Bey here. My prediction since fall 2002 was two-fold:
    – JFK would win the nomination. (He did.)
    – JFK would defeat Bush by 5% (49-44).
    Now, I am willing to let you know the hot tip. The first poll that come AFTER 10/28 (JFK last meet-up) will show that JFK is leading.
    GOP will not get back the White House for another 16-20 years.
    AKB

  7. Spitballing the election

    With the election so heated that crack cocaine is being used as an inducement to register voters (hat tp to Orin Kerr) and with the polls bouncing around all over the place, predicting the outcome is fraught with peril (for…

  8. Sbzuck, Eyeballing den Beste’s plot I was struck by the fact that if he had started the Kerry track at the earliest Kerry low point, as he did the Bush track, the slope of the Kerry plot would be practically the same.
    Of course the slope were not Steve’s point; rather he was arguing that the September numbers which had Bush surging ahead were an anomaly which he contends, wrongly I think, was a deliberate attempt to skew the Bush numbers up to set the stage for a big Kerry comeback in October with its attendent band wagon effect.
    I comment further here,
    http://reviewing.blogspot.com/2004/10/denbeste-looks-at-pollsit-is-great-to.html

  9. The polling system is no longer truely representative (excludes too many people for various reasons – technical and otherwise) My bet is that Kerry is ahead and will win.

  10. In 2000, Gore should have won by 10-12 points, he was effectively the incumbent following eight years of a growing economy. The polls didn’t show the effect of effective incumbency. I’d bet the following: Bush wins by about 4-5 points and a much larger electoral college win. Something the polls are not showing that is dangerous for us Democrats: while our voter identification has not increased materially, GOP identification has. I think most polls are underestimating Bush’s actual lead.

  11. Party ID is not necessarily a good indicator. Lots of Democrats voted for Reagan, and lots of Republicans voted for Clinton. It’s not that it doesn’t matter at all. It’s just not the number one factor, IMO. But the issue of identity is important in a larger sense. Currently, the GOP is spectacular at the politics of identity. Clinton got very close to it. That’s one reason why Kerry is having so much trouble. He focuses on issues and ignores identity. It’s a tough one for Democrats. The rural identity is one of exclusion and victimization by smart, educated, coastal elites that spit on their ancient virtues of faith, etc. No matter that the GOP is run by that same coastal elite. The GOP just knows that cultural identity trumps issues more often than not.

Comments are closed.