« Exit Polls: More Questions | Main | Reader Survey Reminder »

November 03, 2004

First Impressions

As with any election, the results leave us much to sift through. Although the final counts are not yet available for all states, we can still reach some conclusions about two issues I raised that have also been the subject of much debate. First, the incumbent rule did not apply to the presidential race. Second, the traditional likely voter models performed reasonably well. There was no hidden turnout favoring John Kerry.

At the national level, as of this hour, George Bush is holding a three-point margin (51.4% to 48.3%) over John Kerry. The average of the final polls had Bush ahead by just under two points (48.7% to 47.1%). As in 2000, the surveys overestimated Ralph Nader's tiny share of the vote. Bush gained more than Kerry compared to the final polls, the opposite of what the incumbent rule predicted.


Table corrected 11/3. Original had inadvertently used allocated results for TIPP

Results at the state level show a similar pattern, although the precise final margins are still subject to change in the few states that continue counting. The table that follows shows how the average of the final polls in each battleground state (as calculated by RealClearPolitics) compares to the current unofficial results. Polls did better in some states than others, but overall, it looks like the margins were close. On first glance, averaging across all battleground states, Bush and Kerry appear to have gained equally compared to their standing in the final state polls.


Updated 5:12 p.m. Nov. 3

Obviously, these topics deserve more careful consideration once we have final results, but it is obvious that (a) the incumbent rule did not apply and (b) that the consensus of the national polls was reasonably close to the final result. While the turnout was heavy, it did not conceal any hidden Kerry vote, as I speculated it might. If anything, the polls slightly underestimated Bush's national margin.

Why? One big clue should have been the failure of the incumbent rule in 2002, when a number of incumbents received more support on Election Day than on their final polls. In retrospect, I dismissed that contradictory evidence too quickly. John Kerry's lead pollster, Mark Mellman, was of clearer mind when he wrote this past Sunday:

We simply do not defeat an incumbent president in wartime. After wars surely, but never in their midst. Republicans have been spinning this fact for months, and they are correct.
A point well taken.

Related Entries - Incumbent Rule, The 2004 Race

Posted by Mark Blumenthal on November 3, 2004 at 09:43 AM in Incumbent Rule, The 2004 Race | Permalink


What went wrong with the exit polls? Sure, each state poll has a margin of error and the final results were mostly within that margin. But, wasn't the error almost always predicting Kerry to do better than he did? That doesn't sound like randomness. And if there was a systemic error, how can we be sure that the post-election analysis from the exit polls is accurate. Can the exit poll results be adjusted based on the actual vote?

Posted by: Josh | Nov 3, 2004 10:20:25 AM

I would like to suggest that the exit polls leaked to the Internet yesterday -- polls that were erroneously biased in Kerry's favor -- became an actor in the election. There is an interesting analogy to the 1992 UK general election, where last-minute exit polls that overestimated Labour support energized last-minute Conservative gotv efforts. See http://www.battlebunny.org/archives/2004/11/why_were_the_ex.php

Posted by: Pinky | Nov 3, 2004 10:26:12 AM

There was systematic error in the exit polls. As it was a collaboration, I doubt anyone's head will roll.

Unfortunately, it will feed into the belief that the MSM is biased against Bush.

Posted by: Eric | Nov 3, 2004 10:57:19 AM

The real reason that polls missed the victory by 2 percentage points is simple. One person out of every hundred said that they would vote for Kerry, but voted for Bush.


Because the MSM and the liberal left has been so nasty to conservatives, that they are "shamed" into lying.

I'll speak for myself. I live in Manhattan, and I have lied to virtually every single friend. I've actually had to fake my sadness today, when I'm happy inside.

I'm a quiet conservative, and there are many more out there.

Posted by: The real reason... | Nov 3, 2004 11:23:22 AM

So what was the final verdict on Zogby strictly weighting by party ID? Did the party ID in the exit polls match the 2000 election?

Posted by: damon | Nov 3, 2004 11:30:38 AM

I want to bring to your kind attention that your table on the final results is inconstistent because for some polls you choose to report allocated final projections and for some your are reporting results where "undecided" have not been allocated. For instance polls by Pew and TIPP have allocated the undecided. The results you quote for Gallup is unallocated. I think you should choose one method for fair comparison -- else it is misleading.

Our results BEFORE allocation of undecided is Bush -- 48.6%, Kerry -- 45.3%, Nader -- 0.9%, Other -- 0.8%, Not sure -- 4.4%.

Our results AFTER allocation of undecided is Bush -- 50.1%, Kerry -- 48.0%, Nader -- 1.1%, Other -- 0.8%.

Posted by: Raghavan Mayur | Nov 3, 2004 12:21:05 PM

I completely agree with the real reason. The constant chant of "The Worst President Ever" was very intimidating to moderate supporters of Bush. It may have been only 10% of Democrats who voted Bush, but I think this was the number whose signifigance was never discussed prior to the election. More Democrats voted Bush than Republicans voted Kerry, while turnout was dead even. That is how Bush won.

Posted by: Leard | Nov 3, 2004 12:24:06 PM

Is there any evidence that the exit polls were more accurate than the final vote tallies? In other words, that FL and OH exit polls (but only FL and OH) showed better Kerry numbers is indicative of vote-tampering in states with strongly partisan Secretaries of State. Check out the threads on dailykos.com for some eye-witness reports of ballot boxes being loaded into trucks, rather than being taken directly to the counting office.

Also, is there any correlation (in FL) of a corellation between "high" exit polls and e-voting w/o a paper trail? I think this would be worth looking into, if only to help restore some faith that our democracy isn't broken.

Posted by: Brian | Nov 3, 2004 12:25:28 PM

Regarding mean and variance estimation in the standard polls, to what extent are respondents weighted by demographic information?

I came across a report by the National Council on Public Polls (Feb. 5, 2004) indicating that 84% of 2002 U.S. Senate polls examined differed from the election outcome by less than the margin of error computed at 95% confidence.

Does this apparent underestimation of the margin of error have precedence in other election cycles?

Have there been any critiques of the variance estimation methods used by major polling agencies with regard to their accuracy, particularly in connection with weighting and its impact on margin of error computation?

Posted by: brian | Nov 3, 2004 12:42:57 PM

How about this for an explanation for the combined failure of the incumbent rule and the exit polls? As suggested above a certain % of the population was reluctant to admit their support for Bush. Pre-election this manifested into them saying they were undecided - hence the failure of the Incumbent Rule. Post election they no longer had the undecided option, so they said they voted Kerry - hence the failure of the exit polls. It would be interesting to see if there's any justification for this from the numbers.

Posted by: alex | Nov 3, 2004 12:48:02 PM

Internationally, aren't exit polls used to determine whether votes were honestly or fraudently or accurately COUNTED?

In Venezuela recently, the reported vote count matched the exit polls conducted by international observers. This closeness granted legitimacy to the reported vote count. If the exit polls did not match the reported results, the results would have been questioned not the exit polls. Are we to believe that the US cannot conduct trustworthy exit polls when the international community uses them to verify elections?

I know you probably have no way of investigating the possibility of systematic errors in vote counting, as opposed to apparent errors in exit polling such as gender imbalance, but please consider the international viewpoint in your analysis. An obvious argument is that a gender imbalance in voting was a reality for this US election. This seems testable with a follow up poll. A deeper argument is that millions of votes are spoiled or for some reason not counted, and the voter of the spoiled ballot has no way of knowing this occured. This was a finding in the Caltech-MIT voting blue ribbon report after the 2000 election. I believe the report concluded spoiled/uncounted balllots were found to affect poorer/minority precints disproportionally.

I eagerly await your statistical analysis comparing exit polls to reported votes. I hope you will consider the international use of exit polling to inform your analysis.

Posted by: Alex in Los Angeles | Nov 3, 2004 1:26:29 PM


Re the incumbent rule: I tried to tell you. :-)

Re your chart: thanks for compiling it; it is informative. However, you have the Wisconsin numbers incorrect.

Posted by: Gerry | Nov 3, 2004 1:50:27 PM

Also, can you please address the exit polls in FL and OH quickly.

Kausfiles concludes that since the early exit polls were wrong this was due to changing voting patterns. And he seems to discount exit polls altogether. But he does not recognize that the FINAL exit polls also did not match the reported totals, at least in FL and OH.

Weren't these the two most extensively exit polled states? Wouldn't these have the lowest margin of error? Why do these states' reported votes have the greatest discepancy with the exit polls? These are serious questions that maybe you can help bring to Kaus's attention, explain to us, and open a debate about.

Maybe you could put a serious together like you did for the likely voter models. Please.

Posted by: Alex in Los Angeles | Nov 3, 2004 1:50:30 PM

Some questions:

So, let me take the opposite tact of most and ask for reasons why the reported results don't live up to exit polls, assuming they are accurate.

***How many people voted in precints compared to how many votes for president were counted? The missing votes for president are the spoiled/overvotes/undervotes, right?

***Does this account for the discrepancy in how people say they voted and what gets counted?

***How many missing votes in OH and FL vs. the other states? Does this explain the greater reported vote/exit poll discrepancies in these states vs the other states?

***Is there a bias by precint in OH FL in #'s of missing votes?

Is any blog commenting on these questions? I feel, your experience expertise is greatly needed on these questions, exit polls, and reported vote accuracies in general.

Posted by: Alex in Los Angeles | Nov 3, 2004 2:55:17 PM

I wonder if the traditional likely voter models really preformed well, or if they just got lucky. In other words, in this particular election, the likely voter models may not have correctly predicted the actual voters, but the votes cast by the actual voters were not that much different than the votes cast by the projected likely voter, and therefore the prediction happened to correspond to the result. Note, I don't know if this is true or not--I have no way of validating the likely voter models against who actually voted. If it is true however, it might mean that while the likely voter models (in the aggregate) got lucky this election, the same thing might not happen in future elections.

Posted by: Alan | Nov 3, 2004 3:16:38 PM

to "the real reason..." How strong could your convictions possibly be, if you were unwilling to admit them to your friends? I have no compassion for Conservatives who lie.

Posted by: jay kay ess | Nov 3, 2004 3:56:38 PM

Exactly the right response to "the real reason." Notice that the only people seriously saying this are those who live in very liberal areas? Guess what? Liberals in conservative areas don't tend to flaunt their beliefs either. But if they go as far as you do I have the same recommendation: get new friends.

Also guess what? Conservatives control all three branches of government and most of the country. Stop whining with your "big evil liberal media makes me feel bad" routine. Conservatives are NOT a persecuted minority. You're going to have to live with being the persecuting majority for the forseeable future, so just get over yourselves.

I mean, for ---sake, how many times do you have to hear "neutral" pundits opining that your not "real Americans" if you don't live in the rural midwest or the deep south. How come it is fine to run a campaign mocking Mass. but you can't "mess with Texas." Sheesh.

Posted by: dittoman | Nov 3, 2004 4:03:04 PM

I don't know that the incumbent rule was wrong -- it seems to me it was the turnout rule that turned out to be wrong. As Noam Scheiber pointed out:


Self-described liberals voted Kerry 86-13, conservatives voted Bush 84-15, but moderates voted Kerry 55-45. The big difference? Conservatives made up 34 percent of the voters, up from 29 percent in 2000.

In other words, it all came down to turnout. Undecideds may well have broken for Kerry at the end, but conservatives were fired up and came out in much greater numbers than moderates and liberals.

Posted by: Mitsu | Nov 3, 2004 7:26:19 PM


* I now believe the exit polls were planted.
* There is strong empirical evidence that, when overall polls show a dead heat, erroneous exit polls significantly mobilize late-afternoon turnout (for the party that believes it might lose) and undermines late-afternoon turnout for the party that thinks it's winning
* The Republican Secretary of State in Ohio knew this, and this knowledge played a role, perhaps dispositive, in his decision to keep the polls open until 10 PM.

In short - we was had. Again.

For more information on this analysis, which has thus far escaped media attention (not to mention professional attention), see http://www.battlebunny.org/archives/2004/11/why_were_the_ex.php


Posted by: Pinky | Nov 3, 2004 7:27:52 PM

Good point.

Posted by: Mitsu | Nov 3, 2004 7:36:34 PM

The effects of the exit polls you describe sounds reasonable.

But doesn't your argument ignore the FINAL Exit polls? Those that are calibrated once the results are in. There is no reason for those to be different from the vote count. That discrepancy is more worthy of serious analysis, statistical and systematic.

We hear that the exit polls are wrong, planted, within the margin of error. But lets have the professionals analyze and report back. Without theories, scientifically. An average of exit polls, a statistical meta-analyses of exit polls. Let's have them examine the discrepancies, because internationally a discrepancy calls into question the reported results, not visa-versa.

Dick Morris, agrees that this is a serious question worthy of national attention:

Posted by: Alex in Los Angeles | Nov 3, 2004 7:57:38 PM

In my younger years, I studied Tae Kwon Do at Iowa State Univerity under Master Yong Chin Pak. Master Pak is Korean, typically the judges at tournements were Korean, and the judging was biased in favor of anyone of Korean decent. Master Pak recognized this and gave his students some very sound advice: if you don't like the judging, knock the other guy out. Then there can be no question who won. I think this same lesson applies to many things in life.

Posted by: Alan | Nov 3, 2004 10:54:24 PM

The paranoia seems extreme here. The exit polls were off early because more Democrats voted early and more Republicans voted late. The final exit polls in OH and FL don't slant toward Kerry at all. The traditional theory that Reps vote early and Dems vote late also made the interpretation more difficult. In the end, there was no radical bias toward Dems in the exit polls.

Posted by: BarryR | Nov 3, 2004 11:55:47 PM

So which pollsters were accurate on the state races this year? Its probably going to be tough to get an accurate measure because so many states fell how the CW said they would but by quick look zogby was just as bad as he was in 2002 and Mason Dixon did exceptionally well again.

Posted by: Damon | Nov 4, 2004 8:36:21 AM

I wrote in an earlier comment that I am a "quiet conservative, and there are many more out there."

Here's an interesting article on "shy conservatives"...


Posted by: The real reason... | Nov 4, 2004 10:48:10 AM

The comments to this entry are closed.