May 02, 2006
MP received email from several readers over the last few days asking about a rumor that the television networks had decided to abandon exit polls for 2006 elections. The blogosphere rumor apparently originates with blogger Mark Crispin Miller (author of the book Fooled Again, How the Right Stole the 2004 Election & Why They'll Steal it Again) who provided the following seemingly authoritative report last week:
This year there will be no official exit polls conducted. The media consortium that paid for them in previous elections isn't going to do it any more, ostensibly because such polls have been "exposed" as unreliable; and so we will have no way to determine whether the official vote-counts can be trusted.
He seemed pretty sure, but it was news to me. So I went to the source and sent an email to the official spokesperson for the National Election Pool (NEP), that news consortion that conducts the exit polls. I received the following official, on-the record response:
The rumors are false. NEP will be conducting exit polls during the off year election.
There you have it. I've also heard unofficially that the exit pollsters are hard at work planning for 2006. So it looks like we'll have network exit polls to kick around again in November 2006 (although they promise to hold back the results, even from reporters, until 5:00 p.m. -- so maybe no mid-day leaks this time).
This rumor episode tempts me to remind readers about placing too much faith in similarly authoritative theories about how the 2004 exit polls are evidence of a stolen election. But that would be awfully snarky of me, wouldn't it?
PS: While the networks will conduct exit polls in November, as far as I know they will not conduct exit polls in connection with any of the primary elections, including tonight's Ohio primary.
However, those concerned -- with good cause -- about the problems with electronic voting and the integrity of our voting system might want to check out this Cleveland Plain Dealer story that mentions what sounds like an innovative exit poll being conducted today in Ohio's Cuyahoga County by the Election Science Institute (ESI). This sort of work has far more potential to help us monitor and improve the conduct of our elections than pouring over the remnants of the 2004 exit polls, which were simply not designed to check for fraud. Hopefully, we will hear more about the ESI project soon.
Related Entries - Exit Polls
I don't understand this obsession some people have with exit polls as a way of confirming or disproving the validity of the official results. Such a position fails to recognize that, in highly polarized environments or when a party/candidate evokes strong passionate negative responses from a large fraction of the electorate, people do have an incentive to hide their vote from exit pollsters.
This is well known elsewhere in the world. In several European countries, exit poll results are always taken with skepticism and more often than not are wrong, and nobody thinks there's anything fishy about the actual results when they differ from the exit polls. The reason is simple: some people are reluctant (read: ashamed) to tell a pollster that they just voted for the incumbent party or candidate that has been taking most of the heat and public criticism for a while; this inevitably results in the exit polls being biased against the incumbent (and, in the case of multi-party systems, often against any extremist or xenophobic parties too). Anybody who regularly follows elections in foreign countries knows that this is exactly what happened in last fall's German election (Schroeder's SPD government was very unpopular, and trailed the opposition CDU/CSU by several points in the exit polls, only to lose narrowly by about one point when all the votes were tallied), the infamous first round in the French presidential election of 2002 (when the polls showed that prime minister Lionel Jospin would face off president Jacques Chirac, only to lose second place to extreme right-wing candidate Jean-Marie Le Pen), every Spanish election between 1993 and 2000, and most recently, the parliamentary election in Italy last month (where outgoing -and unpopular- prime minister Silvio Berlusconi was predicted to lose by four to five points but ended up losing the lower chamber by only a tenth of a percentage point). Applying the same patterns here, it is then easy to see how one can explain the discrepancy between the exit poll results and the final outcome of the 2004 US presidential election: voters approached by pollsters were more likely to hide the fact that they had just voted to re-elect such a polarizing and highly disliked figure as Bush, than they were to hide having voted for the challenger.
By the same token, I predict that the same thing will happen this November: because the current climate clearly favors Democrats, any differences between the outcome and the exit poll results will be in the form of Republicans coming significantly closer to Democrats than people looking at the exit polls would expect. (Mark my words!)
Posted by: Alexis | May 3, 2006 12:43:19 AM
Assuming this is true, how can we trust the media consortium? In 2004, they refused to release the raw exit polling data.
It's pretty simple. The government (those who make the rules for elections, and stand to benefit from those rules) have a conflict of interest. Similarly, the corporate media (who have benefited from deregulation) have a conflict of interest.
While you may not agree with the conclusion that the 2004 elections were fraudulent, the truth is that election officials have never offered any evidence that they were legitimate. All they offer is a number. Here's a number: 9,504,807. What do you make of it? If I said that's the number of voters whose votes were stolen in 2004, could you prove it's not so? (Disregarding the fact that I am admitting that number has no basis in fact.)
When we are talking about the most fundamental right in a democracy, the burden of proof is on the government. Machines can spit out any number they want. People in back rooms can decide vote tallies based on how many votes their party needs to win. We, the people, have a right to be assured that our elections are free and fair. At the present time, we have no such assurances.
If exit polling was used to conclude that there was fraud in the Ukrainian elections, why can't they be used for the same purposes here? Does anyone really believe that the people in office right now wouldn't cheat to gain & maintain power? It's not partisan -- it's just common-sense.
If the media consortium wants to do exit polling, fine. They should fund exit polling by an academic institution to remove the conflict of interest that was so clearly demonstrated in 2004, when the consortium went along with claims that their polling was not accurate (which, on the surface & assuming no conflicts, would be an extremely poor business decision).
Posted by: Lewis Miller | May 3, 2006 12:51:16 AM
Regarding the claim by Alexis that "it is then easy to see how one can explain the discrepancy"...
That is at best a theory that has not one shred of evidence. It is a supposition. Say your hypothesis is "The election was not stolen." The scientific method does NOT tell you to then go looking for any possible explanation to support your hypothesis. You actually have to PROVE something.
On the one hand, you have the election results. On the other, you have exit polls.
The exit polls have quantifiable margins of error, and the methods are known. There are variables that pollsters use to adjust for clustering effects. They've gotten quite good at it over the years.
The elections, on the other hand, use different methods depending on where the voting is taking place. Some places they use optiscans, some places they use electronic machines, some places computerized. Tabulating the votes is often done away from the public eye, whether it is on the computerized machines or in a back room where the media has been expelled due to some vague "security threat."
The point is not which is more accurate or whether there is fraud. The point IS to have a check on the legitimacy of the elections. If you just have the election & no exit polling, exactly how do you figure that no tampering is going on, or that the tampering that is going on doesn't rise to the level where it might affect who is declared the winner? (By the way, nobody tampers if they don't think it'll affect the results.)
Sure, it's possible to legislate things like a verified paper trail... In case you haven't noticed, that hasn't happened yet, and HAVA has actually increased the potential for cheating. However, no legislation is needed for exit polls. Maybe that's why people have this "obsession". Because they want to make sure that the damn elections aren't fixed!
Posted by: Lewis Miller | May 3, 2006 1:07:21 AM
Regarding Lewis' response to my comment above:
You claim that my point that exit polls are typically biased against unpopular incumbents or extremists is "at best a theory that has not one shred of evidence," yet I have given you evidence from recent elections in France, Italy and Germany. Are you suggesting that in those four countries the elections are fixed? Those are developed, G-8 member nations, not third-world banana republics. Moreover, nobody in any of those countries (neither politicians, nor analysts, nor citizens' rights groups, nor the media) believes their elections have ever been rigged.
I think the problem is that Americans hardly ever look at foreign experience for answers to their questions or ideas to solve their problems. One of the reasons why Germans or Italians trust their electoral system is because it is a centralized one, whereby all polling stations use the same reliable, transparent voting method which makes recounts easy and for that very reason unnecessary. Voters in most European countries pick a piece of paper with the name of the party/candidate already written on it (or simply check the box next to the party of their choice), and
they put that piece of paper in an envelope which is then deposited in an urn. They don't have to worry about there not being a paper trail, or about getting used to a new system when they move to a different county. They often don't have to worry about registering to vote either, because the Census, updated annually, usually serves as the default voting roll, and individuals only have to check that they have included and that their information is correct. And you know what? Nobody in any EU country is worried about whether their vote will get counted come election time. On election night, the votes are tallied in a timely and reliable fashion (usually the final number of votes for each candidate is known by midnight of the election night), and challenges (which are very infrequent) are resolved quickly too.
So here's my suggestion: let's adopt the voting system of any one of those other developed countries that have been running clean elections for years (even close ones) with absolutely no problems. Then we'll be able to take exit polls for what they are: an interesting but not necessarily very informative snapshot of the preferences revealed by a subset of willing, unashamed voters in a subset of polling stations, which should serve during election night as a snack for people hungry for the actual results.
Posted by: Alexis | May 3, 2006 1:48:44 AM
Alexis, I agree that we need uniform election standards across the entire country, they should be auditable, and certain conditions should trigger audits automatically (going back to paper records for the vote count).
My claims about your theory apply to the American elections only. That is, there was not any evidence produced to support the claim that the 2004 exit polls were skewed because of reluctant responders. All that was ever offered is a "maybe".
In those other developed countries, it's likely that the voting system is more transparent. The system in the U.S. is a mess, and some fraud has been going on for quite a while. Unfortunately, since 2000 the potential for large scale fraud has gone way up, and the government has done nothing substantial to reassure those with concerns. In fact, things have actually gotten worse. So it's either (a) incompetence OR (b) willful manipulation of the system for political gain. Take your pick, it doesn't really matter to me which it is, because as long as there are huge gaping holes that allow fraud, it's incredibly naive to believe that nobody would possibly take advantage of it.
Posted by: Lewis Miller | May 3, 2006 2:01:37 AM
"there was not any evidence produced to support the claim that the 2004 exit polls were skewed because of reluctant responders."
Well, the evidence is indirect, but many of the precinct and interviewer 'effects' documented in the Edison/Mitofsky report make more sense as correlates of non-response bias in the poll than of fraud.
Also, there are lots of missing fraud correlates. For instance, I don't think any fraud theorist ever predicted that the largest exit poll discrepancies would occur in lever-machine precincts; or that precinct-level change in vote share from 2000 to 2004 would be uncorrelated with exit poll discrepancy; or that the states where Bush did better than expected in pre-election polls would tend to have _smaller_ exit poll red shifts, or even exit poll overstatements of Bush's vote share.
I'm a bit tired of being told what the exit polls do and don't provide shreds of evidence for. Folks who want exit polls to serve as a check on the legitimacy of the elections have to face up to the problems with exit polls -- not just rehash talking points from late 2004.
Posted by: Mark Lindeman | May 3, 2006 7:37:06 AM
I actually think exit polls are unreliable and are just conducted under the influence of politicians to encourage public opinion in their favour.
For outsourcing voice, chat, email or back-office support, visit the website of Aumenta Call Center, India.
Posted by: Call Center Outsourcing | May 3, 2006 8:37:53 AM
Mark Lindeman, if there is no way to verify the "official" vote tallies, then we have to ask why not?
If exit polls are insufficient, then perhaps we need better exit polls, or some other method. The premise of a democracy is that the power to govern is derived from the consent of the governed. That means that the governed have a right to be assured beyond a reasonable doubt that officials are being elected in a legitimate manner.
Just explain to me why anyone would have faith in the accuracy of "reported" results, beyond "because we say so". Maybe Americans trust their government, but why not have some actual basis for that trust? Banks trust their tellers (who wouldn't be hired otherwise), but they still make sure that the money is counted by someone else at the end of the day.
Posted by: Lewis Miller | May 3, 2006 12:14:59 PM
Lewis, mark me down alongside Alexis in favor of verifiable elections. I don't know whether we can simply adopt some other country's system, because we vote for many more offices and issues than in the countries mentioned. But certainly nothing in my post conveyed a call for faith in the accuracy of official results.
I'm not even opposed to using exit polls as one source of information about the accuracy of the count (in fact, I do). I like more information, not less. I just don't see how we will ever be able to _rely_ on them for verification.
The Ukraine case rather makes the point: there seems to have been more difference between the two sets of Ukraine exit polls in the first 2004 run-off (8 points on the margin) than there was between the U.S. exit polls and the official returns. (Granted, one set of Ukraine exit polls used face-to-face interviews, so maybe we could agree that that approach was Just A Mistake. The other set differed from the official returns by 12 points on the margin, so the discrepancy was much larger than in the U.S.) Many observers "conclude(d) that there was fraud in the Ukrainian elections" because of extensive direct evidence; at most, the exit polls offered additional support.
As for data release, you can download raw data for some 70,000 interviews that were tabulated individually on election day; these seem consistent with the 110K+ interviews that were incorporated in the election day projections. I should also note that many, many survey analysts actually believe that the surveys manifested non-response bias, and so we think it was both honest and pragmatic for the exit pollsters to say so. I could gripe about Edison/Mitofsky and/or corporate media, but these are not the points I would gripe about.
I'm just saying: if we are going to talk about exit polls, we should try to get that part right.
Oh, just to be evenhanded: Alexis, since the U.S. exit polls seem to have skewed Democratic in at least five consecutive presidential elections, most particularly in 1992, I'm not convinced that the 'bashful Bushie' hypothesis holds. Your prediction may be right for the wrong reasons.
Posted by: Mark Lindeman | May 3, 2006 4:08:52 PM
Electronic voting would be fine if they provided a paper trail for it. The idiots who designed the machines didn't design them to print out a vote. Brilliant!
Posted by: PoliticalCritic | May 4, 2006 2:20:33 PM
1.) Mechanical lever machines, which have been used throughout the northeast since the 1950's, and which can be pre-set with vote totals, have never had a paper trail. Why does no one ever demand a paper trail for those machines? It seems a little hypocritical for people to raise a hue and cry about the electronic machines, but say nothing about the lever-style machines, which can be manipulated just as easily.
2.) The fact that exit polls have skewed Democratic for the last 5 elections just confirms that the Republicans have been involved in stealing elections for well over a decade. Based on my research on the subject, I am of the opinion that the Republicans haven't legitimately won a Presidential election since Eisenhower in 1956. It's truly sad that we live in one of the most corrupt "democracies" in the world.
Posted by: Aaron | May 4, 2006 4:03:04 PM
1) It's at least open to debate whether lever-style machines can be manipulated as easily as DREs, but that's probably off topic here.
2) Umm, whatever. If you think that a 2-point exit poll discrepancy in 1996 helps to buttress the claim that Reagan stole the 1984 election, who am I to dispute it? (Or is that not what you meant by "legitimately won"?)
Posted by: Mark Lindeman | May 4, 2006 6:32:37 PM
Sorry I'm late to the party, but I have to bring up New Hamsphire again as I have in every Exit Poll thread here.
New Hampshire had the second or third largest exit poll discrepancy of any state. New Hampshire had paper ballots. Ralph Nader paid for a hand recount of something like 8% of the state and found essentially no difference between the official tally and the recount. At that point, everyone agreed that the official tally was correct, and went home.
This proves conclusively that the exit poll in New Hampshire-again, the 2nd or 3rd biggest discrepancy of any state-was wrong. There is no reason to assume that the exit polls were any better in the rest of the country. Claiming exit poll results show fraud in the 2004 election is not supportable.
Posted by: Marty H | May 4, 2006 8:00:20 PM
Mark: Reagan's 1984 victory was illegitimate not because of cheating at the polls, but because he never should have been in office in the first place to run for re-election. It's been well-documented that in 1980, Republican nominee Ronald Reagan was working with the Iranians to impede President Carter's progress in freeing the hostages. Reagan actually set up the deal to release them the day he was inaugurated. Absent Reagan's illegal interference, Carter wins the 1980 election. Reagan was a traitor for subverting a sitting US President.
Marty H: You must be joking to suggest that counting 8% of the ballots in NH is enough to conclusively prove anything. How do you know they counted votes in the precincts where the cheating occurred? You don't know that. Obviously, the GOP doesn't cheat in every single precinct. They handpick certain ones. Sorry, but 8% is nowhere near a large enough sample size to be sure you hit the select precincts where fraud is being orchestrated.
Posted by: Aaron | May 4, 2006 10:53:42 PM
Aaron, I won't wrangle on the 1980 election -- off topic again. As for whether the small national exit poll discrepancies in 1988, 1996 and 2000 indicate some sort of fraud, well, they might, but it isn't obvious.
On the New Hampshire front, first to correct Marty: AFAICT the recount was of 11 precincts, chosen because Bush seemed to have done surprisingly well. That said, I don't think it is all that easy to steal a double-digit vote share while eluding a _targeted_ recount of even 11 precincts. Heck, to elude an _untargeted_ recount of that size, you might concentrate fraud in about 5% of precincts and cross your fingers -- but then you have to alter the margins by something like 70% in those precincts to get the double-digit swing. Anyone here think that actually happened?
Posted by: Mark Lindeman | May 5, 2006 1:27:36 AM
Looked up the numbers on New Hampshire.
49,871 ballots recounted by hand. (From a Dartmouth paper by Herron and Wand)
677,738 ballots cast. (From Wikipedia)
Exit polls: 1,833 respondents for NH (From CNN)
Aaron-Nader selected the wards to recount because the Bush votes were higher there than expected. In other words, they ran a statistical analysis, and said "If there's fraud, it's here" and then they found no signs of fraud.
Contrary to "conclusively proving nothing," the recount proves that those "suspect" precincts were tallied correctly. Absent evidence of fraud (which is totally lacking) one must assume that the rest of the state is also tallied correctly. Certainly Ida Briggs, the instigator of the process, and Ralph Nader believe that the state was tallied correctly.
And Aaron, if counting 7% of the ballots isn't enough to conclusively prove anything, then how in the world can interviewing 0.3% of the voters mean anything? A larger sample size trumps a smaller one; a count trumps a sample. You're arguing that a seven high hand beats a flush. It doesn't.
Posted by: Marty H | May 5, 2006 3:32:27 AM
Marty: Even if I stipulate everything you've said as being correct, it still doesn't prove anything. Just because a recount matches the original vote doesn't mean fraud didn't occur. People could have voted multiple times under diffrent ID's, or ballots could have been manufactured out of whole cloth. That's another major reason why having a "paper trail" doesn't solve anything. If someone is smart enough to program electronic voting machines to tally fraudulent votes, then they are also smart enough to program it to print out a fraudulent paper record of said fraudulent votes.
Posted by: Aaron | May 5, 2006 8:16:44 AM
If I'm ever a defendant in a case, I want you on the jury.
Do you have any evidence that any of the causes of fraud you recite occurred in New Hampshire? "It could have" isn't a very convincing case.
Posted by: Marty H | May 5, 2006 12:53:59 PM
Sorry, Marty, I forgot to run it both ways: about 4% of New Hampshire precincts (I think -- there seem to be 300 or so), but indeed 7% to 8% of ballots, which is probably more salient in this context. I just misinterpreted your phrase "of the state."
Aaron, I can't tell whether you actually think there was double-digit fraud in New Hampshire, or you are making an abstract point about the possibility of fraud, or you can't decide yourself.
Posted by: Mark Lindeman | May 5, 2006 8:14:51 PM
Both, Mark. I do believe widespread fraud occurred during the 2004 election. And I am also making the abstract point that just because a recount matches the original vote count, it does not conclusively follow that fraud did not occur.
Posted by: Aaron | May 6, 2006 11:28:19 AM
OK, Aaron, I agree with the abstract (or general) point, although I don't think it has much force in the New Hampshire context. Someone could've fiddled or stuffed all the op-scan ballot boxes (I think all the precincts recounted used op-scan), but I have no particular reason to believe so. I have a bunch of reasons to think the exit poll there was wrong, starting with the fact that it put Kerry up +15.0, whereas the average of the last six state polls (using the electoralvote.com database) was Kerry +2.5.
There are many things that I have no conclusive proof didn't happen, but I worry a lot more about some than others. You can worry about New Hampshire 2004 if you like, but I don't see how it is going to build a groundswell of support for election transparency.
Posted by: Mark Lindeman | May 6, 2006 4:58:51 PM
I might as well correct the URL: that's electoral-vote with a hyphen. There is a handy little spreadsheet-compatible file at http://www.electoral-vote.com/2004/info/allpolls.csv
Posted by: Mark Lindeman | May 11, 2006 8:50:41 PM
Even later to the party, sorry.
Lewis Miller wrote:
'That is at best a theory that has not one shred of evidence. It is a supposition. Say your hypothesis is "The election was not stolen." The scientific method does NOT tell you to then go looking for any possible explanation to support your hypothesis. You actually have to PROVE something.'
This kind of statement seems to me to be acquiring the characteristics of an urban myth.
First of all, there is substantial evidence (way more than "one shred") that factors relating to polling methodology were associated with apparenty Kerry-wise error in the poll. This evidence is documented in the Edison-Mitofsky January report.
What seems to be at the root of the myth that there is "not one shred of evidence" that Bush voters had a lower participation rate than Kerry voters is the fact that there is no DIRECT evidence. By definition, the votes of those who did not participate in the poll CANNOT BE not known.
So to test the hypothesis, you cannot simply start with the hypothesis "the election was not stolen" which has no operational value, and in case is a theory, rather than an operational hypthesis. What you CAN test are predictions arising from theory.
Say we take two contrasting theories to account for the exit poll discrepancy:
1. Kerry Votes were transferred to Bush
2. Kerry voters were more willing than Bush voters to participate in the poll.
We can then derive testable operational hypotheses from each.
For example, while we cannot test the fraud hypothesis directly, we can make testable predictions that arise from it. For example, if shifting Kerry votes to Bush were responsible for the exit poll discrepancies at precinct level, we are likely to see greater apparent bias in favour of Kerry in precincts where Bush did rather better than, say, his 2000 performance, and less bias in favor of Kerry in precincts where Bush did less well. In other words, it predicts a correlation between the degree "swing" to Bush since 2000 and the degree of "redshift" (votecount "redder" than poll). This hypothesis was tested by ESI for Ohio, and by Mitofsky for the 1250 precincts analysed in E-M report, and in both cases no correlation was found. Given the large statistical power of the second analysis, it is hard to draw anything other than the conclusion that if fraud were responsible for a even a proportion of the total "redshift" in the poll, it was for a proportion too small to be statistically detectable in virtually the entire sample of precincts. Other, similar predictions can be made, for example, whether the state-level exit poll error was correlated with the degree to which the state results diverged from state pre-election polls (no). There are more, but to date I have not found a single one that supports the vote-shift hypothesis, and it is not for want of trying.
Turning to the second hypothesis, if Kerry voters were more willing to participate in the poll, then we would expect to see greater bias in Kerry's favour in precincts where factors likely to compromise random selection protocol were present (i.e. where such factors were likely to make any underlying differential willingness manifest). And indeed,this is exactly what the E-M January evaluation reports.
As Lewis Miller will be aware, you cannot prove causality with this kind of analysis, nor can you even prove an association - it is why we deal in p values. But we can apply the scientific method, i.e. use a theory to make a testable prediction - and test it. And while there is considerably more than a "shred" of evidence to support theory 2, there is, to my knowledge, barely a "shred" of evidence to support theory 1.
And another urban myth:
'The exit polls have quantifiable margins of error, and the methods are known. There are variables that pollsters use to adjust for clustering effects. They've gotten quite good at it over the years.'
Yes, confidence intervals can be quantified fairly readily, even when clustering is involved. However, you must know that these confidence intervals simply tell you how confident you can be that your sample is representative of the population assuming a random sample. In other words it is a measure of sampling error. It tells you absolutely nothing about non-sampling error, as regular readers of MP must surely know. It is based on the assumption that the sample was random. It is precisely this assumption that is called into question by the discrepancy that occurred in 2004. The evidence suggests that the sample was not random, and that the more difficulty there was likely to be in getting a random sample, the greater was the bias in Kerry's favor.
Posted by: Febble | May 13, 2006 5:17:45 AM
Sorry, did not mean to make an anonymous post. My name is Elizabeth Liddle.
Posted by: Febble | May 13, 2006 7:04:50 AM
"For example, if shifting Kerry votes to Bush were responsible for the exit poll discrepancies at precinct level, we are likely to see greater apparent bias in favour of Kerry in precincts where Bush did rather better than, say, his 2000 performance"
Hypothetical situation: Precinct Swingy in Middle County of Arizona has a vote percentage of 47% Gore, 53% Bush in the 2000 election. 4 years later, due to different campaign issues and massive get out the vote campaigns, a lot of Democrats in Precinct Swingy decide to vote, and some independents decide to go for Kerry. As a result, the vote totals for Precinct Swingy in the 2004 election are: 55% Kerry, 45% Bush. This precinct happens to be sampled by the exit polls. Meanwhile, a hacker hacks into the Middle County central tabulating server, and implants a virus that shifts many precincts towards Bush, shifting Precinct Swingy 6%. Thus the official vote totals for Precinct Swingy are: 49% Kerry, 51% Bush.
Thus, while Precinct Swingy has a large "red shift," Bush actually did worse in it than it 2004. Your hypothesis would only be valid if the 2004 election results were exactly like those of the 2000 results, and fraud was the only independent variable.
Posted by: Timothy Charters | May 13, 2006 11:59:37 AM
The comments to this entry are closed.