Yesterday’s "Real Time" column by the Wall Street Journal‘s Hanrahan and Fry made essentially the same point I did about the exit poll/blogger controversy.
My point, for those who may have missed it, was less about defending "the likes of Wonkette" (whatever that means) than acknowledging that the existance of bloggers makes the rapid proliferation of highly sought after information (or raw data or whatever you want to call it) inevitable and automatic. Here is how Hanrahan and Fry put it:
The reaction to this whole controversy is so 1997 — a predictable volley of off-target shots by chin-waggers who either haven’t figured out that the world’s changed or refuse to admit that it has…
What’s changed is that once you had to be a media or political person of a certain minimum altitude to be let in on the secret; now, all you need is a Net connection and rudimentary typing skills. Those who used to have the clubhouse all to themselves have found the door wide open to anyone and everyone. And some of them don’t like it.
Which, come to think of it, really is so 1997: Information formerly reserved for a cadre of insiders now spreads more quickly than those insiders can control it. In other words, the middlemen have been eliminated. (If you spent 100 large at business school, you call that "disintermediation.") Just took a bit longer to hit the Election Day carnival, that’s all.
Well said. I am told the Wall Street Journal Online is free to non-subscribers this week, so read it all.
P.S. Say what you will about Ana Marie Cox, in addition to the leaked exit polls, she also posted (at 5:40 p.m. Election Day) the following leaked disclaimer that the National Election Pool officials had shared with the network honchos: “problems with exits in the following states…could be tipping numbers toward kerry: MN, NH, VT, PA, VA, CT, DE.” Not that it mattered. This was the same disclaimer that the editors of the Post and the Times either did not hear or discounted like everyone else.
Mark, don’t you want to see the final round of exit data for yourself? You clearly have a great deal of respect for Mitofsky and I have no doubt it is well placed. But don’t you think that data should be released publicly? Do you think it ever will be?
“Trust, but verify.”
Ronald Wilson Reagan
Hi, me again. The more I think about it the more I can’t shake the idea that those exit polls are doing damage to this election. I don’t think perception that somthing smells funny is going to die very easily. Despite what you wrote in your blog entry, do think maybe the reason the data wasn’t released is that there was incompetence at Edison/Mitofsky? Or worse?
Isn’t the most likely reason for the exit poll problem an error in sample precinct selection? My guess would be that the precincts polled were chosen based upon 2000 turnout and were not representative of actual 2004 turnout.
Regarding values voters versus security voters–is the vote correlation by population density as strong or stronger than the “how often do you go to church on Sunday” correlation? The red/blue maps by county indicate that it might be.
How much are values a product of population density?
Here is how I have been trying to explain exit polls to friends – is this correct?
In normal polling, people try to find random people to ask, and they ask enough people to be reasonably confident that the sample reflects the overall population.
But in exit polling, the precincts are not selected randomly. So, it’s not as simple a matter as being able to assume that exit polling would reflect the voting population as a whole.
Exit polling is designed to get a sense of a subgroup of the voting population – but to then later be *weighted* (or rebalanced, or renormalized) based on the later vote results. So you’d actually expect the preliminary exit poll data to be wrong – exit poll data doesn’t make much sense by itself, until actual turnout data starts getting applied to it. The whole point is for it to be corrected by the full voter data, so we can have a better sense of the overall makeup of the electorate.
In other words, it’s more designed to understand the demographics of the electorate, rather than to predict the results of the election.
Is that close to correct?
Tune, that’s my theory. And that’s all it is really, a theory. Yes, early polling data has to be wrong because voting is a 12-hour long activity and you need to get a reprentative sample within all 12 hours, not just one hour. Because the one hour sample could easily be biased.
But what if there is another sampling problem: like researchers don’t get a random sample? Maybe some groups won’t talk to researchers. Maybe some groups want too much to talk to researchers. Whatever the case, this could have come up.
But it’s just a theory. Hard to prove.
I would appreciate your comments on the article published today, “47 State Exit Poll Analysis Confirms Swing Anomaly”, By Jonathan Simon, Introduction by Scoop Co-Editor Alastair Thompson
http://www.scoop.co.nz/mason/stories/HL0411/S00142.htm
Thanks.
Well, I’ve seen in multiple places that the precincts aren’t randomly selected. If that’s true, then it means by definition that exit polls aren’t based off of random samples.
And, I’ve seen in one other source – an article on techcentralstation.com – that confirms that as well – that exit polling uses stratified samples, not random samples. Stratified means badly weighted polls, in order to get more information (in this case, demographic information) from smaller groups that would not make up a large part of random samples.
And if you think about it, it makes sense. Of course exit pollsters would lean towards the more diverse precincts for richer data sets. I have also read in other sources that exit pollsters lean towards “bellwhether” precincts, small locations which tend to predict the results for the entire state. In swing states, this would of course be swing districts.
And, diverse districts would lean towards the diversity party; Democrats. And, swing districts would mean less emphasis on the deep-red districts.
And since we know Bush’s success was in turning out deep-red people… honestly, it makes sense that most of the exit polling would lean more towards Kerry than the vote total would.
To expand on the previous comment, it also explains why the gap would be BIGGER in the CLOSER states. That graph that is floating around compares paper ballot states to non paper ballot states, but the same graph also shows that the gaps are big for close states, and small for non-close states.
Tunesmith, Dr. Cookie,
That is excellent. The randomness of precinct selection is fundamental. I have been assuming precincts are selected randomly, but if not then I lose much of my confidence in final exit poll’s ability to predict elections. Could you please post your sources on precinct selection for this year’s exit poll? Please note Mystery Pollster’s explanation:
http://www.mysterypollster.com/main/2004/11/exit_polls_what.html
For me, this would nearly settle the question, because internationally random precinct selection weighted by expected turnout is a prerequisite in exit polling for audit purposes.
Maybe, Mystery Pollster did not adequately address precinct selection. Many of us were left with the impression from his initial explanation that precincts are selected randomly. Probability of selection weighted by prior/expected turnout, but randomly selected.
“But in exit polling, the precincts are not selected randomly. So, it’s not as simple a matter as being able to assume that exit polling would reflect the voting population as a whole.”
If your theory is correct, i.e. certain precints are manually selected for whatever reason or chance of selection is weighed by factors other than turnout, then the raw exit poll data will not be a random sample. This is a fundamental question. Thank you for reopening the debate on the validity of exit polls for audit purposes.
Here are the list of exit poll issues I think are still open:
1. Precinct selection bias
2. Missed GOP turnout (see 1)
3. Missed Latino GOP turnout (see 1)
4. Gender bias in Final exit polls (all I’ve seen is gender bias in afternoon exit polls, which is irrelevant)
5. Non-respondant bias (Is the response rate really in the 50% range?)
Would you add or delete?
I forgot:
6. Accounting for Spoiled Ballots
7. Accounting for Provisional Ballots
These #6 and #7 I think are the real culprits along with possibly #1 and #5.
Tunesmith:
I think you are wrong. Precincts are selected randomly with a weighting for precinct turnout. At least according to what I’ve read on this blog to date.
Will you be able to post a source stating that precincts are not selected randomly?
Thank you,
Alex