Last week, Gallup posted an analysis of the long term trend in party identification. Unfortunately, by the time I got a chance to read Jeff Jones article, it had already gone behind Gallup’s subscription-only wall. However, today’s Frank Newport video briefing (which is always free to non-subscribers) includes a brief summary of that analysis, so I want to pass along the link.
The Gallup analysis looked at roughly 8,000 interviews conducted per quarter since 2004. The bottom line (from the original Jones analysis):
Democratic identification has been remarkably consistent at roughly 33% since the beginning of last year. Republican identification fell from 35% in the first quarter of 2005 to roughly 32% since that time. Independent identification has increased from 31% to 34% during the same period.
A few weeks ago, our friend Professor Charles “Political Arithmetik” Franklin performed a similar analysis. He also looked at trends in party identification, but he went further, applying his usual graphically intense approach to the national party identification data released by eleven different public pollsters since January 2005 (and none of these pollsters weight or adjust their results by party ID). Franklin found essentially the same trend – a decrease in Republican identification offset by a rise in independents.
In today’s video briefing, Frank Newport mentions some historical context also included in the Jeff Jones analysis last week:
Past Gallup polling shows that the percentage of independents typically declines in a presidential election year — when partisan politics is at its most intense — and then usually increases in the year following the election.
What was different this time is that the increase in the independent percentage came mostly from Republicans. Usually, in the Gallup data, both parties lose identifiers in the year following a presidential election.
Update (4/5) – promoted from the comments section: Joran is absolutely right to recommend this “fantastic” post today by Charles Franklin on the weak relationship between party ID and presidential job approval.
Also, be careful not to confuse party identification with vote preference. While party ID has always been a very strong predictor of the vote, it is far from a perfect predictor. Historically, Democratic presidential candidates have typically received a smaller percentage of Democratic identifiers than their Republican opponents. Check the National Election Studies data on party. The Democratic party ID advantage was much larger than it is today even in years when Republicans won lanslides — 1952 (+23), 1956 (+13), 1972 (+18), 1984 (+9).
I’m not sure what the details are, but what this shows is that the partys are about even. But the polls that don’t adjust for party ID seem to always quite a few more democrats than republicans. If you compare the presidenital popular vote to the ANES party ID, you find the vote percentages have the GOP consistantly pulling a higher percentage of voters than the republicans and independants leaning republicans. Often they republicans are even pulling into the democratic percentages. But the democrats never pull from the republican percentages.
All this raises the question to me of what are these polls for? If we know that there is a sharp split based on party ID (80+% for for one party and 80+% against for the other party) and then get party ID percentages that are no where close to equal, then is the poll really telling us something or is it just cocooning?
I just did a quick look over at real clear politics.
The Time poll had 28% republican 32% democrat, 25% independant, 12% other and 3% don’t know/no answer.
The democracy corps had 37% dems to 33% republicans which goes to 50% dems to 41% republicans when you include leaners. But the dems have only once in the last 40 years been able to put together a majority in the presidential popular vote (Carter 1976 50.08%).
The other two didn’t give party ID information.
There seems to be a standard and consistant tilt in the polls towards having significantly more democrats than republicans in the sample.
The CBS poll a month ag that generated so many headlines had a raw split of 31.5% republicans, 32.2# dems and 36.3% independants, which they then weighted to 28.9% republicans, 34.2% dems and 37% independants. Surprise surprise, that showed “Bush’s lowest point ever”. Of course that can’t have anything to do with the reweighting to take what was essentially equal party ID and move it to a 5% lower party ID.
Sorry but I’m forseeing a huge pollster credibility gap coming, just like the MSM credibility gap, if they can’t find a way to address the issue.
“The CBS poll a month ag that generated so many headlines had a raw split of 31.5% republicans, 32.2# dems and 36.3% independants, which they then weighted to 28.9% republicans, 34.2% dems and 37% independants.”
Sigh.
“Despite what you may have read elsewhere, CBS News does not weight its polls by party identification. Neither do Gallup, the Pew Research Center, Harris Interactive, Time, Newsweek, AP-IPSOS, Fox News, LA Times/Bloomberg or ABC News/Washington Post among others.** These organizations do typically weight or statistically adjust their samples of adults so to match US census estimates of demographic characteristics like gender, age and race. The procedure eliminates minor errors in the demographic representation of these samples due to either random sampling variation or response bias.” http://www.mysterypollster.com/main/2006/week11/index.html
Now it may be true that the process of reweighting for *other* demographic factors has the *incidental result* of reweighting party ID. But so what? If a poll has underweighted, say, blacks, should it refuse to adjust simply because doing so has the effect of increasing the Democratic percentage? And incidentally, how do you know that Gallup’s party ID percentages (which tend to be more favorable for Republicans than those of other polls) are right and that of other polls are wrong? Virtually *all* polls including Gallup’s show Bush’s popularity down very considerably from 2004 and it would seem more counterintuitive to say that party ID has remained about the same since then than it would be to say that the GOP has taken *some* substantiall toll in party ID (if not losses to Democrats, then at least to Independents.)
(Moreover CBS’s adjustment would not affect the “new Bush low” status unless they had *not* adjusted in earlier polls…)
“Sorry but I’m forseeing a huge pollster credibility gap coming, just like the MSM credibility gap, if they can’t find a way to address the issue.”
If by “issue” you mean that there is a growing population of people who follow politics _intensely_, are generally pretty bright but have comparatively little understanding of the meaning/context of sample/survey statistics and hence _obsess_ over things they “see” in polls that aren’t really there, than yes, I agree there will soon be a credibility gap among pollsters.
But the “issue” resides in the audience, not the pollsters.
And perhaps our society’s ability to provide stats in K-12 education (of which there is currently effectively none).
As has been discussed on here ad nauseum, the weighting by party “problems” in polls are usually not really there at all.
Republicans Losing Numbers
New Gallup poll: 33% of Americans currently identify themselves as Democrats, 34% as Independents, and 32% as Republicans. These numbers represent a 4-point downswing for Republicans since 2000, with the percentage of Independents increasing by the sam…
Assuming the polls are correct, the GOP has lost some folks, while the Dem’s have stayed even. Why is that? My theory is that the Dems have been hyper-partisan for 6 years, thus keeping their self-identifiers, but not reaching out to invite new folks. The GOP has been wimping out since the Nov ’04 election. On top of wimping out, they’ve been dissing their base (Ports deal, immigration), been incompetent and haven’t been getting much accomplished (tax cuts, Bueller). Base pubbies are disgusted with the McCains, Hagels, Grahams, Chafees, Snowes and all the other media pandering lib GOP Senators.
David T,
Yes, the folks at CBS give the same defense of their polling as they did the TANG memos “trust us”. But lets look at it another way. Lets assume that they weighted for party ID, but not age. If they then consistantly had polls that were non-representative of the population with respect to age (and told us that they weren’t going to weight by age) would you be okay with that being a “representative sample”. The point I am trying to make is when the issues have a very partisan divide based on a chrachteristic, and you don’t account for that characteristic, then you are not likely to be getting a fair representation of what you are trying to measure.
If I am doing a quality sample on production and I know a charateristic such as which portion of the implant machine the wafer was in, then to sample and not try to get representative selection from the different positions wouldn’t provide me the data I need. A political poll that is ignoring a key characteristic is useful for what?
Joran,
The American people and voters aren’t stupid (notice the number of presidential popular vote majorities for the republicans vs the democrats over the last 40 years). So I think the credibility gap issue will arise. What do we lose if people stop trusting polls? Thats a good question. But my gut says that while we shouldn’t govern to polls they do provide useful information about the sense of the country. But if we know that the polls are skewed, how much trust should we put in them. I suspect that successful campaigns will always have honest polls by a process of natural selection. And the successful campaign may have skewed polls for release, though as people notice the polls don;t match reality the skewed poll value will go down. But for the MSM, what possible reason would they not want to have the best, most accurate poll information to present? And if there is a known factor that is not being controlled for that would skew polls, what possible reason would you not have for fixing that problem? Or do we just wait for what has happened and is happening to the MSM, that people notice the skewing and stop trusting the source?
yetanotherjohn-
I clearly didn’t make myself clear (redundant?). My point was that the “issue” you raise about there being a serious problem with polls not getting partisan balance “right” is nonexistent. You are imagining it.
My further point was that there are _so_ many people obsessing over what is (essentially) a nonissue, that yes, eventually pollsters may have a credibility issue. But it will be because the public doesn’t properly understand polling, not because the pollsters are doing something wrong, or failing to fix a problem.
Check out the _fantastic_ post today by Charles Franklin,
http://politicalarithmetik.blogspot.com/2006/04/party-id-and-presidential-approval.html
I just posted an update to the main page that addresses some of these comments. And thanks to Joran for pointing out that outstanding post by Charles Franklin.
Joran,
Let me try to make myself clearer also.
Let’s assume we did a poll on women in the workplace. We want to find out the national opinions of those age 20 to 64 on whether it is an advantage, doesn’t matter or is a disadvantage to be a women in the workplace. The results come in and the opinions expressed don’t seem to match with other data such as the number of women in the workplace, number of women in management, etc. Its not that the numbers are totally wrong, but they seem to be consistantly off by a little and consistantly off in one direction. Then at the bottom you see that the poll is skewed 54% one gender, 46% the other. Now lets assume a series of polls that more or less continue the gender imbalance. And lets further say that we can calculate the “impact” of the gender imbalance and say that the effect would be to swing the numbers between 1 and 2 point for about 12% of the surveys and between 0 and 1 point for the remaining 88% of the surveys. Now lets further say that we see that house effects can have a definate effect on the percentage of the genders, though each house seems to cluster around a different set of numbers.
Now you can say, this is close enough. Sure there is a consistant skew, but its not much. Or you could ask, why is there a consistant skew in one direction? Why don’t they try to get a more accurate survey? What would a more accurate survey look like if it didn’t have the skew? Is the skew we can see also hiding something else that we can’t see. Are we getting a disproportionate number in one gender who is single, has young kids or whatever that might be impacting an even larger difference that we can’t see just by correcting for the gender imbalance?
Or do you just say, hey the tip of the iceberg that we can see is skewed doesn’t seem to be tilting the balance very much. There is no reason to try to improve polling. People may look at the polls and like the endorse or dismiss it, even if the known effects seem to be small, but who cares. Lets not worry about the fact that our sample which is supposed to represent the whole seems to be consistantly skewed because the impact we can see is relatively small.
Now gender identification is obviously an easier issue to poll and correct for than party identification. We have census information and human poll takers could potentially raise questions about the truthfulness of the respondants.
I’m not saying that black is white, up is down and all the polls would flip if we didn’t have the skewing. What I am trying to say is the sample doesn’t seem to be representing the whole, the skewing seems to be consistant and that just because you like/dislike the results of the poll we should still be concerned with a repeated skewing. Because trying to address and correct known skews should help us produce a more reliable poll. Which should be the objective of polling. We should be caring more about the objectiveness of the poll than how well we like or dislike the results of the poll.
“Now gender identification is obviously an easier issue to poll and correct for than party identification. We have census information and human poll takers could potentially raise questions about the truthfulness of the respondants.”
This is a massive understatement. You’re still implicitly assuming that the party ID polling numbers, either across all polls, or within a single pollster, are “wrong” compared to some idea in your head about what they “should” be.
The point is that party ID is such a wishy-washy idea (for many people, not all) that it is extraordinarily difficult to measure. Hence, we don’t have anything coming close to the “true” party ID numbers with which to compare our polls. In this sense, party ID numbers arising from national polls _are_ our best measurement (but a measurement of what?). There’s simply nothing better to compare them to.
In my opinion, this notion of a discrepency between party ID numbers in polls and “reality” arises from a misunderstanding of just how variable the Independent respondants are. We really have no way of knowing what the heck people mean when they claim to be Independent (or Dems/Reps, but the problem is more pronounced with Ind’s). It could mean wildly different things to different people. So it is entirely possible that there are many people who claim to be Ind’s, but who have essentially never voted Dem, and if you had a 10 minute conversation with them about politics, you’d come away describing them are rather conservative.
Or Dem leaning people could be calling themselves Independents? Who knows what the split is?
I agree that this would an interesting subject in which to do more extensive polling, but it would never really be applicable to “fixing” our sampling for national polls. The basic rule of thumb is that we should only adjust our sample for characteristics for which we can all agree on what they mean. It is reasonable to assume that we all know what it means to live in a city, or to be male, black or to live in California. But how do we define what it means to “identify with Dems”? It seems unreasonable to me to expect that to mean the same thing to everyone, or even most people.