A story on the front of Sunday’s New York Times shines new light on the work on support for the Iraq War by Chris Gelpi, Peter Feaver, and Jason Reifler that MP wrote about last summer. With the work of these past and present Duke academics again in the spotlight, I thought I would post some links to their work as well as to some dissenting opinions and offer a few additional thoughts of my own. One notable problem with the Gelpi, et. al. thesis is the same issue I raised in my last post. The most crucial questions used in their research probe attitudes which – for many voters – may be formed on the spot during the course of the interview.
First, a conflict re-declared: Jason Reifler is both a friend and a former valued employee of MP’s firm. I reviewed my clumsy discovery of his work on this subject back in June.
MP recommends reading the story in full, but here is a quick summary: According to the Times‘ Scott Shane, the "relentless focus on the theme of victory" in last week’s speech by President Bush reflects the growing influence of Feaver, who joined the staff of the National Security Council earlier last year. Shane also neatly summarizes the central thesis from the surveys and statistical models by Feaver and his academic colleagues: "Americans would support a war with mounting casualties on one condition: that they believed it would ultimately succeed."
Shane also goes to great lengths (including a call to Adobe) to demonstrate the Feaver’s role in drafting the new strategy. "Despite the president’s oft-stated aversion to polls," the administration recruited Feaver after he presented his "analysis of polls about the Iraq war." Moreover, Shane noticed Feaver’s name listed as "author" in the "document properties" of the PDF of the strategy document and as a result got the White House to confirm that Feaver "helped conceive and draft the plan."
[As of this writing, "feaver_p" remains listed as the document "author." To see for yourself, download the document, open with the standalone Acrobat reader, use the menus to select File / Document Properties and check the "author" field under the "Description" tab].
Now, some links. First the two key papers by Gelpi, Feaver and Reifler:
- Christopher Gelpi, Peter D. Feaver, and Jason Reifler. 2004. "Casualty Sensitivity and the War in Iraq." This is the paper referenced in the Times yesterday that will be published in an upcoming addition of the journal International Security.
- Christopher Gelpi, Jason Reifler, and Peter D. Feaver. 2005. "Iraq the Vote: Retrospective and Prospective Foreign Policy Judgments, Candidate Choice, and Casualty Tolerance." A companion paper that looks at the influence of attitudes about the Iraq war on the 2004 presidential vote.
Next, the writings of those who have been critical of the Gelpi et. al. thesis that perceptions of success drive support for the war (or take a somewhat different approach):
- John Mueller. 2005. "The Iraq Syndrome." Foreign Affairs, November/December 2005. Mueller’s 1973 book War, Presidents and Public Opinion, argued that public support for war declines as casualties mount.
- Adam Berinsky. 2005. "Assuming the Costs of War: Events, Elites, and American Public Support for Military Conflict." Berinsky’s paper (which covers material included in his book Silent Voices) argues attitudes about potential success in Iraq may stem from general views about the war, not the other way around. See especially footnote #3 in his paper, or his comments here back in June.
- David Moore. 2005. "’Succeeding’ in Iraq" (subscription required). Like Berinsky, Gallup’s David Moore also took issue with the direction of "causality," between perceptions of success and support for the war.
Finally, two more academics – one an occasional commenter on this site -take a slightly different approach to similar data (though they are not necessarily critical of Gelpi et. al.):
- Erick Voeten and Paul Brewer. 2004. "Public Opinion, the War in Iraq, and Presidential Accountability." Brewer, a sometimes blogger also discussed his findings on his own site and in a comment on MP.
Now, I must confess that in this academic debate, I am more a student and observer than participant. However, to the extent that these authors disagree, I find Adam Berinsky’s arguments both persuasive and consistent with the variations in survey data on Iraq that I described last week.
To consider Berinsky’s point, let’s review the Gelpi et. al. findings. They used statistical modeling to show that "prospective evaluations" of how likely the US is to succeed in Iraq are most important in predicting overall support for the war. Berinsky raises some good questions about the underlying attitudes those questions were measuring. In looking at American’s attitudes about both World War II and Iraq, Berinsky finds that "large segments of the mass public possessed no knowledge of the most basic facts of these conflicts" (p. 2). He further argues:
[T]he causal arrow between perceived success and war support could run from the later to the former, rather than vice-versa, as Gelpi, Feaver, and Reifler (2004) argue. Analysis of data from the Iraq War Causality Survey (see below) suggests that, at an empirical level, perceived success is best characterized as another measure of support for war, itself influenced by partisan elite discourse.
Gallup’s David Moore makes a similar argument in his (subscription only) analysis:
The causal model cannot be proved, as least by the data obtained by the three authors; in this case, causality is more an act of faith than a provable dynamic.
Moore’s skepticism about these models arises from essentially the same issue I raised in my last post about questions about prospective policy in Iraq. The huge variation depending on question wording suggests that a large portion of Americans are not quite sure what they want to do next, and as such, tend to form judgments in reaction to specific language of the poll. As I highlighted back in June (and as Moore pointed out in his analysis), pollsters get very different results in questions about success in Iraq depending on how they define "success." A Westhill/Hotline survey found that the percentage expressing "confidence" in success in Iraq varied between 37% and 64% depending on the definition of success.
The basic argument that Berinsky and Moore make is that when a pollster asks Americans to rate the chances of success in Iraq in an intentionally vague way (as Gelpi, Feaver and Reifler do), the answers tend to reflect their general views about the war rather than specific pre-existing attitudes about the probability of American success. So, they argue, two different ways of measuring the same thing end up being highly correlated and the fact that perceptions of success appear to predict support for the war in a statistical model is a bit spurious. At least, that’s the argument.
MP remains intrigued by this research, the continuing academic debate and its importance in driving the Bush administration’s communications strategy. The practical political question, however, is whether that PR strategy on Iraq will help bolster approval of his conduct of the war, especially if casualties remain at current levels. On that score, count MP as dubious.
[Typo corrected 12/5]
This is totally off the subject, but after reading your credentials, I did not think it would hurt to ask you a question and I didn’t see an email link anywhere on your site.
Anyway, I have a paper due tomorrow night in my statistics class. We are to write 3 pages on what went wrong in the exit polls in 2004 and what could or should be done differently in 2008. I am not asking you to write my paper or anything. I have already been researching and reading so much information. It is all pretty much the same and there isn’t much definite reasoning and I have yet to come across any ideas for solutions to avoid this in 2008. Would you care to either email or remark just some short ideas that I could perhaps research and ellaborate more on myself in my paper?
What a great example of the correlation vs. causality problem in statistical reasoning! No way to cleanly resolve this except to do the experiment, and that will be hard to generalize…
“Everyone knows” that a ball team performs better if both the players AND THE FANS have high morale and enthusiasm for the game. But when the home team is our armed forces and the whole country is in the stands, we’re not sure about standing up to cheer. Analogy seems as poor a tool as statistics.
However, this could be just the tip of the iceberg. If the Bush administration has finally taken to heart the idea that asymmetric warfare has a large media and information component–and they’re going to get serious about fighting in that arena–then we should expect to see evidence like Feaver’s work fairly often.
You guys have probably argued this one before, but it seems that polling can somewhat measure people’s perceptions, but perceptions are always based on faulty data since nobody ever sees the total picture of what is really going on. Based on the Holy Spirit’s messages on The Christian Prophet blog, from an expanded spiritual point of view victory has already been achieved in Iraq.
Thanks again for devoting MysteryPollster space to a discussion of our work.
I do take the problem of endogeneity and causal inference seriously, and the folks who flag this as a concern are right to do so.
We have, however, made several attempts to address the issue of causal direction. For example, we have constructed a simultaneous equations model to estimate any possible relationship between casualty tolerance and perceptions of success. It indicated the causal flow was from success to casualty tolerance. Moreover, in order to guard against the possibility that these were all simply part of a broader “support Bush” attitude, we reran our analyses separately for Republicans and Democrats and for those who approve of Bush and those who do not. Our basic result (i.e. perceptions of success has the largest impact on casualty tolerance) holds up across all four categories. Interestingly, our model fits better for those who DO NOT approve of Bush.
We are currently trying to set up a panel study to reinterview folks we interviewed in 2004. Again, this may give us further leverage on causal direction.
Nonetheless, I agree with the point that this issue may be best addressed through experimental work. I hope to do so in the (not too distant) future.
Finally, on the issue of perceptions vs. reality I would agree that we cannot parse precisely where these perceptions come from (i.e. how much does spin matter), but neither I nor my coauthors would construe our results as suggesting that simply by SAYING the word victory that Bush can rally the public.
Best,
Chris Gelpi
PS – I appreciate MP’s effort to collect up a number of academic works on this subject. It is a good group of papers (perhaps worthy of a special issue in a journal or something!).
I did want to flag for people that I have a response to John Mueller’s Foreign Affairs article that MP cites. It will be in the upcoming issue of Foreign Affairs – along with a rebuttal from Mueller.
Thanks for the nice post. And thanks to Chris for a thoughtful post. I plan to post a more lengthy discussion later, but I have a couple quick questions to reply to Chris’s post.
1. Why do you use casualty sensitivity as your primary dependent variable, rather than war support? The later seems to me to be the more interesting and important quantity, especially since you have argued (in this paper and elsewhere) that casualties are not the primary determinant of support for war. Using war support instead of casualty sensitivity (as I did in my survey) gives me results that differ in spirit from yours.
2. (Of the more technical nature). Simultaneous equation models are heavily dependent on exclusion restrictions. I saw an early version of the paper that had some debatable restrictions (support for the military in general, as I recall). The point here is not that simultaneous equations models are not useful; it’s just that their usefulness is heavily dependent on the assumptions that underlay them. So I don’t think this solves your problems. Experiments are almost certainly the way to go. What those would look like, I’m not sure…
Hi Adam – good questions and I thought they deserved a better response than the above comment on them 😉
As for using casualty tolerance, I certainly would not want to say that it is the ONLY important attitude toward the war, and it seems possible to me that other attitudes may have have different structures.
We focus on casualty tolerance because we are specifically interested in the question of whether the public is willing to support continuing a conflict once it has begun. We focused on that question because it seemed to match most closely to the policy problem facing policy makers. Given the prevalence of rally effects and the wide discretion given to the President in foreign affairs, getting a conflict started did not seem like a problem from a public opinion standpoint. The key issue was maintaining support.
We believe that casualty tolerance measures that attitude better than war support primarliy because “support” for the war may conflate attitudes toward whether the decision to launch the war was the “right thing” and whether we should continue a mission that is underway. While I don’t know of anyone else asking about casualty tolerance the way we do (since it is expensive), I think the closest type of question in commercial polls would be questions about whether respondents support withdrawing troops. Clearly, however, one would need to be careful in how to structure the response options since both Bush and Murtha would say that they support withdrawing troops – the question is under what conditions.
As for the exclusion restriction problems and the need for experiments – I agree on both points. I am working on ideas for experiments.
Many thanks to Gelpi and Berinsky for posting comments and adding great value to this discussion. Please note that I deleted a bit of comment SPAM (the “above comment” that Gelpi referred to).