The latest poll release from Gallup includes what appears to be a new approach for reporting their data on the job approval rating of President George Bush. They include a "three-poll rolling average" as a "smoothed approval estimate" based on the "Samplmiser" program created by Yale Political Scientists Donald Green and Alan Gerber.
The smoothed averages are highly relevant to the results from the latest Gallup/CNN/USAToday survey, noted here yesterday, which provide a common dilemma. As the new release from Gallup puts it (available to non-subscribers as of this posting), the latest rating of 39% approve on the survey conducted Feb. 9-12 "marks a slight, three-point drop" from the 42% reported by Gallup on a survey conducted Feb. 6-9. But is that change real or just the result of the usual random variation, also known as sampling error, that we get from drawing a random sample of voters? Gallup’s Frank Newport and Joseph Carroll point out, appropriately, that "the three-point decline in approval between the two recent polls is not significant given the margin of error."
However, the release also provides numbers based on the two new averaging methods. Both methods estimate Bush’s current approval at 41%, the same result we get by averaging the national conventional telephone surveys released by other pollsters so far in February:
The table [in the release] presents two additional ways of approaching the issue of presidential job approval. The middle column displays the three-poll rolling average. The right-hand column displays a smoothed estimate calculated using the Samplemiser program developed by Yale University professors Donald Green and Alan Gerber. This program uses the available data from the most recent series of poll results — rather than just the latest in isolation — to estimate the population value on job approval.
As can be seen, both of these approaches yield the same estimate — 41%. In other words, either "big picture" approach to estimating the population value of Bush’s job approval rating at this point suggests that it is slightly above 40%. In essence, both the Samplemiser and three-poll rolling average procedures adopt a "wait and see" approach, discounting the influence of the most recent poll taken by itself, and instead emphasizing longer-term trends using larger numbers of interviews.
If Bush’s job approval ratings continue to be below 40% in future polls, these procedures will adjust downward, under the assumption that the underlying population value for job approval has more definitely dropped. And, if future polls show Bush’s approval ratings to be back above 40%, the smoothed estimates will remain at their current levels or will gradually move back up.
A few notes: Green and Gerber’s "Samplemiser" program is available through their Yale University web site. Anyone can plug in data and specifications from their own surveys (or any available series of public polls) and generate "filtered and smoothed" estimates. The site also provides examples using actual data and a PDF of a 1999 article that Green and Gerber wrote with Suzanna De Boef for Public Opinion Quarterly (63:2) that describes their
The gist of the Samplemiser procedure appears to be as follows: It uses a statistical model to filter out and smooth over the purely random variation between polls. Unlike a simple rolling average, the procedure creates a "weighted average" that takes into account the sample size of each poll, the time lag between polls and any long term trend in the results. It also allows for the calculation of a revised margin of error for the Samplemiser estimates, something not provided in the Gallup summary. MP, though previously unfamiliar with the Samplemiser procedure, is intrigued and hopes to learn more.
Gallup’s decision to report both the three-day averages and Samplemiser estimates appears to be in reaction to calls from critics to weight survey results by party identification. MP notes that the most recent Gallup survey shows a slight decline in Republican identification consistent with the non-significant drop in Bush’s job rating. Party ID on the Feb 9-12 survey was 30% Republican, 31% Democrat, 39% independent. The Republican percentage was three points lower and the independent percentage six points higher than the average of party results reported on their first four surveys in 2006 (33% Republican, 32% Democrat, 33% independent – complete party data is available to Gallup subscribers).
Actually the difference in party identification between the two most recent Gallup samples is somewhat larger if you include party leaners. The prior Gallup sample had a 1 point Democratic identification advantage–the smallest in some time. The current sample has a 7 point Democratic advantage–close to the average for the past several months. The decline in Bush approval may be explained entirely by this change in the partisan composition of the sample–Bush’s approval rating among Democrats, independents, and Republicans changed very little and the changes that occurred were apparently offsetting.