Another update on my last post on the mysterious difference in opinion on whether Jeb Bush should run for president as measured by two publicly released surveys of registered voters in Florida: One by the Quinnipiac University Poll and one by Strategic Vision, a public relations firm that polls for Republican candidates.
Late on Friday last week, Doug Schwartz of Quinnipiac sent an email that answered all of my questions about the demographic and partisan profile of his sample. Although David Johnson of Strategic Vision emailed some comments on Friday, he did not provide any data on their survey’s demographics or partisanship and did not respond to a second specific request for such data that I sent yesterday morning.
The following table shows the Quinnipiac data (I posted on the jump more information provided by Quinnipiac on the text of the their party identification question and definitions of their regional breaks):
All of this leads me to an important general point: Much of the survey research process is hidden from view. We can argue about the relative merits of various likely voter screens or of random digit dialing vs. list based sampling, but so much of what makes a quality survey – such as the experience and training of the interviewers, the procedures used to administer the sample – are out of reach of ordinary data consumers. So how do we assess the work of a polling organization? We often rely on the reputation of the pollster or of the news organization that sponsored the survey, but there is one objective standard that ought to get more emphasis: Disclosure. To put it simply, I have the most faith in organizations that go out of their way to be transparent and to answer questions when controversies arise.
In this case, the Quinnipiac Survey stepped forward and released data on the demographic, regional and partisan make-up of their sample. If nothing else, the Quinnipiac Poll should be strongly commended for its transparency on this issue.
So far, at least, Strategic Vision has not. Hopefully, we will hear from them soon.
Also Released by the Quinnipiac Poll
Text of party identification question: Generally speaking, do you consider yourself a Republican, a Democrat an Independent, or what?
Strategic Vision makes up their polls. That is why their data does not match up. They claim to do all of their polling in-office. Yet, I have been to their office in Atlanta and A) it isn’t big enough to be a call center and B) during the Presidential contest, when they were releasing 8 polls that all were done during the same 3 day period, there was nobody there during the day. Basically, for the polls to be valid, they’d have to be doing serious work during the 5-9 pm time slot and also call backs during the day.
I spoke with Gallup about how many people work at their call centers and calculated that Strategic Vision would need way more than could comfortably be working out of their suite in Atlanta. In addition, Gallup’s low response rates (which are consistent through the industry, from what I understand) coupled with how long it takes even to complete a simple 15 question survery cast a lot of doubt on Strategic Vision’s alleged operation.
Rumor has it that they are also affiliated somehow with some high level GOP/Bush campaign officials, but I can’t remember exactly what the link was supposed to be at this time. Keep up the scrutinty. These frauds need to be exposed so they can’t influence public opinion in Georgia and Florida in 2006.