The Question That Answers Itself

A few weeks ago, our friend Mickey Kaus described a question asked on a recent Time Magazine poll as having "comically biased wording."  I was not ready to be quite so harsh about that particular poll.  Well, this week courtesy of Carl Bialik of the Wall Street Journal Online, we have a different poll conducted by Zogby International whose questions and their wording truly meet the "comical" standard.   

The second half of Bialik’s weekly Numbers Guy column looks at a recent Zogby survey on online gambling sponsored by the online gambling industry.  "It appears that the sponsor of the poll influenced the way it was conducted," Bialike writes, "particularly in the way the questions were phrased."  He is putting it mildly.

Here is the most brazen of the questions used in the survey press release to support the assertion that "Americans overwhelmingly do not want" federal laws restricting online gambling:

More than 80% of Americans believe that gambling is a question of personal choice that should not be interfered with by the government. Do you agree or disagree that the federal government should stop adult Americans from gambling with licensed and regulated online sports books and casinos based in other countries?

Yes, you’re reading that right.  The text of the Zogby question actually answers itself.  Or, to be more precise, it tells the respondent what "80% of Americans believe" about government regulation of gambling just before asking them what they believe about such regulation.  It is thus not exactly surprising, as Bialik put it,

that after being told that most Americans don’t want the government to interfere, some 71% of the respondents to this question signaled they, too, were against a government ban.

To be serious for a moment, the issue here is that the poll press release makes the following claim: 

[The poll] establishes that Americans overwhelmingly do not want the federal government enacting laws that restrict a recreational activity such as online gambling.

No.  At best, these results establish that a pollster can push respondents to oppose such restrictions.  The obviously leading nature of the questions cited in the release makes them of little value in measuring the opinions Americans currently hold about online gambling. It is one thing to design "projective" questions in order to "see how different arguments play" (as Humphrey Taylor of Harris Interactive puts it in the Bialik article).  It is quite another to try to pass off such projective questions as a "fair and balanced" reading on what Americans currently think, which is exactly what this press release does. 

There is much more in Bialik’s piece, including reaction from AAPOR President Cliff Zukin and a response from Zogby spokesman Fritz Wenzel.  It is definitely worth reading in full.

However, MP has a hunch there is more to this story.

I am doing some additional digging, but here’s a hint:  The press release describes the study as a "scientific poll of over 30,000 likely voters" interviewed over a two week period with a "margin of error" of " 0.6 percentage points."  Moreover, according to Bialik’s column, the survey sponsor claims they paid "less than $10,000" for the survey. 

I’ll put it this way:  I’m aware of no pollster or calling center that will complete a telephone survey of 30,000 likely voters for less than 33 cents an interview. 

More to come…