« Still More Connecticut | Main | CT: No Exit Polls Today (Make that One Exit Poll - See Updates) »

August 08, 2006

A Lesson Learned the Hard Way

For those waiting on the results of the today's Connecticut primary, here is a cautionary tale about the limits of polling in a primary election.  Four years ago this week, I learned a lesson the hard way about what pre-election polls mean, and sometimes, what they don't.

Four years ago, I served as the campaign pollster for Rep. Lynn Rivers, then a four-term Democratic incumbent from Michigan who, by virtue of Republican redistricting, had been had been thrown into the same district with long time Democratic Congressman John Dingell.  The August 2002 primary in which they faced off has some parallels to the Connecticut contest worth considering. 

That primary. according to the Almanac of Politics, was "Michigan's most expensive House primary ever," with the two candidates spending a combined $4 million plus millions more spent by interest group independent expenditures on both sides.  Dingell, who served in Congress for more than 45 years including years as a powerful committee chair had worked more recently with prominent Republicans to pass legislation.  In the campaign, Dingell raised millions from favored interests, organized labor and prominent national figures in Washington.  Rivers had, by all accounts, a far more liberal voting record and the backing of groups supporting the environment, abortion rights, gay rights and arms control and of course, the EMILY's List national donor network. 

I hesitate to raise the Dingell-Rivers example because the differences with the Lieberman-Lamont race are in many ways as striking as the similarities.  John Dingell, unlike Lieberman, was and remains a revered figure to his Democratic base.  Unlike Lieberman, he maintained high favorable ratings from Democrats throughout the new district during the 2002 race.  Moreover, both candidates had pledged to support the eventual winner, so there was never any talk of either running as an independent.  While they differed on issues like the environment and abortion rights, Dingell and Rivers had very similar voting records on the issues of central concern to Democrats in 2002, such as the economy, health care and education.  The race ultimately turned on contrasts of age, style and experience.  And of course, the Iraq War -- the dominant issue of 2006 -- was still barely on the horizon in August 2002 (although two months after the primary both Dingell and Rivers voted against the resolution that authorized the war).   

The valuable lesson from this story involves the limitations of polling. Dingell led in most of the early polls, but the race appeared to tighten as Rivers ran television advertising toward the end.  The final polls conducted by the Detroit Free Press, EMILY's List and yours truly on behalf of the Rivers campaign all indicated either a dead heat or a slight but insignificant Rivers lead.  My last poll had Rivers two points ahead.  I never heard the exact numbers, but consultants for the Dingell campaign later told me that their internal polls showed essentially the same thing, except that they had Dingell ahead by a few insignificant points rather than behind.  On Election Day, given the trend, they expected to lose.  Yet when all the votes were counted, Dingell had won by a resounding eighteen-point landslide, 59% to 41%. 

So what went so wrong?  Other pollsters may disagree, but I believe we erred in our assumptions about turnout and the way we sampled "likely voters."   Michigan, like Connecticut, had recently switched to an held an August primary** and had not seen a seriously contested primary in years.  Turnout for the August primary in the four previous off-year Democratic primaries had varied between thirty and fifty thousand voters.   The most optimistic forecasters guessed turnout could go as high as sixty thousand voters.  But the actual turnout stunned everyone:  98,952.     That was nearly three quarters of the number that would later cast a ballot for Dingell against his Republican opponent in November. 

This brings me to an important structural difference between the Michigan and Connecticut primaries.  Unlike Connecticut, Michigan has no party registration and the 2002 primary was open to all voters regardless of their prior vote history.  So independents and even Republicans were free to cast ballots but just showing up on Election Day.  Moreover, most of the pollsters were sampling from lists of registered voters (rather than using a random digit dial or RDD sample to reach all telephone households) largely to match the gerrymandered district boundaries, but also to more efficiently identify those with a past history of primary voting.  We will never know for sure, but my assumption is that all the pollsters erred by screening to tightly for Democrats and past primary voters, the kinds of voters who were more likely to support a liberal candidate like Rivers.   I believe that our methodology led to an underestimate of the turnout surge in support of Dingell. 

So what does this imply for today's Connecticut primary?  Again, I am reluctant to draw too close a parallel because Connecticut's primary is open to registered Democrats only, and the public polls have been based on RDD sampling.  The combination makes it less likely that the polls will miss as badly as in the Rivers-Dingell race. 

Where the Dingell-Rivers race may be most instructive is regarding turnout.  As in Michigan's 15th District four years ago, Connecticut voters have been bombarded with an unprecedented flow of information through all media.  As Connecticut's Secretary of State Susan Bysiewicz put it yesterday, "You can't turn on the television, pick up the newspaper or listen to the radio and not know there's a primary,"  Add in door-knocking, phone banking, robo-calling and other activity and you have one inescapable message:  There's a primary and it features a choice between two candidates with very different positions on the Iraq War.  So Bysewicz has good reason to guess that, according Greenwich Time, "turnout for the primary could reach 40 percent, well above the 25 percent historic average for primaries featuring a governor's race on the ballot." 

Some may argue that a larger turnout helps Joe Lieberman in the same way it helped John Dingell four years ago.  I'm not so sure.  This primary is open to only Connecticut's 696,823 registered Democrats.   Any cross-over phenomenon is limited to the roughly 29,000 who registered for the first time or switched their registration to vote in this primary.   Among the bulk of registered Democrats, I don't see much of an opening for an influx of enthusiastic, energized Joe Lieberman supporters who do not normally vote in primary elections.  The potential for a big turnout among passionate anti-Bush, anti-war Democrats is another story entirely. The Quinnipiac poll completed in mid-July reported results for all registered Democrats, not just the "likely voter" subgroup.  Ninety-one percent (91%) of the Democrats disapproved of President Bush's job performance, 93% disapproved of his performance on Iraq and only 13% agreed with Joe Lieberman's position on the war. 

So here's my bottom line:  Given the turnout puzzle means, I wouldn't be surprised to see tonight's result differ significantly -- in either direction -- from the recent poll results.  Historically, polls in this sort of primary have not been the most reliable.  However, if I had to guess, the energy and intensity appear to be all on Lamont's side and a large turnout probably works in his favor today.  But that's just a guess.  Time will tell.

**Correction:  I obviously erred in the original version of this post in stating that Michigan had recently switched to an August primary in 2002.  As multiple commenters point out, they have had August primaries for a long time.  I'm not sure what memory lapse led to that goof, but I apologize for the error.   

Related Entries - Likely Voters, The 2006 Race

Posted by Mark Blumenthal on August 8, 2006 at 02:17 PM in Likely Voters, The 2006 Race | Permalink

Comments

Mark,

It would be interesting to overlay the age demographic in the Dingell/Rivers primary with the polling and actual results.

As with the Lieberman/Lamont race, primary polling results not heavily weighted towards older voters will always overstate the “young” voter’s choice for the “new” candidate.

Unless Joe was caught on camera with two naked alter boys and a pound of coke, it would be very hard to imagine him loosing the primary.

Posted by: jwest | Aug 8, 2006 3:16:59 PM

Mark:

Sorry to be so blunt, but it's ridiculous to suggest John Dingell wasn't and isn't great on the environment. This is something I wrote back when I was a regular guest blogger at Daily Kos:

The younger Dingell first worked in Congress in the late 1930's--as a page--where he witnessed the backlash against the later initiatives of the New Deal and the opposition to FDR's attempt to pack the Supreme Court. Dingell later served in the army, worked as a park ranger, and was an assistant county prosecutor when, after the death of his father in 1955, he was first elected to Congress.

Dingell was mentored by Sam Rayburn and others veterans of the legislative process, and he began working his way up the ranks. He held the gavel when the House passed Medicare. In 1964, after redistricting put him into a seat held by the only Northern Democrat to oppose the Civil Rights Act, he won a decisive victory based on his support of labor and civil rights. He eventually sponsored many of the legislative pillars of our environmental laws--the National Environmental Policy Act, the Endangered Species Act, the Marine Mammal Protection Act, The National Wildlife Refuge Administration Act and the Clean Air Act Amendments of 1990. Under his chairmanship, 40% of all bills went through his Energy and Commerce committee, inlcuding the break-up of AT&T. Dingell's position was "if it's sold, moves or burns, it's mine."

Dingell particularly relishes investigations, where his sharp mind and pugnacity strike fear in lazy bureaucrats and unscrupulous contractors. It was Dingell's committee that discovered that Stanford University charged the federal government for the wedding of the daughter of the University's president, and it was Dingell who found out the Pentagon had paid $640 for a toilet seat.

In every Congressional session for fifty years he's introduced for a single-payer national health care program similar to Canada's...

You make some good points about the pitfalls of devising your likely voter models. However, there are a few additional points to be made by someone who had just moved in to that district a few months before the 2002 primary and had been around Michigan politics for a long time:

1. The Rivers campaign appears to have been incompetantly run. This was in great contrast to the Dingell campaign, which was solid. In one area in particular the Dingell campaign destroyed the Rivers effort: voter ID and GOTV. I had just moved to Ann Arbor, and got a call from the Rivers campaign on election day--I was working at home for part of the day--urging me to go vote. I had already voted--for Dingell--and was not someone the Rivers campaign should have been contacting. But since they apparently hadn't done any ID work, they didn't know who their voters were.

2. The charges have to be believable. EMILY's list had an awful run that primary season. They picked the losers in numerous races, including backing a woman in the Chicago area who's father made anti-Semetic comments which she was slow to denouce. (For those who don't remember, that woman lost her race to a guy name Emanuel. First name Rahm. Currently has a good gig as Chair of the DCCC.) They ran ads against Dingell attacking him for voting against raising CAFE standards. That's not actually a hugely negative vote in SE Michigan, as most people there will side with the American auto industry on that issue. Furthermore, it was BS, because with the exception of one Republican, EVERY member of Congress from Michigan voted against raising CAFE standards. Yes, that included Lynn Rivers.

3. Rivers shouldn't have run in that seat. She could have shifted over to the seat now held by Thaddeus McCotter, who I think she would have beaten. But she didn't want to move out of Ann Arbor, and while she won in the city of AA (and I think the entire Washtenaw County area) she was destroyed in the rest of the district, where nobody knew her.

4. On the money thing, if it weren't for EMILY's list, Rivers wouldn't have had enough money to buy the staff lunch at Zingerman's deli in A2. Almost all her money came from EMILY's list.

In short, your warnings about models should be heeded. But citing the Dingell/Rivers campaign is dicey, because it wasn't statewide, yet it was occuring in the broader context of a highly contested statewide primary for governor. Plenty of factors bigger than the house campaign played in to the turnout of that race.

Oh, one final thing: I think you were probably hurt by the flaws in the research provided to you by the campaign. For instance, besides the flaws in the view of Dingell, there's the past campaign comparisons. Michigan has had August primaries forever, at least back to the 1970's and I suspect all the way back to the implementation of the state's current constitution, in 1963. Maybe there hadn't been a contested race in the Congressional primary, but Michigan had had contested primaries for Gov in 1998 and 1994, and in 1994 there was also a contested primary for Senate. Thus, you were probably operating under some bad assumptions that weren't any fault of yours but stemmed from bad research.

Posted by: DHinMI | Aug 8, 2006 4:34:48 PM

An important (though possibly unanswerable) question should be, what moves one to register democrat? I would venture to guess that these ~4.7% of the overall democrat registered electorate are not the very liberal who feel marginalized. The bell curve of American political opinion is pretty tight (in contrast to, say, Europe) and politically active voters would likely already be registered, certainly after the 2004 call to arms. Centrist voters, however, who might normally be ambivalent may feel the need to weigh in on the side of the moderate Dem. They may also be composed of GOP who feel that an open seat battle would be an easier battle, though I am skeptical that people are really that scheming (due to laze more than anything else).

Posted by: zev | Aug 8, 2006 7:43:30 PM

They may also be composed of GOP who feel that an open seat battle would be an easier battle, though I am skeptical that people are really that scheming...

FYI - In CT, an independent can re-registe rinto either paerty effective essentially immediately (I think independents had to re-register by Monday noon to vote as a Dem on Tuesday.)

However, if a Rep attempts to re-register, there is a three month deferral; anyone scheming Reps had to re-register as a Dem by early May.

I only learned this because I began my own scheming much too late (actually, I wasn't sure if I was a Rep or an Ind - turns out I'm a Rep.)

Posted by: | Aug 8, 2006 7:58:25 PM

The perils of being a beltway pollster (even a smart one!): first Tuesday in August has been the primary date in Michigan for at least the last 30 years (my voting lifetime).

Note to DHinMI:
Being against raising CAFE standards really precludes a rating of "great" on the environment.

Posted by: nony | Aug 9, 2006 1:06:02 AM

nony: you seemed to have missed several points. For instance, if Dingell isn't good on the environment because of his vote on CAFE standards, Rivers was just as "bad," because like every Michigan Democrat and 15 of the 16 Members from Michigan, she also voted against raising CAFE standards.

It's very important to be binary, as you point out. Never mind all the stuff Dingell has done in his career, like passing the legislation that created the EPA. If he's bad on one thing, then he sucks on all of them. Thanks for that lesson in proportionality.

Posted by: DHinMI | Aug 9, 2006 12:44:35 PM

DHinMI and nony: You are right and I goofed. Michigan has held Democratic primaries in Michigan for ages. Even Michael Barone emailed to call me on that one. But blame that goof on my memory, suddenly going south just a few years past my 40th birthday, not the Rivers campaign. Not sure how I had that confused - my apologies for the error.

DHinMI:

First, The point of my post while the Dingell-Rivers primary could somehow help predict outcome of Lamont-Lieberman, but that it was a good model for the factors that produce an unprecedented high turnout and (sometimes) erroneous poll results. Yesterday's CT turnout exceeded 41% of registered Democrats, which according to this web site makes it a bigger turnout than any of the Connecticut elections gathered by this web site since 1970 and considerably bigger than the 25% reported by CT's Secretary of State as average for Gubernatorial elections.

Second, we were certainly aware of the contested primary in 1994, since that was the election in which Rivers won a open August primary (with my help) after the retirement of Congressman Bill Ford (though I now see I also goofed by labeling those statistics as applying to "off year" elections). Here is the data I have on hand for turnout in the precincts that now make up Michigan's 15th District:

1994: 45,347
1996 31,782
1998 49,774
2000 33,537
2002 98,863

So again, 2002 turnout in the 15th District was more than double the turnout in 2002 and 1998 (when Michigan had a hotly contested primary for Governor.

One of the challenges of gathering this data in a Congressional primary (rather than a statewide contests) is the redrawing of precinct boundaries. The 15th District did not exist before 2002, of course, so to tabulate the data above for 1994 to 2000, we had to go back and gather precinct level data, determine which precincts fell into the new 15th District (which meant dealing with newly split or renamed precincts in some cases).

The contested 2002 governor's race certainly helped increase turnout that year (just as Connecticut's governor's race presumably also helped boost turnout there). But that race does not come close to explaining why turnout in the 15th District was nearly twice as high that year as in 1998. The total statewide turnout in 1998 was 729,665, in 2002 it was 1,046,680 - a 43% increase yes, but not double.

Finally, if you want to use this forum to re-fight the Rivers Dingell race, that's your prerogative, but it was not my intent. I did not say that Dingell "wasn't and isn't great" on the environment (although the Rivers campaign certainly did). Rather, my point here was that (a) that the environment was one of the issues about which they differed and that (b) those issues were far less salient to voters in 2002 than Iraq has been in 2006. In addition to nony's point, they differed on the Bush energy bill (the one that would have opened the Alaska wilderness to oil drilling) and Rivers was endorsed by the Sierra Club. Even fans of Dingell acknowledged that there were differences on their environmental philosophies.

Posted by: Mark Blumenthal | Aug 9, 2006 1:10:39 PM

you suck

Posted by: jhon | Dec 24, 2006 9:57:24 PM

The comments to this entry are closed.