Impeachment Update: No Signs of Non-Response Error

Impeachment Uncategorized

Support for impeaching and removing President Donald Trump from office has increased to near majority support on a wide variety of polls since September. And while some struggle to reconcile the national trend with recent state-specific results, there is no indication the latest numbers are marred by the sort of “non response” error I speculated about on FiveThirtyEight a few weeks ago. 

First, the partisan composition of national polling samples has remained stable. On five national polls (CNN, Quinnipiac, Marist/NPR, NBC/WSJ and Washington Post) party ID remains essentially unchanged before and after the big increase in coverage of the Ukraine scandal in late September. An average of results across the five organizations shows no change at all – exactly 32% identify as Democrats and 27% as Republicans before and after House Speaker Nancy Pelosi announced support for an impeachment inquiry in September. 

Second, a new survey by the Pew Research Center that interviewed the same respondents over time found “about one-in-ten adults (9%) who had opposed the House opening impeachment proceedings last month now approve of the decision to conduct an impeachment inquiry.” This individual-level change implies that the aggregate changes we are seeing in other polling is real. 

So while there has been a surge in news coverage of the Trump/Ukraine scandal and the impeachment inquiry, it has not thrown off polling the way the “exceptionally big” news events I recounted in the FiveThirtyEight article did in recent years. 

That said, attention paid to the story did increase significantly in late September. Just before Pelosi endorsed an impeachment inquiry (September 22–24), 32% of Americans surveyed by the Economist/YouGov poll said they had heard “a lot” about news about “President Trump urging the President of Ukraine to investigate allegations of corruption in that country.” A week later (Sep. 28-Oct 1), 54% had heard a lot about the story. 

After that initial bump up, attention paid to the story leveled off and now may be trending slightly down. YouGov’s next three surveys tweaked the language of their question slightly – asking about “Donald Trump withholding military aid to Ukraine while urging the President of Ukraine to investigate allegations of corruption against the Biden family” – and found those reporting “a lot” of attention ticked down from 52% to 51% to 47% over the past three weeks. 

The caution offered in my FiveThirtyEight piece remains, because while the impeachment story is big, it could be far bigger. Using the same answer scale, the Economist/YouGov survey found 60% of Americans had heard “a lot” about Brett Kavanaugh’s nomination in a surveyconducted in early October 2018, just after the initial Kavanaugh/Blasey-Ford hearing.

Perhaps more important, the prior instances of differential non-response bias I cited all exhibited some of what political scientist John Zaller calls a “one message environment.” News following the initial presidential debate in 2012 featured bipartisan agreement that President Obama had performed poorly. Similarly, the initial news coverage of the Access Hollywood recording in early October 2016 featured condemnation of then-candidate Donald Trump from across the partisan spectrum. While the Kavanaugh/Blasey-Ford episode was something of an exception, coverage shift to a focus on Kavanaugh as a winner and his opponents as unsuccessful as his confirmation neared. 

In such an environment, the partisans of the “winning” side may be more likely to participate in surveys than the “losing” side. That’s the theory, at least. So far, such a dynamic has not been in play in the impeachment story.

Mark Blumenthal

Mark Blumenthal is the principal at MysteryPollster, LLC. With decades of experience in polling using traditional and innovative online methods, he is uniquely positioned to advise survey researchers, progressive organizations and candidates and the public at-large on how to adapt to polling’s ongoing reinvention. He was previously head of election polling at SurveyMonkey, senior polling editor for The Huffington Post, co-founder of Pollster.com and a long-time campaign consultant who conducted and analyzed political polls and focus groups for Democratic party candidates.