Opinion

DailyKos poll scandal: Lies, damn lies, and statistics

Kristen Soltis Contributor
Font Size:

In a speech to the House of Commons in 1941, Sir Winston Churchill declared, “Nothing is more dangerous in wartime than to live in the temperamental atmosphere of a Gallup poll, always feeling one’s pulse and taking one’s temperature.”

While we may not be in a formally declared state of war, the realm of politics these days is in a constant state of battle. And now more than ever, campaigns, pundits and the media have the ability to engage in an interminable cycle of pulse-checking and temperature-taking.  From the original Gallup polls to today’s fire hose of public data, the public’s ability to consume data about its own opinions has increased dramatically.

Yet as the amount of data available has grown, the knowledge we have about the sources of that data has not.  The appetite for information about political races at all levels has exploded, and the speed demands of the internet age mean it is better to blog now, worry later.

Enter DailyKos and the Research 2000 (R2K) scandal this week.  In recent months, R2K’s polling on behalf of DailyKos had been criticized for offering data that was suspiciously favorable to Democratic narratives.  They showed Bill Halter leading Blue Dog Democrat Blanche Lincoln in Arkansas – a result that was not to be.   They showed Tom Campbell surging against Carly Fiorina in the California Senate primary just as the opposite situation was unfolding.

Their research also was used to make the case that large quantities of Republicans hold “fringe” beliefs, and that, among other things, Republican leaders like John Boehner were very widely disliked by the electorate.

On Tuesday, Markos Moulitsas sent out shockwaves with a post alleging that R2K’s research was fraudulent.

I have no idea if the data produced by R2K was fabricated entirely, manipulated after the fact or produced properly in good faith and just happened to yield results that raise suspicion.  Plenty of bright analysis has been done by others casting doubt on the results, and I’ll leave the final verdict to the courts (R2K and DailyKos are now locked in a legal battle over the scandal).

I don’t know why, on request, R2K didn’t “show their work” by sending their raw interview data to DailyKos, a simple move that should be standard practice and something that probably could have put an end to this controversy right off the bat.

But what I do know is that there is at least one positive outcome of this situation:  consumers of polling data got a much needed wake-up call.

There’s a perception that polling numbers are simply the results of interviewing a certain number of people.  If a poll shows 50 percent of people believe a statement, it must mean that exactly half the people the pollster called said they believed it.  There’s a sort of blind trust placed in polling data, and that data ends up driving huge political narratives and strategic decisions in campaigns about where to allocate funds, who to reach out to, and what to say.

What most non-industry consumers of polling data don’t understand is the amount of creative control a pollster has over the outcomes the poll shows.

Many polls are of “likely voters,” and each pollster’s definition of what a “likely voter” is can have a huge role in what the survey shows.  Pollsters also usually “weight” their data after the fact, making it match back up with certain demographic realities.

In most cases, weighting or screening of respondents isn’t done with intent to skew data one way or another.  On the contrary, it is intended to create more useful results.  Yet when pollsters refuse to disclose the way in which they model the data, the public as a whole has no good way to judge whether or not a poll is worth looking at.

It isn’t just about disclosure.  It is entirely possible that a pollster would conduct a poll, use mediocre methods, fully disclose them, and the public at large still wouldn’t notice that something strange was afoot.  Blogs like Pollster.com and FiveThirtyEight are helping to educate political observers about what to look for in good polling. (Disclosure: I am also a contributor to Pollster.com.)

But when a hot new poll yields headline-making results, it can be all too easy for the media to ignore those doubts.  It is clearer than ever that the burden falls on the media to truly vet the pollster and make sure their findings have legitimacy.

There’s a chance that this whole situation will cast doubt on all public polling data, and that a charge of data fabrication will become an unfortunate way to try to discredit data unfavorable to a certain party’s narrative.

But the good guys shouldn’t worry.

While this most recent scandal has certainly rocked the polling industry, I am hopeful that the end result will be a demand for quality over quantity when it comes to survey research.

We may never stop being fascinated by the constant temperature taking that survey research facilitates.  But with any luck, this controversy will encourage a more critical look at the ways some choose to check the public pulse.

Kristen Soltis is the Director of Policy Research at The Winston Group, a Washington, D.C.-based strategic consulting and opinion research firm.  Soltis is a contributor at The Huffington Post, Pollster.com and The Next Right and has also provided political commentary for The American Spectator, BBC Radio and Ireland’s RTE.

PREMIUM ARTICLE: Subscribe To Keep Reading

Sign up

By subscribing you agree to our Terms of Use

You're signed up!

Sign up

By subscribing you agree to our Terms of Use

You're signed up!
Sign up

By subscribing you agree to our Terms of Use

You're signed up!

Sign up

By subscribing you agree to our Terms of Use

You're signed up!
Sign up

By subscribing you agree to our Terms of Use

You're signed up!

Sign Up

By subscribing you agree to our Terms of Use

You're signed up!
Sign up

By subscribing you agree to our Terms of Use

You're signed up!
Sign up

By subscribing you agree to our Terms of Use

You're signed up!
BENEFITS READERS PASS PATRIOTS FOUNDERS
Daily and Breaking Newsletters
Daily Caller Shows
Ad Free Experience
Exclusive Articles
Custom Newsletters
Editor Daily Rundown
Behind The Scenes Coverage
Award Winning Documentaries
Patriot War Room
Patriot Live Chat
Exclusive Events
Gold Membership Card
Tucker Mug

What does Founders Club include?

Tucker Mug and Membership Card
Founders

Readers,

Instead of sucking up to the political and corporate powers that dominate America, The Daily Caller is fighting for you — our readers. We humbly ask you to consider joining us in this fight.

Now that millions of readers are rejecting the increasingly biased and even corrupt corporate media and joining us daily, there are powerful forces lined up to stop us: the old guard of the news media hopes to marginalize us; the big corporate ad agencies want to deprive us of revenue and put us out of business; senators threaten to have our reporters arrested for asking simple questions; the big tech platforms want to limit our ability to communicate with you; and the political party establishments feel threatened by our independence.

We don't complain -- we can't stand complainers -- but we do call it how we see it. We have a fight on our hands, and it's intense. We need your help to smash through the big tech, big media and big government blockade.

We're the insurgent outsiders for a reason: our deep-dive investigations hold the powerful to account. Our original videos undermine their narratives on a daily basis. Even our insistence on having fun infuriates them -- because we won’t bend the knee to political correctness.

One reason we stand apart is because we are not afraid to say we love America. We love her with every fiber of our being, and we think she's worth saving from today’s craziness.

Help us save her.

A second reason we stand out is the sheer number of honest responsible reporters we have helped train. We have trained so many solid reporters that they now hold prominent positions at publications across the political spectrum. Hear a rare reasonable voice at a place like CNN? There’s a good chance they were trained at Daily Caller. Same goes for the numerous Daily Caller alumni dominating the news coverage at outlets such as Fox News, Newsmax, Daily Wire and many others.

Simply put, America needs solid reporters fighting to tell the truth or we will never have honest elections or a fair system. We are working tirelessly to make that happen and we are making a difference.

Since 2010, The Daily Caller has grown immensely. We're in the halls of Congress. We're in the Oval Office. And we're in up to 20 million homes every single month. That's 20 million Americans like you who are impossible to ignore.

We can overcome the forces lined up against all of us. This is an important mission but we can’t do it unless you — the everyday Americans forgotten by the establishment — have our back.

Please consider becoming a Daily Caller Patriot today, and help us keep doing work that holds politicians, corporations and other leaders accountable. Help us thumb our noses at political correctness. Help us train a new generation of news reporters who will actually tell the truth. And help us remind Americans everywhere that there are millions of us who remain clear-eyed about our country's greatness.

In return for membership, Daily Caller Patriots will be able to read The Daily Caller without any of the ads that we have long used to support our mission. We know the ads drive you crazy. They drive us crazy too. But we need revenue to keep the fight going. If you join us, we will cut out the ads for you and put every Lincoln-headed cent we earn into amplifying our voice, training even more solid reporters, and giving you the ad-free experience and lightning fast website you deserve.

Patriots will also be eligible for Patriots Only content, newsletters, chats and live events with our reporters and editors. It's simple: welcome us into your lives, and we'll welcome you into ours.

We can save America together.

Become a Daily Caller Patriot today.

Signature

Neil Patel