Opinion

Polls, lies, damned lies and statistics

Lanny Davis Former Special Counsel to President Bill Clinton
Font Size:

Throughout the world of political reporters and pundits, the blogosphere and social media — left, right and center — the conventional wisdom was impressive: Obama’s negative ads on Bain Capital and outsourcing were working in the 12 battleground states where they were exclusively running.

First, two weeks ago, came the Wall Street Journal/NBC poll, which triggered an avalanche of reporting reinforcing the themes that the Obama negative ads were hurting Romney and helping Obama big time. The national results were a statistical dead heat (47 percent Obama, 44 percent Romney). But in the 12 battlegrounds — Colorado, Florida, Iowa, Michigan, Nevada, New Hampshire, New Mexico, North Carolina, Ohio, Pennsylvania, Virginia and Wisconsin — where Obama has spent tens of millions on negative ads since April, Obama was ahead 50 percent to 42.

The Journal/NBC poll also showed that in the battleground states, Romney suffered a net increase of 11 percent in negative attitudes toward his business experience — from late May (36 percent favorable to 36 percent unfavorable) to late June (30 percent favorable, minus 6 percent, to 41 percent unfavorable, plus 5 percent). The cause reported by the Journal and NBC: the Obama negative ads.

But not so fast.

Voters were asked in the national survey whether they would feel more or less positive toward a businessman who managed a company that specialized in restructuring and selling companies. To anyone watching Obama’s TV ads — which were seen by most of the country on either TV or the Internet — those words could only describe Bain Capital. The answer between January 2012, before the negative ads ran, and June, after they ran, was no change at all (from minus 6 percent in January to minus 5 percent in June).

In addition, I could find no disclosure in the reporting of the Journal/NBC poll of the subsample total number of voters surveyed in the 12 battleground states. The total surveyed nationally was just 819 registered voters — with a fairly large plus-minus margin of error of 3.4 percent. It is therefore possible that the number of voters surveyed in the subsample among 12 battlegrounds was a fraction of 819 — meaning a considerably higher margin of error, which could make the 50-42 percent “lead” for Obama in the battlegrounds statistically meaningless.

USA Today’s Susan Page reported that among the 1,200 voters surveyed in the 12 battlegrounds, 8 percent of that total, or 96 voters surveyed, said they “changed” their minds because of the ads, from one candidate to the other. Of those who changed, 76 percent (of the 8 percent, or 73 voters across 12 states) said they switched to Obama, and 16 percent (23 voters across 12 states) said they switched to Romney. Page described this data as evidence that Obama’s negative ads were the “clear” winner.

That’s not so clear to me.

First, the subsample is very small — 8 percent of the total sample of 1,500, or 96 people spread out over 12 states — even though the 5:1 change ratio favoring Obama is statistically significant.

Second, to conclude the ads “caused” the change, it is required to have a control group, i.e., the same number of voters surveyed who have not seen the ads. For example, if the results among that control group show the same or similar percentage of changes favoring Obama, that would negate the inference that the ads “caused” the change. Without a control group, you simply cannot be sure of the cause.

Third, in April, a USA Today/Gallup poll in battleground states showed Obama ahead 51 percent to 42. But in this late-June sample, the results were Obama, 47 percent, to Romney at 45 percent. In other words, after Obama spent an estimated $68 million or more over Romney in the battlegrounds on negative ads during this time period, Obama lost a net 7 percent.

As Republican pundit Rich Mullens noted in his must-read daily blog, Mullings:

“According to the RealClearPolitics average of polling in 12 swing states, Obama has leads outside the margin of error in only Pennsylvania, New Hampshire and Nevada, Romney leads in two (North Carolina and Missouri) and the rest are essentially tied, such as Obama’s 0.8 percent lead in Florida.”

Doesn’t USA Today/Gallup’s own comparison between April and June pool results in the battlegrounds as well as this RealClearPolitics data demonstrate that, if anything, Obama’s negative ads have actually hurt … Obama?

The moral of the story: When you see all the media jumping in one direction in interpreting polls, beware.

When they don’t tell you the total sample, or subsample in the battlegrounds, or the margin of error of the subsample, or whether there were control groups to determine whether the negative ads “caused” the change of minds, beware.

In other words, when you read poll results interpreted by a political media prone to rush to judgment all at once in the same direction, remember what Mark Twain frequently said, quoting former British Prime Minister Benjamin Disraeli’s famous expression:

“There are lies, damned lies and statistics!”

Lanny Davis, the principal in the Washington law firm of Lanny J. Davis & Associates, which also specializes in legal crisis management, served as President Clinton’s special counsel from 1996-98 and as a member of President George W. Bush’s Privacy and Civil Liberties Oversight Board (2006-07). He currently serves as Special Counsel to Dilworth Paxson. He is the author of the book “Scandal: How ‘Gotcha’ Politics Is Destroying America” and the forthcoming book, “Crisis Tales — Five Rules for Handling Scandal in Business, Politics and Life,” to be published by Simon & Schuster.

PREMIUM ARTICLE: Subscribe To Keep Reading

Sign up

By subscribing you agree to our Terms of Use

You're signed up!

Sign up

By subscribing you agree to our Terms of Use

You're signed up!
Sign up

By subscribing you agree to our Terms of Use

You're signed up!

Sign up

By subscribing you agree to our Terms of Use

You're signed up!
Sign up

By subscribing you agree to our Terms of Use

You're signed up!

Sign Up

By subscribing you agree to our Terms of Use

You're signed up!
Sign up

By subscribing you agree to our Terms of Use

You're signed up!
Sign up

By subscribing you agree to our Terms of Use

You're signed up!
BENEFITS READERS PASS PATRIOTS FOUNDERS
Daily and Breaking Newsletters
Daily Caller Shows
Ad Free Experience
Exclusive Articles
Custom Newsletters
Editor Daily Rundown
Behind The Scenes Coverage
Award Winning Documentaries
Patriot War Room
Patriot Live Chat
Exclusive Events
Gold Membership Card
Tucker Mug

What does Founders Club include?

Tucker Mug and Membership Card
Founders

Readers,

Instead of sucking up to the political and corporate powers that dominate America, The Daily Caller is fighting for you — our readers. We humbly ask you to consider joining us in this fight.

Now that millions of readers are rejecting the increasingly biased and even corrupt corporate media and joining us daily, there are powerful forces lined up to stop us: the old guard of the news media hopes to marginalize us; the big corporate ad agencies want to deprive us of revenue and put us out of business; senators threaten to have our reporters arrested for asking simple questions; the big tech platforms want to limit our ability to communicate with you; and the political party establishments feel threatened by our independence.

We don't complain -- we can't stand complainers -- but we do call it how we see it. We have a fight on our hands, and it's intense. We need your help to smash through the big tech, big media and big government blockade.

We're the insurgent outsiders for a reason: our deep-dive investigations hold the powerful to account. Our original videos undermine their narratives on a daily basis. Even our insistence on having fun infuriates them -- because we won’t bend the knee to political correctness.

One reason we stand apart is because we are not afraid to say we love America. We love her with every fiber of our being, and we think she's worth saving from today’s craziness.

Help us save her.

A second reason we stand out is the sheer number of honest responsible reporters we have helped train. We have trained so many solid reporters that they now hold prominent positions at publications across the political spectrum. Hear a rare reasonable voice at a place like CNN? There’s a good chance they were trained at Daily Caller. Same goes for the numerous Daily Caller alumni dominating the news coverage at outlets such as Fox News, Newsmax, Daily Wire and many others.

Simply put, America needs solid reporters fighting to tell the truth or we will never have honest elections or a fair system. We are working tirelessly to make that happen and we are making a difference.

Since 2010, The Daily Caller has grown immensely. We're in the halls of Congress. We're in the Oval Office. And we're in up to 20 million homes every single month. That's 20 million Americans like you who are impossible to ignore.

We can overcome the forces lined up against all of us. This is an important mission but we can’t do it unless you — the everyday Americans forgotten by the establishment — have our back.

Please consider becoming a Daily Caller Patriot today, and help us keep doing work that holds politicians, corporations and other leaders accountable. Help us thumb our noses at political correctness. Help us train a new generation of news reporters who will actually tell the truth. And help us remind Americans everywhere that there are millions of us who remain clear-eyed about our country's greatness.

In return for membership, Daily Caller Patriots will be able to read The Daily Caller without any of the ads that we have long used to support our mission. We know the ads drive you crazy. They drive us crazy too. But we need revenue to keep the fight going. If you join us, we will cut out the ads for you and put every Lincoln-headed cent we earn into amplifying our voice, training even more solid reporters, and giving you the ad-free experience and lightning fast website you deserve.

Patriots will also be eligible for Patriots Only content, newsletters, chats and live events with our reporters and editors. It's simple: welcome us into your lives, and we'll welcome you into ours.

We can save America together.

Become a Daily Caller Patriot today.

Signature

Neil Patel