Opinion

Big Data’s Big Moment

Robert Cresanti Contributor
Font Size:

It’s no surprise that we’re collectively taken aback by the accuracy of Nate Silver’s predictions, along with the predictions of the few other researchers who follow similar models. After all, so many months of punditry, analyses, and polls were instantly overshadowed by the application of data functions to the Electoral College.

Quite simply, it’s big data’s big moment.

But rather than breathlessly celebrating this achievement, we should take a step back and realize that Nate Silver’s prediction isn’t an anomaly. Rather, it’s a straightforward example of what big data can, and does, achieve on a daily basis. At SAP, we understand this – we provide big data tools to clients throughout the public sector, and during the campaign, we conducted sentiment analyses of the big data resulting from social media platforms to help provide another level of insight into the public’s thinking.

In truth, the name “big data” is a bit of a mischaracterization. The concept isn’t just about the amount of data itself – it’s about how we aggregate, analyze, and apply that data to the real world – just as Nate Silver did.

The ability to harness the wealth of information in the world – and to apply it in a way that gives us better predictability and heightened efficiencies – offers untold possibilities. And in the U.S., where we are confronting rapid population growth and limited resources, the importance of this cannot be denied.

We must learn to better take advantage of big data – not just because the insights could offer us new benefits, but because we also must avoid information overload by managing the wealth of data created each year.

According to a recent report from TechAmerica (which was co-authored by a colleague within SAP), in 2011, 1.8 zezabytes of information were created globally, and that amount is expected to double every year. This is the equivalent of 200 billion, two-hour high-definition movies, which one person could watch for 47 million years straight. This impacts both companies and governments, and in the absence of appropriate management tools, it threatens to overwhelm them.

But it also presents opportunities. Mr. Silver doesn’t make public his processes, but one can assume that, like most big data calculations, it involves incorporating many different data sets – for his purposes, for example, he would look not just at the results of polls, but also of the historical voter turnout in each state. He must evaluate the whole picture in order to achieve an accurate outcome.

That’s exactly how big data can, and does, affect our lives now. And with greater adoption of real-time (rather than retroactive) big data tools, the effects in the future will be even more dramatic.

For example, local police departments can use real-time big data tools to develop predictive models about when and where crimes are likely to occur by identifying patterns that might otherwise have gone unnoticed. That creates a system where police are not just solving crimes – they are preventing them.

Or, using real-time big data tools, regional power providers can measure ongoing power consumption from individual houses, identifying periods of peak demand in order to moderate energy use or warning when the grid might be subject to overload, averting a blackout.

And big data can improve medical researchers’ ability to aggregate the results of clinical trials, leading to more-thorough medical understanding and, potentially, more-targeted treatments. And once those new treatments, whether they be pharmaceuticals or medical devices, come to the market, both manufacturers and physicians will be better able to monitor patients in case of any emerging safety concerns.

Big data’s analytical and predictability capabilities were made clear through Mr. Silver’s election predictions, but its potential impact on businesses, the government, and the public can impact us daily – not just every four years.

Tags : big data
Robert Cresanti

PREMIUM ARTICLE: Subscribe To Keep Reading

Sign up

By subscribing you agree to our Terms of Use

You're signed up!

Sign up

By subscribing you agree to our Terms of Use

You're signed up!
Sign up

By subscribing you agree to our Terms of Use

You're signed up!

Sign up

By subscribing you agree to our Terms of Use

You're signed up!
Sign up

By subscribing you agree to our Terms of Use

You're signed up!

Sign Up

By subscribing you agree to our Terms of Use

You're signed up!
Sign up

By subscribing you agree to our Terms of Use

You're signed up!
Sign up

By subscribing you agree to our Terms of Use

You're signed up!
BENEFITS READERS PASS PATRIOTS FOUNDERS
Daily and Breaking Newsletters
Daily Caller Shows
Ad Free Experience
Exclusive Articles
Custom Newsletters
Editor Daily Rundown
Behind The Scenes Coverage
Award Winning Documentaries
Patriot War Room
Patriot Live Chat
Exclusive Events
Gold Membership Card
Tucker Mug

What does Founders Club include?

Tucker Mug and Membership Card
Founders

Readers,

Instead of sucking up to the political and corporate powers that dominate America, The Daily Caller is fighting for you — our readers. We humbly ask you to consider joining us in this fight.

Now that millions of readers are rejecting the increasingly biased and even corrupt corporate media and joining us daily, there are powerful forces lined up to stop us: the old guard of the news media hopes to marginalize us; the big corporate ad agencies want to deprive us of revenue and put us out of business; senators threaten to have our reporters arrested for asking simple questions; the big tech platforms want to limit our ability to communicate with you; and the political party establishments feel threatened by our independence.

We don't complain -- we can't stand complainers -- but we do call it how we see it. We have a fight on our hands, and it's intense. We need your help to smash through the big tech, big media and big government blockade.

We're the insurgent outsiders for a reason: our deep-dive investigations hold the powerful to account. Our original videos undermine their narratives on a daily basis. Even our insistence on having fun infuriates them -- because we won’t bend the knee to political correctness.

One reason we stand apart is because we are not afraid to say we love America. We love her with every fiber of our being, and we think she's worth saving from today’s craziness.

Help us save her.

A second reason we stand out is the sheer number of honest responsible reporters we have helped train. We have trained so many solid reporters that they now hold prominent positions at publications across the political spectrum. Hear a rare reasonable voice at a place like CNN? There’s a good chance they were trained at Daily Caller. Same goes for the numerous Daily Caller alumni dominating the news coverage at outlets such as Fox News, Newsmax, Daily Wire and many others.

Simply put, America needs solid reporters fighting to tell the truth or we will never have honest elections or a fair system. We are working tirelessly to make that happen and we are making a difference.

Since 2010, The Daily Caller has grown immensely. We're in the halls of Congress. We're in the Oval Office. And we're in up to 20 million homes every single month. That's 20 million Americans like you who are impossible to ignore.

We can overcome the forces lined up against all of us. This is an important mission but we can’t do it unless you — the everyday Americans forgotten by the establishment — have our back.

Please consider becoming a Daily Caller Patriot today, and help us keep doing work that holds politicians, corporations and other leaders accountable. Help us thumb our noses at political correctness. Help us train a new generation of news reporters who will actually tell the truth. And help us remind Americans everywhere that there are millions of us who remain clear-eyed about our country's greatness.

In return for membership, Daily Caller Patriots will be able to read The Daily Caller without any of the ads that we have long used to support our mission. We know the ads drive you crazy. They drive us crazy too. But we need revenue to keep the fight going. If you join us, we will cut out the ads for you and put every Lincoln-headed cent we earn into amplifying our voice, training even more solid reporters, and giving you the ad-free experience and lightning fast website you deserve.

Patriots will also be eligible for Patriots Only content, newsletters, chats and live events with our reporters and editors. It's simple: welcome us into your lives, and we'll welcome you into ours.

We can save America together.

Become a Daily Caller Patriot today.

Signature

Neil Patel