Is big data over hyped?

We’ve definitely hit the point in the “big data” hype cycle where people are looking critically at the term and asking what all the fuss is about. Some pundits have even speculated about a big data bubble that’s sure to burst after everyone realizes the term has been over-used, and technologies like in-memory processing and high-performance computing (HPC) are not actually in high demand.

Folks, there’s no bubble here. You can’t put the data back in the bottle, so to speak. Big data – however you define it – isn’t going away and it isn’t getting smaller. It’s going to keep growing.

Maybe the term “big data” will change or devalue in significance, but you can’t poke holes in the concept itself, which is this: there are significant business benefits to be gained from storing and analyzing large volumes of data more efficiently.

That doesn’t mean big data is an issue for everyone or that high-performance computing is the answer for everyone.   But there is a business case to be made for the use of high performance analytics in many arenas, and those that see the whole idea as overhyped are looking at it too narrowly.

Here’s how I like to look at it: High-performance computing is, simply, an enabler. Most importantly, it enables you to get answers faster than before. But – and this is important – high performance computing is only as good as what you’re computing. If you’re getting summary statistics about your business portfolio, HPC will give you those reports faster. However, if your system is predicting risk exposure on thousands of assets, you’re going to get those predictions faster than before. Or, if you’re optimizing markdowns for millions of SKUs at hundreds of retail locations, you’ll be able to optimize those prices more quickly.

You see the difference?

A lot of big data proponents are promising things bigger, better and faster. But if the information you’re getting is backward looking, it’s still going to be looking at the past when you get it in a shorter timeframe. You’re still only understanding the past faster than before. No matter how fast you go with summary statistics, you’re never going to get to the future.

Only predictive analytics like forecasting and optimization will bring you out of the past and into the future. When you use high performance analytics to predict things like risk, customer satisfaction or marketing optimization, you’re getting your predictions sooner than before, and you can react more quickly. When you’re computing forward-looking results, the speed really can make a difference.

At its most basic level, high-performance computing reduces the time dimension. You have to decide what types of answers you want more quickly: standard reports or predictive analytics.

Once you know that, you can start asking questions and making decisions that could change your business. Or your world. I don’t think that’s hype. It’s as real as you can get, and the organizations that get it first are going to be the ones that make it further into the future.