Lots of enterprise representatives are making claims about their movement into big data, but when you look behind the curtain, you can see they're moving at the rate of molasses. Here are some suggestions about why this may be the case, and how your organization can take steps to speed up the transition process:
One reason may be that your organization has little idea of what it might do with so much information and therefore lacks direction and motivation to begin implementing appropriate projects or installing necessary technology.
Certainly the Big Dogs—Amazon, Apple, Facebook, Google, HP, Microsoft, Netflix, and Twitter—are working hard to get a handle on Big Data and apply the results to their business models. But smaller or less tech-focused companies—and that includes relatively small firms manipulating hundreds of gigabytes of data all the way up to giant firms with hundreds of terabytes or even a petabyte or two of data to mine—often remain uncertain about things like:
- What to look for in their data
- How to find it
- What to do with the new insights once they extract them
The answers to these questions, of course, depend on what business you're in and what data you're collecting. But the answers exist, and they're worth digging up. The results of large-volume data analysis for some companies have been remarkable, allowing them to detect and prevent financial fraud, ascertain their risk in volatile business situations, and focus on higher value marketing campaigns.
Sifting through petabytes of data requires specialists, of course, who can find ways to capture, curate, store, search, analyze, and report on the information gems buried in there. But the desire and conviction necessary to push through these difficulties and obtain the potential results needs to come from all parts of the organization.
A second reason for proceeding slowly toward Big Data may be an unwillingness to exploit the newest technologies. Despite efforts to make them work, well-established business intelligence tools and relational databases are insufficient for cost-effective analysis of Big Data. If you're unwilling to look at Hadoop and NoSQL databases, for example, the Big Data train is likely to leave the station without you on board.
A third reason may be the avalanche of ideas, suggestions, advice, and information surrounding the advent of Big Data technology and practice, much of it coming from vendors and consultants with a clear and self-serving agenda, or at any rate a strong bias in one direction or another. Amid such a whirlwind, newcomers to the field are justifiably intimidated, confused, and just plain fearful of following the wrong "guru" and taking a foolish or unproductive step.
This is also understandable, but insufficient cause to delay any organization's reasonable efforts to gain at least the low-hanging fruit that can be gleaned from even the simplest analysis of Big Data.
Fortunately, there are relatively impartial advisers in the field who can bring order to the chaos and suggest some lines of exploration and experimentation to any organization interested in learning more about the potential benefits of working with Big Data.