Turning raw data into actionable insights has been the goal of many a business since the 1990s. Back then, ‘business intelligence’ was the buzzword, and since the tweenies and teenies it’s been ‘big data’. Now the two are combining (‘Big Intelligence’, maybe?) as the promise of big data is finally capable of delivering the business intelligence that companies have long dreamed about.
However, the cost and complexity of deploying such an endeavour is still somewhat of a challenge. Big data is a complicated thing, consisting, as it does, of reams and reams of unstructured data, covering everything from Twitter tweets to telephone conversation logs.
What is more, there is currently more big data than there are skilled professionals and resources to cope with analysing it all. Big data is, by its very nature, getting bigger all the time. Every time you do something on the internet – and this really means anything at all, from clicking on an advert, to opening an email, to reading this blog – you are adding another piece to the gigantic puzzle that is big data.
IBM calculated that 2.5 billion gigabytes (or 2.5 exabytes) was generated every single day in 2012. Goodness knows what that figure is now. And the problem the world currently faces is a genuine shortage of data scientists and other big data professionals who have the skills to make any sense of this data. In the US alone, it is reported that there is a shortage of between 140,000 and 190,000 professionals.
What’s To Be Done?
Well, just as with big data itself, there are solutions and innovations popping up all of the time (though not quite at the rate, of course, that big data is being generated, but no matter). A lot of the business intelligence big data solutions, however, do still want to remain fixed on the reams of structured data buried in databases around the world. Now, while all of this information – i.e. sales transactions figures, customer records etc. – does indeed make up some of what we call big data, as for current up to date business intelligence, the information gleaned makes a poor fit for what’s happening now.
However, there are of course some great things that are being created to give unprecedented business intelligence to companies using software that has been designed to pick apart big data.
ZoomData is one of the newest kids on the block in the big data BI market. They’ve got some great venture backers – including Accel Partners and NEA – as well that is helping them to drive their software forward.
Of course, there are already plenty of instances where big data BI is being scrutinised by various companies and programs around the world – just look at what’s been accomplished by Hadoop for example. But what is setting Zoomdata apart is that it is actually built for the unstructured world of big data, rather than for stripping away the rich data sets in things like MongoDB or Cassandra.
Whereas these models move the data into a black box or traditional RDBMS, Zoomdata moves the question to the data, and this creates a number of efficiencies – most notably in bandwidth and time. What Zoomdata has remarkably managed to figure out a way of doing is to stream process the results of a query straight back from an original data source, and then display results in a sketch view, which proceeds to become sharper as more data is processed.
Lots of Competition
Ok, so I’ve focussed on Zoomdata here, but there are lots of vendors in this space from which businesses can choose. Indeed, there are some very interesting independent providers to watch out for – Lucky Sort, MetaLayer and Tableau, to name but a few.
Although all of these companies are competing in the same business space – big data BI – they all offer very different solutions, and their focus is often different as well.
So far, businesses haven’t had the technology to allow them to become actively engaged with their data, and, as a result, an emerging trend of ‘information activism’ is appearing. As the technology and software starts to become available for companies to maintain a ‘self-service’ record and analysis of their big data BI, they will inevitably move away from being simply passive consumers of the data to become active users and analysts of it.
Real-time interaction with the data will also become important. More and more, companies will expect to be able to glean all the important information from real-time data at a glance and make decisions and take action accordingly with immediate effect. Creating easy-to-interpret visualisations from big data will be key to realising this, as this is what people are used to. The need for speed has always been important to business, and this is not going to change at any point soon. Indeed, the speed of business is always accelerating, and so is the growth of big data. The time has most certainly come for software innovations that can keep up with it all.
What do you think about the growing trends in big data and business intelligence? Is your company involving itself in big data analytics? Let us know in the comments below.
Editorial image: Linking Open Data cloud diagram 2014, by Max Schmachtenberg, Christian Bizer, Anja Jentzsch and Richard Cyganiak. http://lod-cloud.net/
Latest posts by John Waldron (see all)
- 10 Ideas for Infographic Content - January 1, 1970
- Is Virtual Reality the New Business Standard? - January 1, 1970
- How to Grow Your Website Traffic with Infographics - January 1, 1970
- 10 Ways Infographics Will Immediately Benefit Your Business - January 1, 1970
- markITwrite Guest Post Guidelines - January 1, 1970