It’s been said that we’re now living in the era of Big Data. Amidst the declaration of the revolution taking place, many may believe that we’ve been here before. Is this another overhyped technology (following in the footsteps of Web 2.0 and other innovations) or a real game changer? Is it something new and radically different or simply an evolution of business intelligence (BI) technology?
To be fair, all of these statements are true to some degree. However, perhaps the best way to frame Big Data is to think of it as going “back to the future” – we’ve always had the goal to transform businesses from insight, but we’ve struggled (until now) to make it happen in a meaningful way. Just take a look at what Michael Porter wrote back in 1979 on the “Five Competitive Forces that Shape Strategy” – and you’ll see what I mean. If you’ve watched the movie “Moneyball,” you see how Oakland A’s General Manager Billy Beane used data analysis back in 2002 to predict future outcomes and literally “change the game” of baseball.
What’s fascinating about Big Data is not only the power to take business to the next level, but also the impact to our daily lives and society at large – tapping into virtually limitless sources of information to help with anything from finding new sources of energy, to improving the health of individuals and entire populations.
What’s Different?
While the basic principles of BI and Big Data share a common foundation/premise (data meets analytics to make better decisions), the nature of Big Data is indeed different. Big Data is distinguished by three key elements:
-
Volume/Scale – extremely large volumes of data – from gigabytes to petabytes. According to a 2011 IDC report, more than 90% of Big Data is unstructured data. It includes data sources such as http streams, sensor grid and machine generated information, but also about content and information captured from documents, conversations, emails, workflows, images, videos, business processes and transactions, etc. It’s also becoming more about the information about the data (i.e. metadata, including location from mobile devices, social interactions, usage, status updates, etc.) that gives greater context to information.
-
Variety – both structured and unstructured information from disparate sources from both inside and outside of the organization.
-
Velocity – need to analyze information in real-time to generate actionable insights for the organization.
Most importantly, Big Data is differentiated in its “Significance,” given its ability to transform businesses through strategic insight and corresponding actions (in real-time). Traditional BI architectures and tools have not been able to deliver on the big data opportunity, given their inherent limitations on the number of sources of information they can consume and needing to do it all in real-time. Where BI has traditionally focused primarily on the “forensics” of the past, Big Data is all about predicting future events and acting directly on those insights. This has resulted in a new set of requirements requiring an entirely new Infrastructure that can accommodate data at a massive scale, purpose-built real-time analytics and business process platform that turns insight into action.
What’s the Value?
In a nutshell…enable and accelerate business transformation. Businesses exploiting Big Data to improve their strategy, execution and ongoing improvement will quickly and dramatically distance themselves from competitors. This tremendous opportunity for business transformation becomes even more apparent when you look at the intersection with ECM, given the wealth of information that exists within organizations (applications, databases, file shares, content repositories, emails, collaboration and social tools, videos, etc.). Combining this structured and unstructured information with sources outside the enterprise gives organizations an unprecedented depth of knowledge and insight – enabling better decisions and the ability to predict future outcomes. Incorporating business process management into this equation provides a dimension of value that has previously eluded organizations. Turning these insights into action essentially allows organizations to transform themselves in real-time. Think about it…the ability to change business processes/workflows, decision frameworks, and/or develop new business models and growth vectors for the organization - in a fraction of the time of what it took before. These new patterns of work and decisioning can be captured and engrained into the very fabric of the organization and monitored to drive continuous improvement. This allows organizations to deliver unparalleled breakthroughs in agility, efficiency, security (risk reduction) and growth compared to their non-Big Data enabled competitors.
Now Insurance companies can provide real-time insurance pricing optimized for specific individuals based on their behaviors, not general risk pools. Manufacturing supply chains can anticipate market movements and weather patterns - making adjustments automatically. CPG companies can quickly identify customer issues before they become problems – creating loyalty out of adversity. Financial Service companies and Government agencies can spot fraud and assess risk with greater precision and speed - saving millions, if not billions of dollars.
So What’s Next?
There is already a ton of great material out there on the basics of Big Data and its associated technology. Make sure to check out:
· Blog by EMC’s Chuck Hollis - http://chucksblog.emc.com/
· Big Data: Big Opportunities to Create Business Value - http://www.emc.com/microsites/cio/articles/big-data-big-opportunities/
· McKinsey Global Institute Report – Big Data: The next frontier for innovation, competition, and productivity. http://www.mckinsey.com/Insights/MGI/Research/Technology_and_Innovation/Big_data_The_next_frontier_for_innovation
· 2011 IDC Digital Universe Study - http://www.emc.com/collateral/analyst-reports/idc-extracting-value-from-chaos-ar.pdf
Technology has finally caught up with the vision of turning insight into strategic advantage – delivering on the promise outlined by Michael Porter and field-tested by Billy Beane years ago. The real question is whether Big Data can provide new, real and lasting value versus being just another technology “hammer” looking for a proverbial nail? So rather than spending time covering the “well-worn” technology path, I thought it would be more valuable to explore various use-cases across the major industries (where very little has been written). We’ll explore this “art of the possible” in future blogs and hopefully spark some great dialogue and ideas along the way. In the meantime, I would love to hear your thoughts, the use-cases you find compelling and the nature of change that you’re witnessing around Big Data.
Post originally appeared in: www.seekingpracticaltruth.com