Big Data Analytics and Records Retention

By Carl Weise posted 03-28-2012 16:05


At AIIM, we recognize the current business trend towards Big Data, addressing the benefits and issues.   A major theme with all of this content and records is deriving value from it through analytics.  For example, it has been said that 2011 was the year of social media and that 2012 in the year of social media analytics.

Data analytics, or Business Intelligence, has been around for a long period, although without the emphasis it is receiving now.  This activity often analyzed past data and transactions to assist in providing direction in the future for the organization.  It has become clear that this is just as bad as picking stocks or mutual funds based on past performance.  As we are constantly reminded, past performance is not an indicator of future performance.

There are new software products that enable organizations to carry out predictive analytics that are considered much better.  This makes me think of a recent story in which a father was very upset with Target Stores and asked if the store was trying to encourage his teenage daughter to become pregnant.  The store was sending her ads and coupons related to pregnancy.  In fact, the store was able to analyze her previous purchase activity and identify that she was, in fact, pregnant before the father learned of her condition.

The point of this blog is to discuss the impact of business intelligence, or data analytics, on our records retention decisions.  Is the need for business intelligence a relatively new administrative value of data and records?  As we try to meet the business drivers for electronic records management – compliance, effective, efficiency and business continuity, is the answer from staff that they need to retain information for business intelligence impeding our retention goals?

Do we have to address this administrative value from an analytic perspective, itself – how long is this data and records valuable from a business intelligence perspective?

I remember many years ago, I was scheduling financial support data and records for the organization I worked at.  The staff wanted to retain that information for ten years.  However, I argued that using data over that long length of time would distort what actually was taking place over the most recent history.   

As I have just noted, this basis for retaining records for analysis and business intelligence has always been with us, but I suggest to a very small degree.

What is your experience?  Are you seeing this factor being presented more these days in your organization as you try to address the retention of your organization’s records?

Instead of just accepting the response that the records have to be kept for very long periods of time for the benefits of business intelligence, what efforts do you take or, perhaps, should take, to make sure that the length of retention is both reasonable and appropriate?


#ElectronicRecordsManagement #ECM #ERM