Blogs

I just gave a talk on defensible disposition at AIIM 2014, and here are the main points (and my slide deck). Let me know if you have questions — happy to address them.

I spend almost no time on the problem of over-retention – 90% of what I say here is about the solution.  But the problem of course is over-retention — organizations have been over-retaining electronic information and failing to dispose of it in a legally defensible manner when business and law will allow. The best way to address this monster problem is to break it into more tractable sub-problems: day-forward information disposition and historical informational disposition. I won’t go into day-forward information disposition here, but it is an easier problem to solve. Let’s therefore stipulate that it’s taken care of and focus on historical information disposition.

Defensible disposition primarily consists of developing and then executing four pieces:

1. The Defensible Disposition Policy
2. The Technology Plan
3. The Assessment Plan
4. The Disposition Plan

1.  Develop your Defensible Disposition Policy

The first step is to develop your Defensible Disposition Policy. This is the design specification that states very clearly the objectives that your methodology will fulfill. You should be able to defend your actions by pointing at your policy for defensible disposition, which shows what you intend to do, and then showing that you are following it.

Here’s how to develop the Policy. First, recognize that you must satisfy 4 demands, 3 of which concern retention:

1. You have Regulatory retention requirements
2. You have Hold retention requirements
3. You have Business retention requirements
4. And then you must address the negative or positive Cost impact of anything you do.

Second, note that everything you do has an impact, whether you do nothing at all or take action aggressively. The impacts result from 1) what you do, and 2) the effects of what you do. In the defensible disposition scenario there are two relevant types of activities. You can sort your files (or assess them – same thing). And you can dispose of your files. Disposition here means the two extremes in the range from purge immediately to keep forever, with many points between.

You can now state your mission in two equivalent ways:

A.      Your mission is to assess and dispose of your organization’s information in order to satisfy your retention demands (1-3) while also minimizing bad cost impact (4)
B.      Your mission is to assess and dispose of your organization’s information in order to maximize good cost impact (4) while also satisfying your retention requirements (1-3)

The two mission statements mean the exact same thing depending on how you specify what the terms mean, but lawyers and records managers like version A more than B because it focuses attention on satisfying retention demands. IT folks like version B more than A because it focuses attention on reducing costs and increasing savings.

Note that your mission statement still needs to be a lot more specific. You need to determine the extent of your regulatory, hold, and business retention requirements. (To pick just one example: is your company clear on what’s a record versus a high value non-record versus a low value non-record?) You also need to determine what “satisfy your retention demands” really means for you. And you need to clarify what your organization means by bad or good cost impacts. You don’t have to write down specific dollar amounts but you have to be pretty clear about e.g. when the costs of storage or a deeper level of document classification are burdensome.

The good news is that you don’t need to be perfect – you don’t have to perfectly satisfy your retention demands. You do need to use the Principle of Reasonableness and act In Good Faith.

So clarify the vague parts of the mission statement and you have your Defensible Disposition Policy. To review, the Policy is the design specification that lays out very clearly the objectives that your methodology will fulfill. The next four pieces of your methodology succeed or fail solely by how well they together fulfill the Policy.

2.  Develop your Technology Plan

Using technology for the heavy lifting in the file assessing and disposing processes is absolutely necessary. But there are two sources of complexity in finding and using the right tools that make it a challenge:

First, the “analysis, classification, and disposition” market is young and a mess. The relevant vendors and products come from file analytics, content analytics, content classification, ECM, E-discovery, search, document capture (!), data loss prevention, and storage management. The delivery channels include products and modules you install at your site, hosted solutions, and service providers who may use a variety of products. The variety of vendors includes IBM/StoredIQ, HP/Autonomy, EMC Kazeon, SAS, Kofax, Equivio, Rational Enterprise, Recommind, Index Engines, and many others. I just did an analysis of classification and archiving solutions for a client that covered 41 tools in 5 different technology categories. Most of these vendors have a sweet spot or spots where they can succeed but it’s not easy to locate that spot and successfully match it up with your needs. And that leads to the second complexity.

Second, to efficiently assess a significant pile of files you typically have to use a variety of different techniques and thus tools, since the tools have different sweet spots. There is no universal assessment technique and thus no universal assessment tool.

This means that you need a Technology Plan that fits the right tools to your Assessment Plan and Disposition Plan, both of which are discussed below.

3.  Develop your Assessment Plan

The Assessment Plan specifies which information and systems you’re investigating and the particular processing rules you’re going to use. The first step in developing the Assessment Plan is to do the legwork and get a good picture of where all the information is, what repositories it’s in, and anything else you can learn about it that will help you create the rules (described below) and a plan of attack. This may take several days or weeks.

You then create processing rules based on the different types of file attributes. There are three categories of attributes that can be used to determine what a file is:

1. Environmental attributes around the file (e.g. file location, ownership)
2. File attributes about the file (e.g. file type, age, author)
3. Content attributes within the file (e.g. keywords, character strings, word proximity, word density)

Note that I’m not using terms like “metadata” and the other terms vendors use when they talk about analytics and classification. We find them confusing. Instead we focus on Environmental attributes around the file, File level attributes about the file, and Content attributes within the file.

You should then combine these attributes and create sets of rules that machines can use to sort files, and to flag as exceptions that need human attention. Start with the simplest rules in which you have the most confidence. And then do multiple passes through the pile, each time using more complex rules on a pile that’s getting successively smaller and smaller. The assessment results after multiple passes will show you how much of the pile is unnecessary “junk” that can be purged, how much is records, how much is high value non-records, and how much just couldn’t be identified. This last piece may be large (we often see 40-50%).

4.  Develop your Disposition Plan

The Disposition Plan evaluates your assessment results against your Defensible Disposition Policy and lays out a roadmap for disposing of the various kinds of files you found.

It includes an analysis of the financial implications of your various disposition options in the overall evaluation for the disposition roadmap. (For example, it extrapolates what you’d save if you purged this particular bucket and starting a 3-year retention clock on that bucket, versus first cleaning up duplicate files wherever they lie, versus focusing all your efforts on an aggressive day-forward effort that would leave most of the pile in place but would significantly stem the flow of new files onto the pile.)

Then you start executing the Disposition Plan. This may require one or more FTEs managing the purges and cleanup over months or even years.

And finally, as you go through your first disposition cycle, you’ll probably want to refine your Defensible Disposition Policy – as you’ve now got a much more realistic picture regarding the real cost impacts of every action you take, and more generally what actions are reasonable.