The State of the Art in Records Management – Part One

By Mark Mandel posted 07-09-2012 15:18


Over the past couple of years I have come to the realization that with the many changes taking place in our industry there is a new “State of the Art” in Records Management.  This change has come about because of new market drivers combined with new vendor solutions in response to these requirements.

Part One of this blog focuses on the new market drivers affecting the industry.  Part Two will focus on how the industry has responded to these new drivers.


The Federal Rules of Civil Procedure dictate that ALL available documents are discoverable in a court case.  This includes electronic content, regardless of media.  Email is the largest type of electronic content that is requested in eDiscovery.  Instant messages, voice mail, copies, drafts, personal correspondence, and so on are all discoverable.  Therefore, it does not matter if your organization has declared “official” records, all content whether declared or not is discoverable.

Discovery costs are very high.  The ramifications of missing documents in a litigation can be very high as well, leading to fines, sanctions, and negative judgments; in some cases leading to jail time for defendants.  Therefore, it is very important that Information Governance is applied to all content so that it can be found easily, so that it can be automatically classified to the correct retention rule, and so that it can be discarded if it is not relevant. This approach lowers discovery and storage costs and eliminates or significantly reduces the missing documents issue.

This approach, called Content Lifecycle Management, applies Information Governance to all content from the time it is created, captured, or ingested, all the way through final disposition.  Governance is applied to email, social media, electronic documents and all versions, file shares, SharePoint, and content created or captured by business applications such as SAP, case management applications, collaboration tools, and more.

This market driver has changed the foundation of records management, leading to a redefinition of a Record.  In the past, a record was “declared” once its business process was completed, and the Records Retention Schedule defined how this static record was to be filed, stored, and retained.  File Plans and Retention Schedules were based on a paper paradigm, focusing on how to organize boxes so that they could be accessioned, archived, and dispositioned effectively.

The new definition is “everything is a record.”  Therefore, all content must be managed.  This approach requires a new Big Bucket Record Series called “Transitory.”  Using this approach content that is considered to be a non-record such as personal email, non-business correspondence, drafts, copies and so on can be explicitly classified as Transitory.  A retention rule is applied such as 120 days, and this Transitory content can be defensibly destroyed.  This approach reduces your content and storage requirements significantly and lowers discovery costs.

In this new paradigm, Records Management (RM) is a subset of an Enterprise Content Management (ECM) system.  The ECM solution enables RM to become part of the DNA of critical business processes such as Human Resources, Accounts Payable, Finance and Accounting, Case Management and more.  These processes receive and create records that are captured and indexed as a function of the business process so that RM classification is automatic.

Building these mainline business processes to include RM produces a full audit trail for all business transactions.  This supports the next key new business driver, Audit Readiness.

Audit Readiness

Audit Readiness is a term coined by some U.S. Department of Defense organizations to refer to the ability of an organization to respond quickly, accurately, and completely to an audit.  Paper based and non-automated solutions to this requirement can produce negative results and serious ramifications.  An automated process, on the other hand, can allow organizations to relatively easily produce a full audit trail of their business transactions, reducing the time and effort required to respond to an audit.

Because failed audits can result in potential costs in the millions of dollars depending on the context, this issue has come to the forefront as a market driver.  Using the Content Lifecycle model, this complete audit trail of information that supports mainline business processes is attainable.  For example, using OpenText’s Extended ECM for SAP, all supporting documentation around key business processes are capture, classified and retained based on a set of business rules.

The Presidential Memo – Managing Government Records

President Obama issued a memorandum to all agencies on Nov. 28, 2011 requiring each agency to appoint a senior official to deliver a plan for moving the agency from paper based records management to electronic records, specifically email, social media, cloud solutions, and so on.  Each agency has submitted their plan to the National Archives and the Office of Management and Budget.  This summer a directive will be generated to all agencies that will incorporate a number of ideas from agencies and the public for how to move to electronic records.

As is the case with many government directives, this is an unfunded mandate.  However, moving from paper to digital based recordkeeping and processes, as we know, has a very significant return on investment.  Therefore any solution deployed to solve this mandate, if deployed wisely, will pay for itself many times over.

Doing More with Less

One of the tag lines in the U.S. Federal Government these days is “doing more with less.”  This refers to the dynamic of shrinking budgets and resources combined with increasing volumes of information to manage.  This applies to the private sector as well as all levels of government in these challenging economic times.  Technology and automation are keys to successfully meeting this challenge.  Tools such as integration of RM and Information Governance into key business processes and email archiving, and using new tools such as Auto Classification help organizations cope with these increasing volumes. 

Big Bucket Retention Schedules

The new paradigm for RM includes a redefinition of Records Retention Schedules.  Legacy schedules were developed around a paper-based paradigm and typically are very granular, with many arbitrary retention rules that are now unnecessary and undesirable.  Many organizations have hundreds or even thousands of individual record series.  In the past several years there is an increasing trend toward Big Bucket Record Schedules that boil everything down to just a handful of “buckets.”  In the context of an ECM/RM solution, the ECM system contains all the metadata needed to search for and find records, so the retention schedule only needs to determine how long to keep it.  A streamlined Big Bucket schedule is easier to implement, is more intuitive, and compliance is much more effective.

The Government Accountability Office (GAO) is a leader in this trend, with a new schedule that contains three big buckets and a total of 27 sub-categories.  Other agencies are following GAO’s lead, and similar moves are being made in the private sector.  NARA has endorsed this approach so it is becoming increasingly more accepted.

To recap, the new market drivers that are now “State of the Art”:

  • eDiscovery and the Federal Rules of Civil Procedure
  • Redefinition of a Record to Include Everything
  • Audit Readiness
  • Presidential Mandate to Move from Paper to Digital Records
  • Do More with Less
  • Big Bucket Retention Schedules

Stay tuned for Part Two of this blog, how industry has responded to these new drivers.

#RM #bigbucket #Records-Management #discoverycosts #doingmorewithless #e-discovery #audit #ElectronicRecordsManagement


07-18-2012 13:02

Yes, NARA has officially endorsed flexible scheduling and big bucket retention schedules, and a number of agencies have moved to this approach. NARA does not require agencies to do this, however to meet the presidential mandate it is really the only pragmatic approach.

07-17-2012 17:33

Great summary of the forces driving significant change in RM thinking and practices. Just one minor point -- my info is that NARA has not actually officially endorsed Big Bucket scheduling of records yet. See conversation in this tweet exchange --