LTFS 1: Linear Tape File System - The Brilliant Breakthrough

By Nick Inglis posted 09-17-2014 10:47


This post is part 1 of a 3 part series about Linear Tape File System.

The Linear Tape File System (LTFS) is one of the most important breakthroughs in magnetic tape storage (yes, it's still alive and is experiencing a resurgence). LTFS is a fully encompassed file system that writes the file structure, files and metadata onto a self-contained magnetic tape. Previously, magnetic tape stored only the actual file, but file structure, metadata etc. were stored separately, which meant that tape had additional requirements for retrieval. Since the new LTFS technology is self-referential, it can be utilized without further information being "fed" into the stream of data from the tape. This new technology makes tape management much easier by making the file structure as familiar as an external hard drive with the limitless storage potential that tape can provide. Michael Richmond, the lead architect of this new technology, discussed the creation of LTFS with Jay Livens of Iron Mountain in a recent podcast.

The Initial Showing

Richmond discussed how, while at IBM, he and his team created and released the LTFS specification as an open source technology. Of the team's goals, Richmond posed it as a simple question: "Why can't tape look like a USB drive?" Richmond further asked, "Why can't our users work with tape the way they work with a drive that they plug into their PC?"

One of the most interesting portions of the conversation between Livens and Richmond was the discussion of the initial showing of the LTFS technology in 2009 at the National Association of Broadcasters (NAB) Show. The IBM team was working around the clock, pushing towards the deadline of NAB 2009, to show their new technology to the world. They had built the initial implementation around Linux and were porting the implementation to the Apple OS X platform to meet another of the team's goals — that the file system and tapes work across platforms. The IBM team believed that, if they only showed the Linux implementation, they would not garner credibility in the media and entertainment industry, which is the audience of the NAB Show. The final work on the implementation in the OS X environment was actually written in an SUV during the drive across the Mojave Desert to NAB 2009.

The feedback for the initial Linear Tape File System showing at NAB 2009 was overwhelmingly positive and was seen as a breakthrough in the space.

The LTFS Launch

Richmond and his then manager were pushing to move the LTFS specification into the open source space through the Linear Tape-Open (LTO) (a consortium of technology companies seeking to develop open standards for linear tape). With the many proprietary formats for tape that exist, Richmond and his manager pushed for LTFS to be moved towards an open standard, believing that companies would be less willing to buy into LTFS if it were released as a proprietary format since it would force those companies to work exclusively with IBM.

IBM agreed with Richmond and allowed him to bring the technology to the LTO, primarily HP and Quantum. By the time of the release of the standard in 2010, again at the NAB, the IBM Almaden Research Center along with the LTO announced the new LTFS specification, and HP announced their LTFS implementation. Within weeks of NAB 2010, IBM, Quantum and HP all had implementations of Linear Tape File System technology utilizing the same open code base developed by Richmond and the IBM Almaden Research Center team. Oracle, not being a member of the LTO consortium, later picked up on the standard as well and released further implementations. Each of the various technology providers, leveraging the same standards, now had interchangeable magnetic tape technology with files self-contained on the tape.

The Brilliant Breakthrough

This brilliant breakthrough of the Linear Tape File System being built into an open source specification is now forming the next generation of magnetic tape. As a long-term open specification, this self-contained and powerful way of saving files will ensure that the tape and its data is available well into the future.

#TieredStorage #StorageMedia #InformationStorage #LTFS #DataStorage #EnterpriseContentManagement