Increasing the efficiency of tape-based storage backends

HSM systems such as the CERN’s Advanced STORage manager (CASTOR) [1] are responsible for storing Petabytes of data which is first cached on disk and then persistently stored on tape media. The contents of these tapes are regularly repacked from older, lower-density media to new-generation, higher-density media in order to free up physical space and ensure long term data integrity and availability. With the evolution of price decay and higher capacity of disk (and flash memory) based storage, our future vision for tape usage is to move away from serving on demand, random, per-file access to non-disk cached files, and to move towards loosely coupled, efficient bulk data transfers where large-volume data sets are stored and retrieved in aggregations, fully exploiting the stream-based nature of tape media. Mechanisms for grouped migration policies and priorities have been implemented, and an innovative tape format optimized for data aggregations is being developed. This new tape format will also allow for increasing the performance of repacking data from old to newer generation tape media with substantially reduced hardware costs. In this paper, we will describe the improvements for migration policies and priorities, as well as the proposed tape format and the changes which will be applied to the architecture of the CASTOR mass storage system.