Using Containers on HPCs for ATLAS Experiment

CVMFS helps ATLAS in distributing software to the Grid, and isolating software lookup to batch nodes’ local filesystems. But CVMFS is rarely available in HPC environments. ATLAS computing has experimented with all-inclusive containers, and later developed an environment to produce such containers for both Shifter and Singularity. The containers include most of the recent ATLAS software releases, conditions database, and other tools extracted from CVMFS. The approach not only helped ATLAS to distribute software automatically to HPCs with an environment identical to those in CVMFS, but also significantly reduced the metadata I/O to HPCs’ shared filesystems. The production operation at NERSC has proved that by using all-inclusive containers, we can both scale up to run many more jobs simultaneously, and transparently fit into the previously developed ATLAS operation methods.