Multi-Threaded Evolution of the Data-Logging System of the ATLAS Experiment at CERN

The ATLAS experiment is currently observing proton-proton collisions delivered by the LHC accelerator at a centre of mass energy of 7 TeV with a peak luminosity of ~1033 cm-2 s-1. The ATLAS Trigger and Data Acquisition (TDAQ) system selects interesting events on-line in a three-level trigger system in order to store them at a budgeted rate of ~200 Hz for an event size of ~1.5 MB. This paper focuses on the TDAQ data-logging system. Its purpose is to receive events from the third level trigger, process them and stream the results into different raw data files according to the trigger decision. The data files are subsequently moved to the central mass storage facility at CERN. The system currently in production has been commissioned in 2007 and has been working smoothly since then. It is however based on an essentially single-threaded design that is anticipated not to cope with the increase in event rate and event size that is foreseen as part of the ATLAS and LHC upgrade programs. This design also severely limit the possible CPU-intensive tasks that can be performed. An example of these tasks is on-line event-level data compression: such a feature would allow to cope with an increased data rate without imposing additional throughput and storage space requirements. Therefore, a novel design that is able to exploit the full power of the multi-core architecture is needed. The main challenge of such a design is the conflict between the largely parallel nature of the data-logging event processing and the constraint of sequential file writing. This is further complicated by the fact that the application behaviour is required to be fully data-driven, since the trigger configuration and decisions cannot be assumed a priori. In this paper, we present a thread-pool based implementation of the TDAQ data-logging software. We report here on the functionality and performance of the new system and on our development experience.