Cargando…

The ATLAS ROOT-based data formats: recent improvements and performance measurements

We detail recent changes to ROOT-based I/O within the ATLAS experiment. The ATLAS persistent event data model continues to make considerable use of a ROOT I/O backend through POOL persistency. Also ROOT is used directly in later stages of analysis that make use of a flat-ntuple based "D3PD"...

Descripción completa

Detalles Bibliográficos
Autores principales: Bhimji, W, Cranshaw, J, van Gemmeren, P, Malon, D, Schaffer, R D, Vukotic, I
Lenguaje:eng
Publicado: 2012
Materias:
Acceso en línea:https://dx.doi.org/10.1088/1742-6596/396/2/022006
http://cds.cern.ch/record/1448601
_version_ 1780924861549379584
author Bhimji, W
Cranshaw, J
van Gemmeren, P
Malon, D
Schaffer, R D
Vukotic, I
author_facet Bhimji, W
Cranshaw, J
van Gemmeren, P
Malon, D
Schaffer, R D
Vukotic, I
author_sort Bhimji, W
collection CERN
description We detail recent changes to ROOT-based I/O within the ATLAS experiment. The ATLAS persistent event data model continues to make considerable use of a ROOT I/O backend through POOL persistency. Also ROOT is used directly in later stages of analysis that make use of a flat-ntuple based "D3PD" data-type. For POOL/ROOT persistent data, several improvements have been made including implementation of automatic basket optimisation, memberwise streaming, and changes to split and compression levels. Optimisations are also planned for the D3PD format. We present a full evaluation of the resulting performance improvements from these, including in the case of selected retrieval of events. We also evaluate ongoing changes internal to ROOT, in the ATLAS context, for both POOL and D3PD data. We report results not only from test systems, but also utilising new automated tests on real ATLAS production resources which employ a wide range of storage technologies.
id cern-1448601
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2012
record_format invenio
spelling cern-14486012019-09-30T06:29:59Zdoi:10.1088/1742-6596/396/2/022006http://cds.cern.ch/record/1448601engBhimji, WCranshaw, Jvan Gemmeren, PMalon, DSchaffer, R DVukotic, IThe ATLAS ROOT-based data formats: recent improvements and performance measurementsDetectors and Experimental TechniquesWe detail recent changes to ROOT-based I/O within the ATLAS experiment. The ATLAS persistent event data model continues to make considerable use of a ROOT I/O backend through POOL persistency. Also ROOT is used directly in later stages of analysis that make use of a flat-ntuple based "D3PD" data-type. For POOL/ROOT persistent data, several improvements have been made including implementation of automatic basket optimisation, memberwise streaming, and changes to split and compression levels. Optimisations are also planned for the D3PD format. We present a full evaluation of the resulting performance improvements from these, including in the case of selected retrieval of events. We also evaluate ongoing changes internal to ROOT, in the ATLAS context, for both POOL and D3PD data. We report results not only from test systems, but also utilising new automated tests on real ATLAS production resources which employ a wide range of storage technologies.ATL-SOFT-PROC-2012-020oai:cds.cern.ch:14486012012-05-14
spellingShingle Detectors and Experimental Techniques
Bhimji, W
Cranshaw, J
van Gemmeren, P
Malon, D
Schaffer, R D
Vukotic, I
The ATLAS ROOT-based data formats: recent improvements and performance measurements
title The ATLAS ROOT-based data formats: recent improvements and performance measurements
title_full The ATLAS ROOT-based data formats: recent improvements and performance measurements
title_fullStr The ATLAS ROOT-based data formats: recent improvements and performance measurements
title_full_unstemmed The ATLAS ROOT-based data formats: recent improvements and performance measurements
title_short The ATLAS ROOT-based data formats: recent improvements and performance measurements
title_sort atlas root-based data formats: recent improvements and performance measurements
topic Detectors and Experimental Techniques
url https://dx.doi.org/10.1088/1742-6596/396/2/022006
http://cds.cern.ch/record/1448601
work_keys_str_mv AT bhimjiw theatlasrootbaseddataformatsrecentimprovementsandperformancemeasurements
AT cranshawj theatlasrootbaseddataformatsrecentimprovementsandperformancemeasurements
AT vangemmerenp theatlasrootbaseddataformatsrecentimprovementsandperformancemeasurements
AT malond theatlasrootbaseddataformatsrecentimprovementsandperformancemeasurements
AT schafferrd theatlasrootbaseddataformatsrecentimprovementsandperformancemeasurements
AT vukotici theatlasrootbaseddataformatsrecentimprovementsandperformancemeasurements
AT bhimjiw atlasrootbaseddataformatsrecentimprovementsandperformancemeasurements
AT cranshawj atlasrootbaseddataformatsrecentimprovementsandperformancemeasurements
AT vangemmerenp atlasrootbaseddataformatsrecentimprovementsandperformancemeasurements
AT malond atlasrootbaseddataformatsrecentimprovementsandperformancemeasurements
AT schafferrd atlasrootbaseddataformatsrecentimprovementsandperformancemeasurements
AT vukotici atlasrootbaseddataformatsrecentimprovementsandperformancemeasurements