Cargando…
The ATLAS ROOT-based data formats: recent improvements and performance measurements
We detail recent changes to ROOT-based I/O within the ATLAS experiment. The ATLAS persistent event data model continues to make considerable use of a ROOT I/O backend through POOL persistency. Also ROOT is used directly in later stages of analysis that make use of a flat-ntuple based "D3PD"...
Autores principales: | , , , , , |
---|---|
Lenguaje: | eng |
Publicado: |
2012
|
Materias: | |
Acceso en línea: | https://dx.doi.org/10.1088/1742-6596/396/2/022006 http://cds.cern.ch/record/1448601 |
_version_ | 1780924861549379584 |
---|---|
author | Bhimji, W Cranshaw, J van Gemmeren, P Malon, D Schaffer, R D Vukotic, I |
author_facet | Bhimji, W Cranshaw, J van Gemmeren, P Malon, D Schaffer, R D Vukotic, I |
author_sort | Bhimji, W |
collection | CERN |
description | We detail recent changes to ROOT-based I/O within the ATLAS experiment. The ATLAS persistent event data model continues to make considerable use of a ROOT I/O backend through POOL persistency. Also ROOT is used directly in later stages of analysis that make use of a flat-ntuple based "D3PD" data-type. For POOL/ROOT persistent data, several improvements have been made including implementation of automatic basket optimisation, memberwise streaming, and changes to split and compression levels. Optimisations are also planned for the D3PD format. We present a full evaluation of the resulting performance improvements from these, including in the case of selected retrieval of events. We also evaluate ongoing changes internal to ROOT, in the ATLAS context, for both POOL and D3PD data. We report results not only from test systems, but also utilising new automated tests on real ATLAS production resources which employ a wide range of storage technologies. |
id | cern-1448601 |
institution | Organización Europea para la Investigación Nuclear |
language | eng |
publishDate | 2012 |
record_format | invenio |
spelling | cern-14486012019-09-30T06:29:59Zdoi:10.1088/1742-6596/396/2/022006http://cds.cern.ch/record/1448601engBhimji, WCranshaw, Jvan Gemmeren, PMalon, DSchaffer, R DVukotic, IThe ATLAS ROOT-based data formats: recent improvements and performance measurementsDetectors and Experimental TechniquesWe detail recent changes to ROOT-based I/O within the ATLAS experiment. The ATLAS persistent event data model continues to make considerable use of a ROOT I/O backend through POOL persistency. Also ROOT is used directly in later stages of analysis that make use of a flat-ntuple based "D3PD" data-type. For POOL/ROOT persistent data, several improvements have been made including implementation of automatic basket optimisation, memberwise streaming, and changes to split and compression levels. Optimisations are also planned for the D3PD format. We present a full evaluation of the resulting performance improvements from these, including in the case of selected retrieval of events. We also evaluate ongoing changes internal to ROOT, in the ATLAS context, for both POOL and D3PD data. We report results not only from test systems, but also utilising new automated tests on real ATLAS production resources which employ a wide range of storage technologies.ATL-SOFT-PROC-2012-020oai:cds.cern.ch:14486012012-05-14 |
spellingShingle | Detectors and Experimental Techniques Bhimji, W Cranshaw, J van Gemmeren, P Malon, D Schaffer, R D Vukotic, I The ATLAS ROOT-based data formats: recent improvements and performance measurements |
title | The ATLAS ROOT-based data formats: recent improvements and performance measurements |
title_full | The ATLAS ROOT-based data formats: recent improvements and performance measurements |
title_fullStr | The ATLAS ROOT-based data formats: recent improvements and performance measurements |
title_full_unstemmed | The ATLAS ROOT-based data formats: recent improvements and performance measurements |
title_short | The ATLAS ROOT-based data formats: recent improvements and performance measurements |
title_sort | atlas root-based data formats: recent improvements and performance measurements |
topic | Detectors and Experimental Techniques |
url | https://dx.doi.org/10.1088/1742-6596/396/2/022006 http://cds.cern.ch/record/1448601 |
work_keys_str_mv | AT bhimjiw theatlasrootbaseddataformatsrecentimprovementsandperformancemeasurements AT cranshawj theatlasrootbaseddataformatsrecentimprovementsandperformancemeasurements AT vangemmerenp theatlasrootbaseddataformatsrecentimprovementsandperformancemeasurements AT malond theatlasrootbaseddataformatsrecentimprovementsandperformancemeasurements AT schafferrd theatlasrootbaseddataformatsrecentimprovementsandperformancemeasurements AT vukotici theatlasrootbaseddataformatsrecentimprovementsandperformancemeasurements AT bhimjiw atlasrootbaseddataformatsrecentimprovementsandperformancemeasurements AT cranshawj atlasrootbaseddataformatsrecentimprovementsandperformancemeasurements AT vangemmerenp atlasrootbaseddataformatsrecentimprovementsandperformancemeasurements AT malond atlasrootbaseddataformatsrecentimprovementsandperformancemeasurements AT schafferrd atlasrootbaseddataformatsrecentimprovementsandperformancemeasurements AT vukotici atlasrootbaseddataformatsrecentimprovementsandperformancemeasurements |