Cargando…

Exploring Object Stores for High-Energy Physics Data Storage

<!--HTML-->Over the last two decades, ROOT TTree has been used for storing over one exabyte of High-Energy Physics (HEP) events. The TTree columnar on-disk layout has been proved to be ideal for analyses of HEP data that typically require access to many events, but only a subset of the informa...

Descripción completa

Detalles Bibliográficos
Autor principal: Lopez Gomez, Javier
Lenguaje:eng
Publicado: 2021
Materias:
Acceso en línea:http://cds.cern.ch/record/2767246
_version_ 1780971281910333440
author Lopez Gomez, Javier
author_facet Lopez Gomez, Javier
author_sort Lopez Gomez, Javier
collection CERN
description <!--HTML-->Over the last two decades, ROOT TTree has been used for storing over one exabyte of High-Energy Physics (HEP) events. The TTree columnar on-disk layout has been proved to be ideal for analyses of HEP data that typically require access to many events, but only a subset of the information stored for each of them. Future accelerators, and particularly HL-LHC, will bring an increase of at least one order of magnitude in the volume of generated data. To this end, RNTuple has been designed to overcome TTree's limitations, providing improved efficiency and taking advantage of modern storage systems, e.g. low-latency high-bandwidth NVMe devices and object stores. In this paper, we extend RNTuple with a backend that leverages Intel DAOS as the underlying storage, proving that RNTuple's architecture can accommodate such changes. From the RNTuple user's perspective, this data can be accessed with minimal changes to the user code, i.e. replacing a filesystem path by a DAOS URI. Our performance evaluation shows that the contributed backend can be used for realistic analyses, while outperforming the compatibility solution provided by the DAOS project.
id cern-2767246
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2021
record_format invenio
spelling cern-27672462022-11-02T22:25:38Zhttp://cds.cern.ch/record/2767246engLopez Gomez, JavierExploring Object Stores for High-Energy Physics Data Storage25th International Conference on Computing in High Energy & Nuclear PhysicsConferences<!--HTML-->Over the last two decades, ROOT TTree has been used for storing over one exabyte of High-Energy Physics (HEP) events. The TTree columnar on-disk layout has been proved to be ideal for analyses of HEP data that typically require access to many events, but only a subset of the information stored for each of them. Future accelerators, and particularly HL-LHC, will bring an increase of at least one order of magnitude in the volume of generated data. To this end, RNTuple has been designed to overcome TTree's limitations, providing improved efficiency and taking advantage of modern storage systems, e.g. low-latency high-bandwidth NVMe devices and object stores. In this paper, we extend RNTuple with a backend that leverages Intel DAOS as the underlying storage, proving that RNTuple's architecture can accommodate such changes. From the RNTuple user's perspective, this data can be accessed with minimal changes to the user code, i.e. replacing a filesystem path by a DAOS URI. Our performance evaluation shows that the contributed backend can be used for realistic analyses, while outperforming the compatibility solution provided by the DAOS project.oai:cds.cern.ch:27672462021
spellingShingle Conferences
Lopez Gomez, Javier
Exploring Object Stores for High-Energy Physics Data Storage
title Exploring Object Stores for High-Energy Physics Data Storage
title_full Exploring Object Stores for High-Energy Physics Data Storage
title_fullStr Exploring Object Stores for High-Energy Physics Data Storage
title_full_unstemmed Exploring Object Stores for High-Energy Physics Data Storage
title_short Exploring Object Stores for High-Energy Physics Data Storage
title_sort exploring object stores for high-energy physics data storage
topic Conferences
url http://cds.cern.ch/record/2767246
work_keys_str_mv AT lopezgomezjavier exploringobjectstoresforhighenergyphysicsdatastorage
AT lopezgomezjavier 25thinternationalconferenceoncomputinginhighenergynuclearphysics