Cargando…
RNTuple performance: Status and Outlook
Upcoming HEP experiments, e.g. at the HL-LHC, are expected to increase the volume of generated data by at least one order of magnitude. In order to retain the ability to analyze the influx of data, full exploitation of modern storage hardware and systems, such as low-latency high-bandwidth NVMe devi...
Autores principales: | , |
---|---|
Lenguaje: | eng |
Publicado: |
2023
|
Materias: | |
Acceso en línea: | https://dx.doi.org/10.1088/1742-6596/2438/1/012118 http://cds.cern.ch/record/2808833 |
_version_ | 1780973117013753856 |
---|---|
author | Lopez-Gomez, Javier Blomer, Jakob |
author_facet | Lopez-Gomez, Javier Blomer, Jakob |
author_sort | Lopez-Gomez, Javier |
collection | CERN |
description | Upcoming HEP experiments, e.g. at the HL-LHC, are expected to increase the volume of generated data by at least one order of magnitude. In order to retain the ability to analyze the influx of data, full exploitation of modern storage hardware and systems, such as low-latency high-bandwidth NVMe devices and distributed object stores, becomes critical.To this end, the ROOT RNTuple I/O subsystem has been designed to address performance bottlenecks and shortcomings of ROOT’s current state of the art TTree I/O subsystem. RNTuple provides a backwards-incompatible redesign of the TTree binary format and access API that evolves the ROOT event data I/O for the challenges of the upcoming decades. It focuses on a compact data format, on performance engineering for modern storage hardware, for instance through making parallel and asynchronous I/O calls by default, and on robust interfaces that are easy to use correctly.In this contribution, we evaluate the RNTuple performance for typical HEP analysis tasks. We compare the throughput delivered by RNTuple to popular I/O libraries outside HEP, such as HDF5 and Apache Parquet. We demonstrate the advantages of RNTuple for HEP analysis workflows and provide an outlook on the road to its use in production. |
id | cern-2808833 |
institution | Organización Europea para la Investigación Nuclear |
language | eng |
publishDate | 2023 |
record_format | invenio |
spelling | cern-28088332023-03-05T03:26:44Zdoi:10.1088/1742-6596/2438/1/012118http://cds.cern.ch/record/2808833engLopez-Gomez, JavierBlomer, JakobRNTuple performance: Status and OutlookOther Fields of PhysicsAccelerators and Storage RingsComputing and ComputersUpcoming HEP experiments, e.g. at the HL-LHC, are expected to increase the volume of generated data by at least one order of magnitude. In order to retain the ability to analyze the influx of data, full exploitation of modern storage hardware and systems, such as low-latency high-bandwidth NVMe devices and distributed object stores, becomes critical.To this end, the ROOT RNTuple I/O subsystem has been designed to address performance bottlenecks and shortcomings of ROOT’s current state of the art TTree I/O subsystem. RNTuple provides a backwards-incompatible redesign of the TTree binary format and access API that evolves the ROOT event data I/O for the challenges of the upcoming decades. It focuses on a compact data format, on performance engineering for modern storage hardware, for instance through making parallel and asynchronous I/O calls by default, and on robust interfaces that are easy to use correctly.In this contribution, we evaluate the RNTuple performance for typical HEP analysis tasks. We compare the throughput delivered by RNTuple to popular I/O libraries outside HEP, such as HDF5 and Apache Parquet. We demonstrate the advantages of RNTuple for HEP analysis workflows and provide an outlook on the road to its use in production.Upcoming HEP experiments, e.g. at the HL-LHC, are expected to increase the volume of generated data by at least one order of magnitude. In order to retain the ability to analyze the influx of data, full exploitation of modern storage hardware and systems, such as low-latency high-bandwidth NVMe devices and distributed object stores, becomes critical. To this end, the ROOT RNTuple I/O subsystem has been designed to address performance bottlenecks and shortcomings of ROOT's current state of the art TTree I/O subsystem. RNTuple provides a backwards-incompatible redesign of the TTree binary format and access API that evolves the ROOT event data I/O for the challenges of the upcoming decades. It focuses on a compact data format, on performance engineering for modern storage hardware, for instance through making parallel and asynchronous I/O calls by default, and on robust interfaces that are easy to use correctly. In this contribution, we evaluate the RNTuple performance for typical HEP analysis tasks. We compare the throughput delivered by RNTuple to popular I/O libraries outside HEP, such as HDF5 and Apache Parquet. We demonstrate the advantages of RNTuple for HEP analysis workflows and provide an outlook on the road to its use in production.arXiv:2204.09043oai:cds.cern.ch:28088332023 |
spellingShingle | Other Fields of Physics Accelerators and Storage Rings Computing and Computers Lopez-Gomez, Javier Blomer, Jakob RNTuple performance: Status and Outlook |
title | RNTuple performance: Status and Outlook |
title_full | RNTuple performance: Status and Outlook |
title_fullStr | RNTuple performance: Status and Outlook |
title_full_unstemmed | RNTuple performance: Status and Outlook |
title_short | RNTuple performance: Status and Outlook |
title_sort | rntuple performance: status and outlook |
topic | Other Fields of Physics Accelerators and Storage Rings Computing and Computers |
url | https://dx.doi.org/10.1088/1742-6596/2438/1/012118 http://cds.cern.ch/record/2808833 |
work_keys_str_mv | AT lopezgomezjavier rntupleperformancestatusandoutlook AT blomerjakob rntupleperformancestatusandoutlook |