Cargando…

Exploring Future Storage Options for ATLAS at the BNL/SDCC facility

The ATLAS Experiment in the HL-LHC area is expected to deliver an unprecedented amount of scientific data. As the demand for disk storage capacity in ATLAS continues to rise steadily, the BNL Scientific Data and Computing Center (SDCC) faces challenges in terms of cost implications for maintaining m...

Descripción completa

Detalles Bibliográficos
Autores principales: Huang, Qiulan, Garonne, Vincent, Benjamin, Doug, Gamboa, Carlos Fernando, Misawa, Shigeki, Liu, Zhenping
Lenguaje:eng
Publicado: 2023
Materias:
Acceso en línea:http://cds.cern.ch/record/2869070
_version_ 1780978258812076032
author Huang, Qiulan
Garonne, Vincent
Benjamin, Doug
Gamboa, Carlos Fernando
Misawa, Shigeki
Liu, Zhenping
author_facet Huang, Qiulan
Garonne, Vincent
Benjamin, Doug
Gamboa, Carlos Fernando
Misawa, Shigeki
Liu, Zhenping
author_sort Huang, Qiulan
collection CERN
description The ATLAS Experiment in the HL-LHC area is expected to deliver an unprecedented amount of scientific data. As the demand for disk storage capacity in ATLAS continues to rise steadily, the BNL Scientific Data and Computing Center (SDCC) faces challenges in terms of cost implications for maintaining multiple disk copies and adapting to the coming ATLAS storage requirements. To address these challenges, , the SDCC Storage team has undertaken a thorough analysis of the ATLAS experiment’s requirements, matching them to the suitable storage options and strategies, and has explored alternatives to enhance or replace the current storage solution. This paper aims to present the main challenges encountered while supporting big data experiments such as ATLAS. We describe the experiment's specific requirements and priorities, particularly focusing on the critical storage system characteristics needed for the high-luminosity run and how the key storage components provided by the Storage team work together: the dCache disk storage system; its archival back-end, HPSS, and its OS-level backend Storage. Specifically, we investigate a novel approach to integrate Lustre and XRootD. In this setup, Lustre serves as backend storage and XRootD acts as an access layer frontend, supporting various grid access protocols. Additionally, we also describe the validation and commissioning tests, including a comparison between dCache and XRootd in performance. Furthermore, we provide a performance and cost analysis comparing OpenZFS and LINUX MD RAID, evaluate different storage software stacks, and showcase stress tests conducted to validate Third Party Copy (TPC) functionality.
id cern-2869070
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2023
record_format invenio
spelling cern-28690702023-08-31T18:34:39Zhttp://cds.cern.ch/record/2869070engHuang, QiulanGaronne, VincentBenjamin, DougGamboa, Carlos FernandoMisawa, ShigekiLiu, ZhenpingExploring Future Storage Options for ATLAS at the BNL/SDCC facilityParticle Physics - ExperimentThe ATLAS Experiment in the HL-LHC area is expected to deliver an unprecedented amount of scientific data. As the demand for disk storage capacity in ATLAS continues to rise steadily, the BNL Scientific Data and Computing Center (SDCC) faces challenges in terms of cost implications for maintaining multiple disk copies and adapting to the coming ATLAS storage requirements. To address these challenges, , the SDCC Storage team has undertaken a thorough analysis of the ATLAS experiment’s requirements, matching them to the suitable storage options and strategies, and has explored alternatives to enhance or replace the current storage solution. This paper aims to present the main challenges encountered while supporting big data experiments such as ATLAS. We describe the experiment's specific requirements and priorities, particularly focusing on the critical storage system characteristics needed for the high-luminosity run and how the key storage components provided by the Storage team work together: the dCache disk storage system; its archival back-end, HPSS, and its OS-level backend Storage. Specifically, we investigate a novel approach to integrate Lustre and XRootD. In this setup, Lustre serves as backend storage and XRootD acts as an access layer frontend, supporting various grid access protocols. Additionally, we also describe the validation and commissioning tests, including a comparison between dCache and XRootd in performance. Furthermore, we provide a performance and cost analysis comparing OpenZFS and LINUX MD RAID, evaluate different storage software stacks, and showcase stress tests conducted to validate Third Party Copy (TPC) functionality.ATL-SOFT-PROC-2023-019oai:cds.cern.ch:28690702023-08-31
spellingShingle Particle Physics - Experiment
Huang, Qiulan
Garonne, Vincent
Benjamin, Doug
Gamboa, Carlos Fernando
Misawa, Shigeki
Liu, Zhenping
Exploring Future Storage Options for ATLAS at the BNL/SDCC facility
title Exploring Future Storage Options for ATLAS at the BNL/SDCC facility
title_full Exploring Future Storage Options for ATLAS at the BNL/SDCC facility
title_fullStr Exploring Future Storage Options for ATLAS at the BNL/SDCC facility
title_full_unstemmed Exploring Future Storage Options for ATLAS at the BNL/SDCC facility
title_short Exploring Future Storage Options for ATLAS at the BNL/SDCC facility
title_sort exploring future storage options for atlas at the bnl/sdcc facility
topic Particle Physics - Experiment
url http://cds.cern.ch/record/2869070
work_keys_str_mv AT huangqiulan exploringfuturestorageoptionsforatlasatthebnlsdccfacility
AT garonnevincent exploringfuturestorageoptionsforatlasatthebnlsdccfacility
AT benjamindoug exploringfuturestorageoptionsforatlasatthebnlsdccfacility
AT gamboacarlosfernando exploringfuturestorageoptionsforatlasatthebnlsdccfacility
AT misawashigeki exploringfuturestorageoptionsforatlasatthebnlsdccfacility
AT liuzhenping exploringfuturestorageoptionsforatlasatthebnlsdccfacility