Cargando…
Experiences with the new ATLAS Distributed Data Management System
The ATLAS Distributed Data Management (DDM) system has evolved drastically in the last two years with the Rucio software fully replacing the previous system before the start of LHC Run-2. The ATLAS DDM system manages now more than 250 petabytes spread on 130 storage sites and can handle file transfe...
Autores principales: | , , , , , |
---|---|
Lenguaje: | eng |
Publicado: |
2017
|
Materias: | |
Acceso en línea: | https://dx.doi.org/10.1088/1742-6596/898/6/062019 http://cds.cern.ch/record/2244347 |
_version_ | 1780953419061657600 |
---|---|
author | Garonne, Vincent Serfon, Cedric Barisits, Martin-Stefan Lassnig, Mario Beermann, Thomas Guan, Wen |
author_facet | Garonne, Vincent Serfon, Cedric Barisits, Martin-Stefan Lassnig, Mario Beermann, Thomas Guan, Wen |
author_sort | Garonne, Vincent |
collection | CERN |
description | The ATLAS Distributed Data Management (DDM) system has evolved drastically in the last two years with the Rucio software fully replacing the previous system before the start of LHC Run-2. The ATLAS DDM system manages now more than 250 petabytes spread on 130 storage sites and can handle file transfer rates of up to 30Hz. In this paper, we discuss our experience acquired in developing, commissioning, running and maintaining such a large system. First, we describe the general architecture of the system, our integration with external services like the WLCG File Transfer Service and the evolution of the system over its first years of production. Then, we show the performance of the system, describe the integration of new technologies such as object stores, and outline some new developments, which mainly focus on performance and automation. |
id | cern-2244347 |
institution | Organización Europea para la Investigación Nuclear |
language | eng |
publishDate | 2017 |
record_format | invenio |
spelling | cern-22443472019-10-15T15:17:18Zdoi:10.1088/1742-6596/898/6/062019http://cds.cern.ch/record/2244347engGaronne, VincentSerfon, CedricBarisits, Martin-StefanLassnig, MarioBeermann, ThomasGuan, WenExperiences with the new ATLAS Distributed Data Management SystemParticle Physics - ExperimentThe ATLAS Distributed Data Management (DDM) system has evolved drastically in the last two years with the Rucio software fully replacing the previous system before the start of LHC Run-2. The ATLAS DDM system manages now more than 250 petabytes spread on 130 storage sites and can handle file transfer rates of up to 30Hz. In this paper, we discuss our experience acquired in developing, commissioning, running and maintaining such a large system. First, we describe the general architecture of the system, our integration with external services like the WLCG File Transfer Service and the evolution of the system over its first years of production. Then, we show the performance of the system, describe the integration of new technologies such as object stores, and outline some new developments, which mainly focus on performance and automation.ATL-SOFT-PROC-2017-042oai:cds.cern.ch:22443472017-02-03 |
spellingShingle | Particle Physics - Experiment Garonne, Vincent Serfon, Cedric Barisits, Martin-Stefan Lassnig, Mario Beermann, Thomas Guan, Wen Experiences with the new ATLAS Distributed Data Management System |
title | Experiences with the new ATLAS Distributed Data Management System |
title_full | Experiences with the new ATLAS Distributed Data Management System |
title_fullStr | Experiences with the new ATLAS Distributed Data Management System |
title_full_unstemmed | Experiences with the new ATLAS Distributed Data Management System |
title_short | Experiences with the new ATLAS Distributed Data Management System |
title_sort | experiences with the new atlas distributed data management system |
topic | Particle Physics - Experiment |
url | https://dx.doi.org/10.1088/1742-6596/898/6/062019 http://cds.cern.ch/record/2244347 |
work_keys_str_mv | AT garonnevincent experienceswiththenewatlasdistributeddatamanagementsystem AT serfoncedric experienceswiththenewatlasdistributeddatamanagementsystem AT barisitsmartinstefan experienceswiththenewatlasdistributeddatamanagementsystem AT lassnigmario experienceswiththenewatlasdistributeddatamanagementsystem AT beermannthomas experienceswiththenewatlasdistributeddatamanagementsystem AT guanwen experienceswiththenewatlasdistributeddatamanagementsystem |