Cargando…

Experiences with the new ATLAS Distributed Data Management System

The ATLAS Distributed Data Management (DDM) system has evolved drastically in the last two years with the Rucio software fully replacing the previous system before the start of LHC Run-2. The ATLAS DDM system manages now more than 250 petabytes spread on 130 storage sites and can handle file transfe...

Descripción completa

Detalles Bibliográficos
Autores principales: Garonne, Vincent, Serfon, Cedric, Barisits, Martin-Stefan, Lassnig, Mario, Beermann, Thomas, Guan, Wen
Lenguaje:eng
Publicado: 2017
Materias:
Acceso en línea:https://dx.doi.org/10.1088/1742-6596/898/6/062019
http://cds.cern.ch/record/2244347
_version_ 1780953419061657600
author Garonne, Vincent
Serfon, Cedric
Barisits, Martin-Stefan
Lassnig, Mario
Beermann, Thomas
Guan, Wen
author_facet Garonne, Vincent
Serfon, Cedric
Barisits, Martin-Stefan
Lassnig, Mario
Beermann, Thomas
Guan, Wen
author_sort Garonne, Vincent
collection CERN
description The ATLAS Distributed Data Management (DDM) system has evolved drastically in the last two years with the Rucio software fully replacing the previous system before the start of LHC Run-2. The ATLAS DDM system manages now more than 250 petabytes spread on 130 storage sites and can handle file transfer rates of up to 30Hz. In this paper, we discuss our experience acquired in developing, commissioning, running and maintaining such a large system. First, we describe the general architecture of the system, our integration with external services like the WLCG File Transfer Service and the evolution of the system over its first years of production. Then, we show the performance of the system, describe the integration of new technologies such as object stores, and outline some new developments, which mainly focus on performance and automation.
id cern-2244347
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2017
record_format invenio
spelling cern-22443472019-10-15T15:17:18Zdoi:10.1088/1742-6596/898/6/062019http://cds.cern.ch/record/2244347engGaronne, VincentSerfon, CedricBarisits, Martin-StefanLassnig, MarioBeermann, ThomasGuan, WenExperiences with the new ATLAS Distributed Data Management SystemParticle Physics - ExperimentThe ATLAS Distributed Data Management (DDM) system has evolved drastically in the last two years with the Rucio software fully replacing the previous system before the start of LHC Run-2. The ATLAS DDM system manages now more than 250 petabytes spread on 130 storage sites and can handle file transfer rates of up to 30Hz. In this paper, we discuss our experience acquired in developing, commissioning, running and maintaining such a large system. First, we describe the general architecture of the system, our integration with external services like the WLCG File Transfer Service and the evolution of the system over its first years of production. Then, we show the performance of the system, describe the integration of new technologies such as object stores, and outline some new developments, which mainly focus on performance and automation.ATL-SOFT-PROC-2017-042oai:cds.cern.ch:22443472017-02-03
spellingShingle Particle Physics - Experiment
Garonne, Vincent
Serfon, Cedric
Barisits, Martin-Stefan
Lassnig, Mario
Beermann, Thomas
Guan, Wen
Experiences with the new ATLAS Distributed Data Management System
title Experiences with the new ATLAS Distributed Data Management System
title_full Experiences with the new ATLAS Distributed Data Management System
title_fullStr Experiences with the new ATLAS Distributed Data Management System
title_full_unstemmed Experiences with the new ATLAS Distributed Data Management System
title_short Experiences with the new ATLAS Distributed Data Management System
title_sort experiences with the new atlas distributed data management system
topic Particle Physics - Experiment
url https://dx.doi.org/10.1088/1742-6596/898/6/062019
http://cds.cern.ch/record/2244347
work_keys_str_mv AT garonnevincent experienceswiththenewatlasdistributeddatamanagementsystem
AT serfoncedric experienceswiththenewatlasdistributeddatamanagementsystem
AT barisitsmartinstefan experienceswiththenewatlasdistributeddatamanagementsystem
AT lassnigmario experienceswiththenewatlasdistributeddatamanagementsystem
AT beermannthomas experienceswiththenewatlasdistributeddatamanagementsystem
AT guanwen experienceswiththenewatlasdistributeddatamanagementsystem