Cargando…

Experiences with the new ATLAS Distributed Data Management System

The ATLAS Distributed Data Management (DDM) system has evolved drastically in the last two years with the Rucio software fully replacing the previous system before the start of LHC Run-2. The ATLAS DDM system manages now more than 200 petabytes spread on 130 storage sites and can handle file transfe...

Descripción completa

Detalles Bibliográficos
Autor principal: Garonne, Vincent
Lenguaje:eng
Publicado: 2016
Materias:
Acceso en línea:http://cds.cern.ch/record/2220926
_version_ 1780952213673213952
author Garonne, Vincent
author_facet Garonne, Vincent
author_sort Garonne, Vincent
collection CERN
description The ATLAS Distributed Data Management (DDM) system has evolved drastically in the last two years with the Rucio software fully replacing the previous system before the start of LHC Run-2. The ATLAS DDM system manages now more than 200 petabytes spread on 130 storage sites and can handle file transfer rates of up to 30Hz. In this talk, we discuss our experience acquired in developing, commissioning, running and maintaining such a large system. First, we describe the general architecture of the system, our integration with external services like the WLCG File Transfer Service and the evolution of the system over its first year of production. Then, we show the performance of the system, describe the integration of new technologies such as object stores, and outline future developments which mainly focus on performance and automation. Finally we discuss the long term evolution of ATLAS data management.
id cern-2220926
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2016
record_format invenio
spelling cern-22209262019-09-30T06:29:59Zhttp://cds.cern.ch/record/2220926engGaronne, VincentExperiences with the new ATLAS Distributed Data Management SystemParticle Physics - ExperimentThe ATLAS Distributed Data Management (DDM) system has evolved drastically in the last two years with the Rucio software fully replacing the previous system before the start of LHC Run-2. The ATLAS DDM system manages now more than 200 petabytes spread on 130 storage sites and can handle file transfer rates of up to 30Hz. In this talk, we discuss our experience acquired in developing, commissioning, running and maintaining such a large system. First, we describe the general architecture of the system, our integration with external services like the WLCG File Transfer Service and the evolution of the system over its first year of production. Then, we show the performance of the system, describe the integration of new technologies such as object stores, and outline future developments which mainly focus on performance and automation. Finally we discuss the long term evolution of ATLAS data management.ATL-SOFT-SLIDE-2016-747oai:cds.cern.ch:22209262016-10-03
spellingShingle Particle Physics - Experiment
Garonne, Vincent
Experiences with the new ATLAS Distributed Data Management System
title Experiences with the new ATLAS Distributed Data Management System
title_full Experiences with the new ATLAS Distributed Data Management System
title_fullStr Experiences with the new ATLAS Distributed Data Management System
title_full_unstemmed Experiences with the new ATLAS Distributed Data Management System
title_short Experiences with the new ATLAS Distributed Data Management System
title_sort experiences with the new atlas distributed data management system
topic Particle Physics - Experiment
url http://cds.cern.ch/record/2220926
work_keys_str_mv AT garonnevincent experienceswiththenewatlasdistributeddatamanagementsystem