Cargando…

Rucio - The next generation of large scale distributed system for ATLAS Data Management

Rucio is the next-generation Distributed Data Management(DDM) system benefiting from recent advances in cloud and "Big Data" computing to address HEP experiments scaling requirements. Rucio is an evolution of the ATLAS DDM system Don Quijote 2 (DQ2), which has demonstrated very large scale...

Descripción completa

Detalles Bibliográficos
Autores principales: Garonne, V, Barisits, M, Beermann, T, Goossens, L, Lassnig, M, Nairz, A, Stewart, GA, Vigne, V, Serfon, C
Lenguaje:eng
Publicado: 2013
Materias:
Acceso en línea:http://cds.cern.ch/record/1606094
_version_ 1780931668450738176
author Garonne, V
Barisits, M
Beermann, T
Goossens, L
Lassnig, M
Nairz, A
Stewart, GA
Vigne, V
Serfon, C
author_facet Garonne, V
Barisits, M
Beermann, T
Goossens, L
Lassnig, M
Nairz, A
Stewart, GA
Vigne, V
Serfon, C
author_sort Garonne, V
collection CERN
description Rucio is the next-generation Distributed Data Management(DDM) system benefiting from recent advances in cloud and "Big Data" computing to address HEP experiments scaling requirements. Rucio is an evolution of the ATLAS DDM system Don Quijote 2 (DQ2), which has demonstrated very large scale data management capabilities with more than 140 petabytes spread worldwide across 130 sites, and accesses from 1,000 active users. However, DQ2 is reaching its limits in terms of scalability, requiring a large number of support staff to operate and being hard to extend with new technologies. Rucio will address these issues by relying on a conceptual data model and new technology to ensure system scalability, address new user requirements and employ new automation framework to reduce operational overheads. We present the key concepts of Rucio, including its data organization/representation and a model of how ATLAS central group and user activities will be managed. The Rucio design, and the technology it employs, is described, specifically looking at its RESTful architecture and the various software components it uses. We show also the performance of the system.
id cern-1606094
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2013
record_format invenio
spelling cern-16060942019-09-30T06:29:59Zhttp://cds.cern.ch/record/1606094engGaronne, VBarisits, MBeermann, TGoossens, LLassnig, MNairz, AStewart, GAVigne, VSerfon, CRucio - The next generation of large scale distributed system for ATLAS Data ManagementDetectors and Experimental TechniquesRucio is the next-generation Distributed Data Management(DDM) system benefiting from recent advances in cloud and "Big Data" computing to address HEP experiments scaling requirements. Rucio is an evolution of the ATLAS DDM system Don Quijote 2 (DQ2), which has demonstrated very large scale data management capabilities with more than 140 petabytes spread worldwide across 130 sites, and accesses from 1,000 active users. However, DQ2 is reaching its limits in terms of scalability, requiring a large number of support staff to operate and being hard to extend with new technologies. Rucio will address these issues by relying on a conceptual data model and new technology to ensure system scalability, address new user requirements and employ new automation framework to reduce operational overheads. We present the key concepts of Rucio, including its data organization/representation and a model of how ATLAS central group and user activities will be managed. The Rucio design, and the technology it employs, is described, specifically looking at its RESTful architecture and the various software components it uses. We show also the performance of the system.ATL-SOFT-SLIDE-2013-800oai:cds.cern.ch:16060942013-10-07
spellingShingle Detectors and Experimental Techniques
Garonne, V
Barisits, M
Beermann, T
Goossens, L
Lassnig, M
Nairz, A
Stewart, GA
Vigne, V
Serfon, C
Rucio - The next generation of large scale distributed system for ATLAS Data Management
title Rucio - The next generation of large scale distributed system for ATLAS Data Management
title_full Rucio - The next generation of large scale distributed system for ATLAS Data Management
title_fullStr Rucio - The next generation of large scale distributed system for ATLAS Data Management
title_full_unstemmed Rucio - The next generation of large scale distributed system for ATLAS Data Management
title_short Rucio - The next generation of large scale distributed system for ATLAS Data Management
title_sort rucio - the next generation of large scale distributed system for atlas data management
topic Detectors and Experimental Techniques
url http://cds.cern.ch/record/1606094
work_keys_str_mv AT garonnev ruciothenextgenerationoflargescaledistributedsystemforatlasdatamanagement
AT barisitsm ruciothenextgenerationoflargescaledistributedsystemforatlasdatamanagement
AT beermannt ruciothenextgenerationoflargescaledistributedsystemforatlasdatamanagement
AT goossensl ruciothenextgenerationoflargescaledistributedsystemforatlasdatamanagement
AT lassnigm ruciothenextgenerationoflargescaledistributedsystemforatlasdatamanagement
AT nairza ruciothenextgenerationoflargescaledistributedsystemforatlasdatamanagement
AT stewartga ruciothenextgenerationoflargescaledistributedsystemforatlasdatamanagement
AT vignev ruciothenextgenerationoflargescaledistributedsystemforatlasdatamanagement
AT serfonc ruciothenextgenerationoflargescaledistributedsystemforatlasdatamanagement