Cargando…

The DMLite Rucio Plugin: ATLAS data in a filesystem

Rucio is the next-generation data management system of the ATLAS experiment. Historically, clients interacted with the data management system via specialised tools, but in Rucio additional methods are provided. To support filesystem-like interaction with all ATLAS data, a plugin to the DMLite softwa...

Descripción completa

Detalles Bibliográficos
Autores principales: Lassnig, M, van Dongen, D, Brito Da Rocha, R, Alvarez Ayllon, A, Calfayan, P
Lenguaje:eng
Publicado: 2013
Materias:
Acceso en línea:https://dx.doi.org/10.1088/1742-6596/513/4/042030
http://cds.cern.ch/record/1606789
_version_ 1780931693082836992
author Lassnig, M
van Dongen, D
Brito Da Rocha, R
Alvarez Ayllon, A
Calfayan, P
author_facet Lassnig, M
van Dongen, D
Brito Da Rocha, R
Alvarez Ayllon, A
Calfayan, P
author_sort Lassnig, M
collection CERN
description Rucio is the next-generation data management system of the ATLAS experiment. Historically, clients interacted with the data management system via specialised tools, but in Rucio additional methods are provided. To support filesystem-like interaction with all ATLAS data, a plugin to the DMLite software stack has been developed. It is possible to mount Rucio as a filesystem, and execute regular filesystem operations in a POSIX fashion. This is exposed via various protocols, for example, WebDAV or NFS, which then removes any dependency on Rucio for client software. The main challenge for this work is the mapping of the set-like ATLAS namespace into a hierarchical filesystem, whilst preserving the high performance features of the former. This includes listing and searching for data, creation of files, datasets and containers, and the aggregation of existing data - all within directories with potentially millions of entries. This contribution details the design and implementation of the plugin. Furthermore, an evaluation of the performance characteristics is given, to show that this approach can scale to the requirements of ATLAS physics analysis.
id cern-1606789
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2013
record_format invenio
spelling cern-16067892019-09-30T06:29:59Zdoi:10.1088/1742-6596/513/4/042030http://cds.cern.ch/record/1606789engLassnig, Mvan Dongen, DBrito Da Rocha, RAlvarez Ayllon, ACalfayan, PThe DMLite Rucio Plugin: ATLAS data in a filesystemDetectors and Experimental TechniquesRucio is the next-generation data management system of the ATLAS experiment. Historically, clients interacted with the data management system via specialised tools, but in Rucio additional methods are provided. To support filesystem-like interaction with all ATLAS data, a plugin to the DMLite software stack has been developed. It is possible to mount Rucio as a filesystem, and execute regular filesystem operations in a POSIX fashion. This is exposed via various protocols, for example, WebDAV or NFS, which then removes any dependency on Rucio for client software. The main challenge for this work is the mapping of the set-like ATLAS namespace into a hierarchical filesystem, whilst preserving the high performance features of the former. This includes listing and searching for data, creation of files, datasets and containers, and the aggregation of existing data - all within directories with potentially millions of entries. This contribution details the design and implementation of the plugin. Furthermore, an evaluation of the performance characteristics is given, to show that this approach can scale to the requirements of ATLAS physics analysis.ATL-SOFT-PROC-2013-013oai:cds.cern.ch:16067892013-10-08
spellingShingle Detectors and Experimental Techniques
Lassnig, M
van Dongen, D
Brito Da Rocha, R
Alvarez Ayllon, A
Calfayan, P
The DMLite Rucio Plugin: ATLAS data in a filesystem
title The DMLite Rucio Plugin: ATLAS data in a filesystem
title_full The DMLite Rucio Plugin: ATLAS data in a filesystem
title_fullStr The DMLite Rucio Plugin: ATLAS data in a filesystem
title_full_unstemmed The DMLite Rucio Plugin: ATLAS data in a filesystem
title_short The DMLite Rucio Plugin: ATLAS data in a filesystem
title_sort dmlite rucio plugin: atlas data in a filesystem
topic Detectors and Experimental Techniques
url https://dx.doi.org/10.1088/1742-6596/513/4/042030
http://cds.cern.ch/record/1606789
work_keys_str_mv AT lassnigm thedmliteruciopluginatlasdatainafilesystem
AT vandongend thedmliteruciopluginatlasdatainafilesystem
AT britodarochar thedmliteruciopluginatlasdatainafilesystem
AT alvarezayllona thedmliteruciopluginatlasdatainafilesystem
AT calfayanp thedmliteruciopluginatlasdatainafilesystem
AT lassnigm dmliteruciopluginatlasdatainafilesystem
AT vandongend dmliteruciopluginatlasdatainafilesystem
AT britodarochar dmliteruciopluginatlasdatainafilesystem
AT alvarezayllona dmliteruciopluginatlasdatainafilesystem
AT calfayanp dmliteruciopluginatlasdatainafilesystem