Cargando…
The DMLite Rucio Plugin: ATLAS data in a filesystem
Rucio is the next-generation data management system of the ATLAS experiment. Historically, clients interacted with the data management system via specialised tools, but in Rucio additional methods are provided. To support filesystem-like interaction with all ATLAS data, a plugin to the DMLite softwa...
Autores principales: | , , , , |
---|---|
Lenguaje: | eng |
Publicado: |
2013
|
Materias: | |
Acceso en línea: | https://dx.doi.org/10.1088/1742-6596/513/4/042030 http://cds.cern.ch/record/1606789 |
_version_ | 1780931693082836992 |
---|---|
author | Lassnig, M van Dongen, D Brito Da Rocha, R Alvarez Ayllon, A Calfayan, P |
author_facet | Lassnig, M van Dongen, D Brito Da Rocha, R Alvarez Ayllon, A Calfayan, P |
author_sort | Lassnig, M |
collection | CERN |
description | Rucio is the next-generation data management system of the ATLAS experiment. Historically, clients interacted with the data management system via specialised tools, but in Rucio additional methods are provided. To support filesystem-like interaction with all ATLAS data, a plugin to the DMLite software stack has been developed. It is possible to mount Rucio as a filesystem, and execute regular filesystem operations in a POSIX fashion. This is exposed via various protocols, for example, WebDAV or NFS, which then removes any dependency on Rucio for client software. The main challenge for this work is the mapping of the set-like ATLAS namespace into a hierarchical filesystem, whilst preserving the high performance features of the former. This includes listing and searching for data, creation of files, datasets and containers, and the aggregation of existing data - all within directories with potentially millions of entries. This contribution details the design and implementation of the plugin. Furthermore, an evaluation of the performance characteristics is given, to show that this approach can scale to the requirements of ATLAS physics analysis. |
id | cern-1606789 |
institution | Organización Europea para la Investigación Nuclear |
language | eng |
publishDate | 2013 |
record_format | invenio |
spelling | cern-16067892019-09-30T06:29:59Zdoi:10.1088/1742-6596/513/4/042030http://cds.cern.ch/record/1606789engLassnig, Mvan Dongen, DBrito Da Rocha, RAlvarez Ayllon, ACalfayan, PThe DMLite Rucio Plugin: ATLAS data in a filesystemDetectors and Experimental TechniquesRucio is the next-generation data management system of the ATLAS experiment. Historically, clients interacted with the data management system via specialised tools, but in Rucio additional methods are provided. To support filesystem-like interaction with all ATLAS data, a plugin to the DMLite software stack has been developed. It is possible to mount Rucio as a filesystem, and execute regular filesystem operations in a POSIX fashion. This is exposed via various protocols, for example, WebDAV or NFS, which then removes any dependency on Rucio for client software. The main challenge for this work is the mapping of the set-like ATLAS namespace into a hierarchical filesystem, whilst preserving the high performance features of the former. This includes listing and searching for data, creation of files, datasets and containers, and the aggregation of existing data - all within directories with potentially millions of entries. This contribution details the design and implementation of the plugin. Furthermore, an evaluation of the performance characteristics is given, to show that this approach can scale to the requirements of ATLAS physics analysis.ATL-SOFT-PROC-2013-013oai:cds.cern.ch:16067892013-10-08 |
spellingShingle | Detectors and Experimental Techniques Lassnig, M van Dongen, D Brito Da Rocha, R Alvarez Ayllon, A Calfayan, P The DMLite Rucio Plugin: ATLAS data in a filesystem |
title | The DMLite Rucio Plugin: ATLAS data in a filesystem |
title_full | The DMLite Rucio Plugin: ATLAS data in a filesystem |
title_fullStr | The DMLite Rucio Plugin: ATLAS data in a filesystem |
title_full_unstemmed | The DMLite Rucio Plugin: ATLAS data in a filesystem |
title_short | The DMLite Rucio Plugin: ATLAS data in a filesystem |
title_sort | dmlite rucio plugin: atlas data in a filesystem |
topic | Detectors and Experimental Techniques |
url | https://dx.doi.org/10.1088/1742-6596/513/4/042030 http://cds.cern.ch/record/1606789 |
work_keys_str_mv | AT lassnigm thedmliteruciopluginatlasdatainafilesystem AT vandongend thedmliteruciopluginatlasdatainafilesystem AT britodarochar thedmliteruciopluginatlasdatainafilesystem AT alvarezayllona thedmliteruciopluginatlasdatainafilesystem AT calfayanp thedmliteruciopluginatlasdatainafilesystem AT lassnigm dmliteruciopluginatlasdatainafilesystem AT vandongend dmliteruciopluginatlasdatainafilesystem AT britodarochar dmliteruciopluginatlasdatainafilesystem AT alvarezayllona dmliteruciopluginatlasdatainafilesystem AT calfayanp dmliteruciopluginatlasdatainafilesystem |