Cargando…

The DIRAC Data Management System and the Gaudi dataset federation

The DIRAC Interware provides a development framework and a complete set of components for building distributed computing systems. The DIRAC Data Management System (DMS) offers all the necessary tools to ensure data handling operations for small and large user communities. It supports transparent acc...

Descripción completa

Detalles Bibliográficos
Autores principales: Haen, Christophe, Charpentier, Philippe, Frank, Markus, Tsaregorodtsev, Andrei
Lenguaje:eng
Publicado: 2015
Materias:
Acceso en línea:https://dx.doi.org/10.1088/1742-6596/664/4/042025
http://cds.cern.ch/record/2134567
_version_ 1780949906197839872
author Haen, Christophe
Charpentier, Philippe
Frank, Markus
Tsaregorodtsev, Andrei
author_facet Haen, Christophe
Charpentier, Philippe
Frank, Markus
Tsaregorodtsev, Andrei
author_sort Haen, Christophe
collection CERN
description The DIRAC Interware provides a development framework and a complete set of components for building distributed computing systems. The DIRAC Data Management System (DMS) offers all the necessary tools to ensure data handling operations for small and large user communities. It supports transparent access to storage resources based on multiple technologies, and is easily expandable. The information on data files and replicas is kept in a File Catalog of which DIRAC offers a powerful and versatile implementation (DFC). Data movement can be performed using third party services including FTS3. Bulk data operations are resilient with respect to failures due to the use of the Request Management System (RMS) that keeps track of ongoing tasks.In this contribution we will present an overview of the DIRAC DMS capabilities and its connection with other DIRAC subsystems such as the Transformation System. This paper also focuses on the DIRAC File Catalog, for which a lot of new developments have been carried out, so that LHCb could migrate its replica catalog from the LCG File Catalog to the DFC. Finally, we will present how LHCb achieves a dataset federation without the need of an extra infrastructure.
id oai-inspirehep.net-1413857
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2015
record_format invenio
spelling oai-inspirehep.net-14138572022-08-10T13:00:53Zdoi:10.1088/1742-6596/664/4/042025http://cds.cern.ch/record/2134567engHaen, ChristopheCharpentier, PhilippeFrank, MarkusTsaregorodtsev, AndreiThe DIRAC Data Management System and the Gaudi dataset federationComputing and ComputersThe DIRAC Interware provides a development framework and a complete set of components for building distributed computing systems. The DIRAC Data Management System (DMS) offers all the necessary tools to ensure data handling operations for small and large user communities. It supports transparent access to storage resources based on multiple technologies, and is easily expandable. The information on data files and replicas is kept in a File Catalog of which DIRAC offers a powerful and versatile implementation (DFC). Data movement can be performed using third party services including FTS3. Bulk data operations are resilient with respect to failures due to the use of the Request Management System (RMS) that keeps track of ongoing tasks.In this contribution we will present an overview of the DIRAC DMS capabilities and its connection with other DIRAC subsystems such as the Transformation System. This paper also focuses on the DIRAC File Catalog, for which a lot of new developments have been carried out, so that LHCb could migrate its replica catalog from the LCG File Catalog to the DFC. Finally, we will present how LHCb achieves a dataset federation without the need of an extra infrastructure.oai:inspirehep.net:14138572015
spellingShingle Computing and Computers
Haen, Christophe
Charpentier, Philippe
Frank, Markus
Tsaregorodtsev, Andrei
The DIRAC Data Management System and the Gaudi dataset federation
title The DIRAC Data Management System and the Gaudi dataset federation
title_full The DIRAC Data Management System and the Gaudi dataset federation
title_fullStr The DIRAC Data Management System and the Gaudi dataset federation
title_full_unstemmed The DIRAC Data Management System and the Gaudi dataset federation
title_short The DIRAC Data Management System and the Gaudi dataset federation
title_sort dirac data management system and the gaudi dataset federation
topic Computing and Computers
url https://dx.doi.org/10.1088/1742-6596/664/4/042025
http://cds.cern.ch/record/2134567
work_keys_str_mv AT haenchristophe thediracdatamanagementsystemandthegaudidatasetfederation
AT charpentierphilippe thediracdatamanagementsystemandthegaudidatasetfederation
AT frankmarkus thediracdatamanagementsystemandthegaudidatasetfederation
AT tsaregorodtsevandrei thediracdatamanagementsystemandthegaudidatasetfederation
AT haenchristophe diracdatamanagementsystemandthegaudidatasetfederation
AT charpentierphilippe diracdatamanagementsystemandthegaudidatasetfederation
AT frankmarkus diracdatamanagementsystemandthegaudidatasetfederation
AT tsaregorodtsevandrei diracdatamanagementsystemandthegaudidatasetfederation