The Rucio Consistency Service

One of the biggest challenge with Large scale data management system is to ensure the consistency between the global file catalog and what is physically on all storage elements. To tackle this issue, the Rucio software which is used by the ATLAS Distributed Data Management system has been extended t...

Descripción completa

Detalles Bibliográficos
Autores principales: Serfon, Cedric, Garonne, Vincent
Lenguaje:eng
Publicado: 2016
Materias:
Particle Physics - Experiment
Acceso en línea:http://cds.cern.ch/record/2219340
_version_ 1780952151078469632
author Serfon, Cedric
Garonne, Vincent
author_facet Serfon, Cedric
Garonne, Vincent
author_sort Serfon, Cedric
collection CERN
description One of the biggest challenge with Large scale data management system is to ensure the consistency between the global file catalog and what is physically on all storage elements. To tackle this issue, the Rucio software which is used by the ATLAS Distributed Data Management system has been extended to automatically handle lost or unregistered files (aka Dark Data). This system automatically detects these inconsistencies and take actions like recovery or deletion of unneeded files in a central manner. In this talk, we will present this system, explain the internals and give some results.
id cern-2219340
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2016
record_format invenio
spelling cern-22193402019-09-30T06:29:59Zhttp://cds.cern.ch/record/2219340engSerfon, CedricGaronne, VincentThe Rucio Consistency ServiceParticle Physics - ExperimentOne of the biggest challenge with Large scale data management system is to ensure the consistency between the global file catalog and what is physically on all storage elements. To tackle this issue, the Rucio software which is used by the ATLAS Distributed Data Management system has been extended to automatically handle lost or unregistered files (aka Dark Data). This system automatically detects these inconsistencies and take actions like recovery or deletion of unneeded files in a central manner. In this talk, we will present this system, explain the internals and give some results.ATL-SOFT-SLIDE-2016-708oai:cds.cern.ch:22193402016-09-26
spellingShingle Particle Physics - Experiment
Serfon, Cedric
Garonne, Vincent
The Rucio Consistency Service
title The Rucio Consistency Service
title_full The Rucio Consistency Service
title_fullStr The Rucio Consistency Service
title_full_unstemmed The Rucio Consistency Service
title_short The Rucio Consistency Service
title_sort rucio consistency service
topic Particle Physics - Experiment
url http://cds.cern.ch/record/2219340
work_keys_str_mv AT serfoncedric therucioconsistencyservice
AT garonnevincent therucioconsistencyservice
AT serfoncedric rucioconsistencyservice
AT garonnevincent rucioconsistencyservice

Ejemplares similares