Cargando…
Performance and impact of dynamic data placement in ATLAS
For high-throughput computing the efficient use of distributed computing resources relies on an evenly distributed workload, which in turn requires wide availability of input data that is used in physics analysis. In ATLAS, the dynamic data placement agent C3PO was implemented in the ATLAS distribut...
Autores principales: | , , , , , |
---|---|
Lenguaje: | eng |
Publicado: |
2018
|
Materias: | |
Acceso en línea: | http://cds.cern.ch/record/2625220 |
_version_ | 1780958783664553984 |
---|---|
author | Maier, Thomas Beermann, Thomas Alfons Vukotic, Ilija Legger, Federica Lassnig, Mario Duckeck, Guenter |
author_facet | Maier, Thomas Beermann, Thomas Alfons Vukotic, Ilija Legger, Federica Lassnig, Mario Duckeck, Guenter |
author_sort | Maier, Thomas |
collection | CERN |
description | For high-throughput computing the efficient use of distributed computing resources relies on an evenly distributed workload, which in turn requires wide availability of input data that is used in physics analysis. In ATLAS, the dynamic data placement agent C3PO was implemented in the ATLAS distributed data management system Rucio which identifies popular data and creates additional, transient replicas to make data more widely and more reliably available. This contribution presents studies on the performance of C3PO and the impact it has on throughput rates of distributed computing in ATLAS. This includes analysis of the placement algorithm selection behaviour regarding the data considered for replication and destination storage elements, usage after the placement decision of the chosen datasets in general and the newly created copies in particular, and the impact on metrics like job waiting times, task completion times and failure rates of tasks. |
id | cern-2625220 |
institution | Organización Europea para la Investigación Nuclear |
language | eng |
publishDate | 2018 |
record_format | invenio |
spelling | cern-26252202019-11-12T11:56:33Zhttp://cds.cern.ch/record/2625220engMaier, ThomasBeermann, Thomas AlfonsVukotic, IlijaLegger, FedericaLassnig, MarioDuckeck, GuenterPerformance and impact of dynamic data placement in ATLASParticle Physics - ExperimentFor high-throughput computing the efficient use of distributed computing resources relies on an evenly distributed workload, which in turn requires wide availability of input data that is used in physics analysis. In ATLAS, the dynamic data placement agent C3PO was implemented in the ATLAS distributed data management system Rucio which identifies popular data and creates additional, transient replicas to make data more widely and more reliably available. This contribution presents studies on the performance of C3PO and the impact it has on throughput rates of distributed computing in ATLAS. This includes analysis of the placement algorithm selection behaviour regarding the data considered for replication and destination storage elements, usage after the placement decision of the chosen datasets in general and the newly created copies in particular, and the impact on metrics like job waiting times, task completion times and failure rates of tasks.ATL-SOFT-SLIDE-2018-393oai:cds.cern.ch:26252202018-06-21 |
spellingShingle | Particle Physics - Experiment Maier, Thomas Beermann, Thomas Alfons Vukotic, Ilija Legger, Federica Lassnig, Mario Duckeck, Guenter Performance and impact of dynamic data placement in ATLAS |
title | Performance and impact of dynamic data placement in ATLAS |
title_full | Performance and impact of dynamic data placement in ATLAS |
title_fullStr | Performance and impact of dynamic data placement in ATLAS |
title_full_unstemmed | Performance and impact of dynamic data placement in ATLAS |
title_short | Performance and impact of dynamic data placement in ATLAS |
title_sort | performance and impact of dynamic data placement in atlas |
topic | Particle Physics - Experiment |
url | http://cds.cern.ch/record/2625220 |
work_keys_str_mv | AT maierthomas performanceandimpactofdynamicdataplacementinatlas AT beermannthomasalfons performanceandimpactofdynamicdataplacementinatlas AT vukoticilija performanceandimpactofdynamicdataplacementinatlas AT leggerfederica performanceandimpactofdynamicdataplacementinatlas AT lassnigmario performanceandimpactofdynamicdataplacementinatlas AT duckeckguenter performanceandimpactofdynamicdataplacementinatlas |