Cargando…
Memory handling in the ATLAS submission system from job definition to sites limits
In the past few years the increased luminosity of the LHC, changes in the linux kernel and a move to a 64bit architecture have affected the ATLAS jobs memory usage and the ATLAS workload management system had to be adapted to be more flexible and pass memory parameters to the batch systems, which in...
Autores principales: | , , , , , , |
---|---|
Lenguaje: | eng |
Publicado: |
2017
|
Materias: | |
Acceso en línea: | https://dx.doi.org/10.1088/1742-6596/898/5/052004 http://cds.cern.ch/record/2242868 |
_version_ | 1780953282439544832 |
---|---|
author | Forti, Alessandra Walker, Rodney Maeno, Tadashi Love, Peter Rauschmayr, Nathalie Filipcic, Andrej Di Girolamo, Alessandro |
author_facet | Forti, Alessandra Walker, Rodney Maeno, Tadashi Love, Peter Rauschmayr, Nathalie Filipcic, Andrej Di Girolamo, Alessandro |
author_sort | Forti, Alessandra |
collection | CERN |
description | In the past few years the increased luminosity of the LHC, changes in the linux kernel and a move to a 64bit architecture have affected the ATLAS jobs memory usage and the ATLAS workload management system had to be adapted to be more flexible and pass memory parameters to the batch systems, which in the past wasn’t a necessity. This paper describes the steps required to add the capability to better handle memory requirements, included the review of how each component definition and parametrization of the memory is mapped to the other components, and what changes had to be applied to make the submission chain work. These changes go from the definition of tasks and the way tasks memory requirements are set using scout jobs, through the new memory tool developed to do that, to how these values are used by the submission component of the system and how the jobs are treated by the sites through the CEs, batch systems and ultimately the kernel. |
id | cern-2242868 |
institution | Organización Europea para la Investigación Nuclear |
language | eng |
publishDate | 2017 |
record_format | invenio |
spelling | cern-22428682019-10-15T15:17:06Zdoi:10.1088/1742-6596/898/5/052004http://cds.cern.ch/record/2242868engForti, AlessandraWalker, RodneyMaeno, TadashiLove, PeterRauschmayr, NathalieFilipcic, AndrejDi Girolamo, AlessandroMemory handling in the ATLAS submission system from job definition to sites limitsParticle Physics - ExperimentIn the past few years the increased luminosity of the LHC, changes in the linux kernel and a move to a 64bit architecture have affected the ATLAS jobs memory usage and the ATLAS workload management system had to be adapted to be more flexible and pass memory parameters to the batch systems, which in the past wasn’t a necessity. This paper describes the steps required to add the capability to better handle memory requirements, included the review of how each component definition and parametrization of the memory is mapped to the other components, and what changes had to be applied to make the submission chain work. These changes go from the definition of tasks and the way tasks memory requirements are set using scout jobs, through the new memory tool developed to do that, to how these values are used by the submission component of the system and how the jobs are treated by the sites through the CEs, batch systems and ultimately the kernel.ATL-SOFT-PROC-2017-018oai:cds.cern.ch:22428682017-01-24 |
spellingShingle | Particle Physics - Experiment Forti, Alessandra Walker, Rodney Maeno, Tadashi Love, Peter Rauschmayr, Nathalie Filipcic, Andrej Di Girolamo, Alessandro Memory handling in the ATLAS submission system from job definition to sites limits |
title | Memory handling in the ATLAS submission system from job definition to sites limits |
title_full | Memory handling in the ATLAS submission system from job definition to sites limits |
title_fullStr | Memory handling in the ATLAS submission system from job definition to sites limits |
title_full_unstemmed | Memory handling in the ATLAS submission system from job definition to sites limits |
title_short | Memory handling in the ATLAS submission system from job definition to sites limits |
title_sort | memory handling in the atlas submission system from job definition to sites limits |
topic | Particle Physics - Experiment |
url | https://dx.doi.org/10.1088/1742-6596/898/5/052004 http://cds.cern.ch/record/2242868 |
work_keys_str_mv | AT fortialessandra memoryhandlingintheatlassubmissionsystemfromjobdefinitiontositeslimits AT walkerrodney memoryhandlingintheatlassubmissionsystemfromjobdefinitiontositeslimits AT maenotadashi memoryhandlingintheatlassubmissionsystemfromjobdefinitiontositeslimits AT lovepeter memoryhandlingintheatlassubmissionsystemfromjobdefinitiontositeslimits AT rauschmayrnathalie memoryhandlingintheatlassubmissionsystemfromjobdefinitiontositeslimits AT filipcicandrej memoryhandlingintheatlassubmissionsystemfromjobdefinitiontositeslimits AT digirolamoalessandro memoryhandlingintheatlassubmissionsystemfromjobdefinitiontositeslimits |