Cargando…
Memory handling in the ATLAS submission system from job definition to sites limits
The ATLAS workload management system is a pilot system based on a late binding philosophy that avoided for many years to pass fine grained job requirements to the batch system. In particular for memory most of the requirements were set to request 4GB vmem as defined in the EGI portal VO card, i.e. 2...
Autor principal: | |
---|---|
Lenguaje: | eng |
Publicado: |
2016
|
Materias: | |
Acceso en línea: | http://cds.cern.ch/record/2220141 |
_version_ | 1780952182413066240 |
---|---|
author | Forti, Alessandra |
author_facet | Forti, Alessandra |
author_sort | Forti, Alessandra |
collection | CERN |
description | The ATLAS workload management system is a pilot system based on a late binding philosophy that avoided for many years to pass fine grained job requirements to the batch system. In particular for memory most of the requirements were set to request 4GB vmem as defined in the EGI portal VO card, i.e. 2GB RAM + 2GB swap. However in the past few years several changes have happened in the operating system kernel and in the applications that make such a definition of memory to use for requesting slots obsolete and ATLAS has introduced the new PRODSYS2 workload management which has a more flexible system to evaluate the memory requirements and to submit to appropriate queues. The work stemmed in particular from the introduction of 64bit multicore workloads and the increased memory requirements of some of the single core applications. This paper describes the overall review and changes of memory handling starting from the definition of tasks, the way tasks memory requirements are set using scout jobs and the new memory tool produced in the process to do that, how the factories set these values and finally how the jobs are treated by the sites through the CEs, batch systems and ultimately the kernel. |
id | cern-2220141 |
institution | Organización Europea para la Investigación Nuclear |
language | eng |
publishDate | 2016 |
record_format | invenio |
spelling | cern-22201412019-09-30T06:29:59Zhttp://cds.cern.ch/record/2220141engForti, AlessandraMemory handling in the ATLAS submission system from job definition to sites limitsParticle Physics - ExperimentThe ATLAS workload management system is a pilot system based on a late binding philosophy that avoided for many years to pass fine grained job requirements to the batch system. In particular for memory most of the requirements were set to request 4GB vmem as defined in the EGI portal VO card, i.e. 2GB RAM + 2GB swap. However in the past few years several changes have happened in the operating system kernel and in the applications that make such a definition of memory to use for requesting slots obsolete and ATLAS has introduced the new PRODSYS2 workload management which has a more flexible system to evaluate the memory requirements and to submit to appropriate queues. The work stemmed in particular from the introduction of 64bit multicore workloads and the increased memory requirements of some of the single core applications. This paper describes the overall review and changes of memory handling starting from the definition of tasks, the way tasks memory requirements are set using scout jobs and the new memory tool produced in the process to do that, how the factories set these values and finally how the jobs are treated by the sites through the CEs, batch systems and ultimately the kernel.ATL-SOFT-SLIDE-2016-720oai:cds.cern.ch:22201412016-09-28 |
spellingShingle | Particle Physics - Experiment Forti, Alessandra Memory handling in the ATLAS submission system from job definition to sites limits |
title | Memory handling in the ATLAS submission system from job definition to sites limits |
title_full | Memory handling in the ATLAS submission system from job definition to sites limits |
title_fullStr | Memory handling in the ATLAS submission system from job definition to sites limits |
title_full_unstemmed | Memory handling in the ATLAS submission system from job definition to sites limits |
title_short | Memory handling in the ATLAS submission system from job definition to sites limits |
title_sort | memory handling in the atlas submission system from job definition to sites limits |
topic | Particle Physics - Experiment |
url | http://cds.cern.ch/record/2220141 |
work_keys_str_mv | AT fortialessandra memoryhandlingintheatlassubmissionsystemfromjobdefinitiontositeslimits |