Cargando…
Preparing the Gaudi framework and the DIRAC WMS for multicore job submission
HEP applications need to adapt to the continuously increasing number of cores on modern CPUs. This must be done at different levels: the software must support parallelization, and the scheduling has to differ between multicore and singlecore jobs. The LHCb software framework (GAUDI) provides a paral...
Autores principales: | , |
---|---|
Lenguaje: | eng |
Publicado: |
2014
|
Materias: | |
Acceso en línea: | https://dx.doi.org/10.1088/1742-6596/513/5/052029 http://cds.cern.ch/record/2025719 |
_version_ | 1780947202005270528 |
---|---|
author | Rauschmayr, N Streit, A |
author_facet | Rauschmayr, N Streit, A |
author_sort | Rauschmayr, N |
collection | CERN |
description | HEP applications need to adapt to the continuously increasing number of cores on modern CPUs. This must be done at different levels: the software must support parallelization, and the scheduling has to differ between multicore and singlecore jobs. The LHCb software framework (GAUDI) provides a parallel prototype (GaudiMP), based on the multiprocessing approach. It allows a reduction of the overall memory footprint and a coordinated access to data via separated reader and writer processes. A comparison between the parallel prototype and multiple independent Gaudi jobs in respect of CPU time and memory consumption will be shown. Furthermore, speedup must be predicted in order to find the limit beyond which the parallel prototype (GaudiMP) does not bring further scaling. This number must be known as it indicates the point, where new technologies must be introduced into the software framework. In order to reach further improvements in the overall throughput, scheduling strategies for mixing parallel jobs can be applied. It allows overcoming limitations in the speedup of the parallel prototype. Those changes require modifications at the level of the Workload Management System (DIRAC). |
id | oai-inspirehep.net-1302153 |
institution | Organización Europea para la Investigación Nuclear |
language | eng |
publishDate | 2014 |
record_format | invenio |
spelling | oai-inspirehep.net-13021532022-08-17T13:27:43Zdoi:10.1088/1742-6596/513/5/052029http://cds.cern.ch/record/2025719engRauschmayr, NStreit, APreparing the Gaudi framework and the DIRAC WMS for multicore job submissionComputing and ComputersHEP applications need to adapt to the continuously increasing number of cores on modern CPUs. This must be done at different levels: the software must support parallelization, and the scheduling has to differ between multicore and singlecore jobs. The LHCb software framework (GAUDI) provides a parallel prototype (GaudiMP), based on the multiprocessing approach. It allows a reduction of the overall memory footprint and a coordinated access to data via separated reader and writer processes. A comparison between the parallel prototype and multiple independent Gaudi jobs in respect of CPU time and memory consumption will be shown. Furthermore, speedup must be predicted in order to find the limit beyond which the parallel prototype (GaudiMP) does not bring further scaling. This number must be known as it indicates the point, where new technologies must be introduced into the software framework. In order to reach further improvements in the overall throughput, scheduling strategies for mixing parallel jobs can be applied. It allows overcoming limitations in the speedup of the parallel prototype. Those changes require modifications at the level of the Workload Management System (DIRAC).oai:inspirehep.net:13021532014 |
spellingShingle | Computing and Computers Rauschmayr, N Streit, A Preparing the Gaudi framework and the DIRAC WMS for multicore job submission |
title | Preparing the Gaudi framework and the DIRAC WMS for multicore job submission |
title_full | Preparing the Gaudi framework and the DIRAC WMS for multicore job submission |
title_fullStr | Preparing the Gaudi framework and the DIRAC WMS for multicore job submission |
title_full_unstemmed | Preparing the Gaudi framework and the DIRAC WMS for multicore job submission |
title_short | Preparing the Gaudi framework and the DIRAC WMS for multicore job submission |
title_sort | preparing the gaudi framework and the dirac wms for multicore job submission |
topic | Computing and Computers |
url | https://dx.doi.org/10.1088/1742-6596/513/5/052029 http://cds.cern.ch/record/2025719 |
work_keys_str_mv | AT rauschmayrn preparingthegaudiframeworkandthediracwmsformulticorejobsubmission AT streita preparingthegaudiframeworkandthediracwmsformulticorejobsubmission |