Cargando…

Preparing the Gaudi framework and the DIRAC WMS for multicore job submission

HEP applications need to adapt to the continuously increasing number of cores on modern CPUs. This must be done at different levels: the software must support parallelization, and the scheduling has to differ between multicore and singlecore jobs. The LHCb software framework (GAUDI) provides a paral...

Descripción completa

Detalles Bibliográficos
Autores principales: Rauschmayr, N, Streit, A
Lenguaje:eng
Publicado: 2014
Materias:
Acceso en línea:https://dx.doi.org/10.1088/1742-6596/513/5/052029
http://cds.cern.ch/record/2025719
_version_ 1780947202005270528
author Rauschmayr, N
Streit, A
author_facet Rauschmayr, N
Streit, A
author_sort Rauschmayr, N
collection CERN
description HEP applications need to adapt to the continuously increasing number of cores on modern CPUs. This must be done at different levels: the software must support parallelization, and the scheduling has to differ between multicore and singlecore jobs. The LHCb software framework (GAUDI) provides a parallel prototype (GaudiMP), based on the multiprocessing approach. It allows a reduction of the overall memory footprint and a coordinated access to data via separated reader and writer processes. A comparison between the parallel prototype and multiple independent Gaudi jobs in respect of CPU time and memory consumption will be shown. Furthermore, speedup must be predicted in order to find the limit beyond which the parallel prototype (GaudiMP) does not bring further scaling. This number must be known as it indicates the point, where new technologies must be introduced into the software framework. In order to reach further improvements in the overall throughput, scheduling strategies for mixing parallel jobs can be applied. It allows overcoming limitations in the speedup of the parallel prototype. Those changes require modifications at the level of the Workload Management System (DIRAC).
id oai-inspirehep.net-1302153
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2014
record_format invenio
spelling oai-inspirehep.net-13021532022-08-17T13:27:43Zdoi:10.1088/1742-6596/513/5/052029http://cds.cern.ch/record/2025719engRauschmayr, NStreit, APreparing the Gaudi framework and the DIRAC WMS for multicore job submissionComputing and ComputersHEP applications need to adapt to the continuously increasing number of cores on modern CPUs. This must be done at different levels: the software must support parallelization, and the scheduling has to differ between multicore and singlecore jobs. The LHCb software framework (GAUDI) provides a parallel prototype (GaudiMP), based on the multiprocessing approach. It allows a reduction of the overall memory footprint and a coordinated access to data via separated reader and writer processes. A comparison between the parallel prototype and multiple independent Gaudi jobs in respect of CPU time and memory consumption will be shown. Furthermore, speedup must be predicted in order to find the limit beyond which the parallel prototype (GaudiMP) does not bring further scaling. This number must be known as it indicates the point, where new technologies must be introduced into the software framework. In order to reach further improvements in the overall throughput, scheduling strategies for mixing parallel jobs can be applied. It allows overcoming limitations in the speedup of the parallel prototype. Those changes require modifications at the level of the Workload Management System (DIRAC).oai:inspirehep.net:13021532014
spellingShingle Computing and Computers
Rauschmayr, N
Streit, A
Preparing the Gaudi framework and the DIRAC WMS for multicore job submission
title Preparing the Gaudi framework and the DIRAC WMS for multicore job submission
title_full Preparing the Gaudi framework and the DIRAC WMS for multicore job submission
title_fullStr Preparing the Gaudi framework and the DIRAC WMS for multicore job submission
title_full_unstemmed Preparing the Gaudi framework and the DIRAC WMS for multicore job submission
title_short Preparing the Gaudi framework and the DIRAC WMS for multicore job submission
title_sort preparing the gaudi framework and the dirac wms for multicore job submission
topic Computing and Computers
url https://dx.doi.org/10.1088/1742-6596/513/5/052029
http://cds.cern.ch/record/2025719
work_keys_str_mv AT rauschmayrn preparingthegaudiframeworkandthediracwmsformulticorejobsubmission
AT streita preparingthegaudiframeworkandthediracwmsformulticorejobsubmission