Cargando…

Evolution of the CMS Submission Infrastructure to support heterogeneous resources in the LHC Run 3

The landscape of computing power available for the CMS experiment is rapidly evolving, from a scenario dominated by x86 processors deployed at WLCG sites towards a more diverse mixture of Grid, HPC, and Cloud facilities, incorporating a higher fraction of non-CPU components such as GPUs. Using these...

Descripción completa

Detalles Bibliográficos
Autores principales: Perez-Calero Yzquierdo, Antonio Maria, Kizinevic, Edita, Khan, Farrukh Aftab, Kim, Hyunwoo, Mascheroni, Marco, Acosta Flechas, Maria, Tsipinakis, Nikos, Haleem, Saqib
Lenguaje:eng
Publicado: 2023
Materias:
Acceso en línea:http://cds.cern.ch/record/2855340
_version_ 1780977447332741120
author Perez-Calero Yzquierdo, Antonio Maria
Kizinevic, Edita
Khan, Farrukh Aftab
Kim, Hyunwoo
Mascheroni, Marco
Acosta Flechas, Maria
Tsipinakis, Nikos
Haleem, Saqib
author_facet Perez-Calero Yzquierdo, Antonio Maria
Kizinevic, Edita
Khan, Farrukh Aftab
Kim, Hyunwoo
Mascheroni, Marco
Acosta Flechas, Maria
Tsipinakis, Nikos
Haleem, Saqib
author_sort Perez-Calero Yzquierdo, Antonio Maria
collection CERN
description The landscape of computing power available for the CMS experiment is rapidly evolving, from a scenario dominated by x86 processors deployed at WLCG sites towards a more diverse mixture of Grid, HPC, and Cloud facilities, incorporating a higher fraction of non-CPU components such as GPUs. Using these facilities heterogeneous resources efficiently to process the vast amounts of data to be collected in the LHC Run3 and beyond, is key to CMS achieving its scientific goals. The Submission Infrastructure is the main component of the resource acquisition and workload to resource matchmaking systems in CMS Offline Computing. It is therefore firstly mandatory to adapt this infrastructure to be able to request and aggregate heterogeneous resources from our providers and to integrate them into the CMS HTCondor infrastructure. Secondly, it is crucial to optimize the matchmaking of CMS workloads to heterogeneous resources, in order for CMS to succeed in effectively exploiting the enormous amount of computing power that is expected to be available in the form of GPUs. This report presents how this has been technically achieved, as well as a brief description of the already existing pool of GPUs ready for CMS use.
id cern-2855340
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2023
record_format invenio
spelling cern-28553402023-04-03T19:01:39Zhttp://cds.cern.ch/record/2855340engPerez-Calero Yzquierdo, Antonio MariaKizinevic, EditaKhan, Farrukh AftabKim, HyunwooMascheroni, MarcoAcosta Flechas, MariaTsipinakis, NikosHaleem, SaqibEvolution of the CMS Submission Infrastructure to support heterogeneous resources in the LHC Run 3Detectors and Experimental TechniquesThe landscape of computing power available for the CMS experiment is rapidly evolving, from a scenario dominated by x86 processors deployed at WLCG sites towards a more diverse mixture of Grid, HPC, and Cloud facilities, incorporating a higher fraction of non-CPU components such as GPUs. Using these facilities heterogeneous resources efficiently to process the vast amounts of data to be collected in the LHC Run3 and beyond, is key to CMS achieving its scientific goals. The Submission Infrastructure is the main component of the resource acquisition and workload to resource matchmaking systems in CMS Offline Computing. It is therefore firstly mandatory to adapt this infrastructure to be able to request and aggregate heterogeneous resources from our providers and to integrate them into the CMS HTCondor infrastructure. Secondly, it is crucial to optimize the matchmaking of CMS workloads to heterogeneous resources, in order for CMS to succeed in effectively exploiting the enormous amount of computing power that is expected to be available in the form of GPUs. This report presents how this has been technically achieved, as well as a brief description of the already existing pool of GPUs ready for CMS use.CMS-CR-2023-035oai:cds.cern.ch:28553402023-03-12
spellingShingle Detectors and Experimental Techniques
Perez-Calero Yzquierdo, Antonio Maria
Kizinevic, Edita
Khan, Farrukh Aftab
Kim, Hyunwoo
Mascheroni, Marco
Acosta Flechas, Maria
Tsipinakis, Nikos
Haleem, Saqib
Evolution of the CMS Submission Infrastructure to support heterogeneous resources in the LHC Run 3
title Evolution of the CMS Submission Infrastructure to support heterogeneous resources in the LHC Run 3
title_full Evolution of the CMS Submission Infrastructure to support heterogeneous resources in the LHC Run 3
title_fullStr Evolution of the CMS Submission Infrastructure to support heterogeneous resources in the LHC Run 3
title_full_unstemmed Evolution of the CMS Submission Infrastructure to support heterogeneous resources in the LHC Run 3
title_short Evolution of the CMS Submission Infrastructure to support heterogeneous resources in the LHC Run 3
title_sort evolution of the cms submission infrastructure to support heterogeneous resources in the lhc run 3
topic Detectors and Experimental Techniques
url http://cds.cern.ch/record/2855340
work_keys_str_mv AT perezcaleroyzquierdoantoniomaria evolutionofthecmssubmissioninfrastructuretosupportheterogeneousresourcesinthelhcrun3
AT kizinevicedita evolutionofthecmssubmissioninfrastructuretosupportheterogeneousresourcesinthelhcrun3
AT khanfarrukhaftab evolutionofthecmssubmissioninfrastructuretosupportheterogeneousresourcesinthelhcrun3
AT kimhyunwoo evolutionofthecmssubmissioninfrastructuretosupportheterogeneousresourcesinthelhcrun3
AT mascheronimarco evolutionofthecmssubmissioninfrastructuretosupportheterogeneousresourcesinthelhcrun3
AT acostaflechasmaria evolutionofthecmssubmissioninfrastructuretosupportheterogeneousresourcesinthelhcrun3
AT tsipinakisnikos evolutionofthecmssubmissioninfrastructuretosupportheterogeneousresourcesinthelhcrun3
AT haleemsaqib evolutionofthecmssubmissioninfrastructuretosupportheterogeneousresourcesinthelhcrun3