Cargando…

The Platform-as-a-Service paradigm meets ATLAS: developing an automated analysis workflow on the newly established INFN Cloud

The Worldwide LHC Computing Grid (WLCG) is a large-scale collaboration which gathers the computing resources of around 170 computing centres from more than 40 countries. The grid paradigm, unique to the realm of high energy physics, has successfully supported a broad variety of scientific achievemen...

Descripción completa

Detalles Bibliográficos
Autores principales: Marcon, Caterina, Carminati, Leonardo, Rebatto, David
Lenguaje:eng
Publicado: 2023
Materias:
Acceso en línea:http://cds.cern.ch/record/2857278
_version_ 1780977555095945216
author Marcon, Caterina
Carminati, Leonardo
Rebatto, David
author_facet Marcon, Caterina
Carminati, Leonardo
Rebatto, David
author_sort Marcon, Caterina
collection CERN
description The Worldwide LHC Computing Grid (WLCG) is a large-scale collaboration which gathers the computing resources of around 170 computing centres from more than 40 countries. The grid paradigm, unique to the realm of high energy physics, has successfully supported a broad variety of scientific achievements. To fulfil the requirements of new applications and to improve the long-term sustainability of the grid middleware, more versatile solutions are being investigated. Cloud computing is becoming increasingly popular among open-source and commercial players. The HEP community has also recognized the benefits of integrating cloud technologies into the legacy grid-based workflows. Since March 2021, INFN has entered the field of cloud computing establishing the INFN Cloud infrastructure. Large data centers of the INFN National Computing Center, connected to a nation-wide backbone maintained by the GARR Consortium, are gathered into a redundant and federated infrastructure. This cloud service supports scientific computing, software development and training, and serves as an extension of local computing and storage resources. Among available services, INFN Cloud administrators can create virtual machines, Docker-based deployments or Kubernetes clusters. These options allow the creation of customized environments, both for individual users and for scientific collaborations. This study investigates the feasibility of an automated, cloud-based data analysis workflow for the ATLAS experiment using INFN Cloud resources. The concept is designed as a Platform-as-a-Service (PaaS) solution, based on a Centos 7 Docker image. The customized image is responsible for the provisioning of CERN’s CVMFS and EOS shared filesystems, from which a standardized ATLAS environment can be loaded. The end user’s only responsibility is to provide a working application capable of retrieving and analysing data, and to export the results to a persistent storage. The analysis code can be sourced either from remote git repositories or from a local Docker bind mount. As a final step in the automation workflow, a Kubernetes cluster will be configured within the INFN Cloud infrastructure to allow dynamic resource allocation and the interoperability with batch systems, such as HTCondor, will be investigated.
id cern-2857278
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2023
record_format invenio
spelling cern-28572782023-04-30T19:25:45Zhttp://cds.cern.ch/record/2857278engMarcon, CaterinaCarminati, LeonardoRebatto, DavidThe Platform-as-a-Service paradigm meets ATLAS: developing an automated analysis workflow on the newly established INFN CloudParticle Physics - ExperimentThe Worldwide LHC Computing Grid (WLCG) is a large-scale collaboration which gathers the computing resources of around 170 computing centres from more than 40 countries. The grid paradigm, unique to the realm of high energy physics, has successfully supported a broad variety of scientific achievements. To fulfil the requirements of new applications and to improve the long-term sustainability of the grid middleware, more versatile solutions are being investigated. Cloud computing is becoming increasingly popular among open-source and commercial players. The HEP community has also recognized the benefits of integrating cloud technologies into the legacy grid-based workflows. Since March 2021, INFN has entered the field of cloud computing establishing the INFN Cloud infrastructure. Large data centers of the INFN National Computing Center, connected to a nation-wide backbone maintained by the GARR Consortium, are gathered into a redundant and federated infrastructure. This cloud service supports scientific computing, software development and training, and serves as an extension of local computing and storage resources. Among available services, INFN Cloud administrators can create virtual machines, Docker-based deployments or Kubernetes clusters. These options allow the creation of customized environments, both for individual users and for scientific collaborations. This study investigates the feasibility of an automated, cloud-based data analysis workflow for the ATLAS experiment using INFN Cloud resources. The concept is designed as a Platform-as-a-Service (PaaS) solution, based on a Centos 7 Docker image. The customized image is responsible for the provisioning of CERN’s CVMFS and EOS shared filesystems, from which a standardized ATLAS environment can be loaded. The end user’s only responsibility is to provide a working application capable of retrieving and analysing data, and to export the results to a persistent storage. The analysis code can be sourced either from remote git repositories or from a local Docker bind mount. As a final step in the automation workflow, a Kubernetes cluster will be configured within the INFN Cloud infrastructure to allow dynamic resource allocation and the interoperability with batch systems, such as HTCondor, will be investigated.ATL-SOFT-SLIDE-2023-110oai:cds.cern.ch:28572782023-04-29
spellingShingle Particle Physics - Experiment
Marcon, Caterina
Carminati, Leonardo
Rebatto, David
The Platform-as-a-Service paradigm meets ATLAS: developing an automated analysis workflow on the newly established INFN Cloud
title The Platform-as-a-Service paradigm meets ATLAS: developing an automated analysis workflow on the newly established INFN Cloud
title_full The Platform-as-a-Service paradigm meets ATLAS: developing an automated analysis workflow on the newly established INFN Cloud
title_fullStr The Platform-as-a-Service paradigm meets ATLAS: developing an automated analysis workflow on the newly established INFN Cloud
title_full_unstemmed The Platform-as-a-Service paradigm meets ATLAS: developing an automated analysis workflow on the newly established INFN Cloud
title_short The Platform-as-a-Service paradigm meets ATLAS: developing an automated analysis workflow on the newly established INFN Cloud
title_sort platform-as-a-service paradigm meets atlas: developing an automated analysis workflow on the newly established infn cloud
topic Particle Physics - Experiment
url http://cds.cern.ch/record/2857278
work_keys_str_mv AT marconcaterina theplatformasaserviceparadigmmeetsatlasdevelopinganautomatedanalysisworkflowonthenewlyestablishedinfncloud
AT carminatileonardo theplatformasaserviceparadigmmeetsatlasdevelopinganautomatedanalysisworkflowonthenewlyestablishedinfncloud
AT rebattodavid theplatformasaserviceparadigmmeetsatlasdevelopinganautomatedanalysisworkflowonthenewlyestablishedinfncloud
AT marconcaterina platformasaserviceparadigmmeetsatlasdevelopinganautomatedanalysisworkflowonthenewlyestablishedinfncloud
AT carminatileonardo platformasaserviceparadigmmeetsatlasdevelopinganautomatedanalysisworkflowonthenewlyestablishedinfncloud
AT rebattodavid platformasaserviceparadigmmeetsatlasdevelopinganautomatedanalysisworkflowonthenewlyestablishedinfncloud