Cargando…

Migrating Engineering Windows HPC applications to Linux HTCondor and Slurm Clusters

The CERN IT department has been maintaining different High Performance Computing (HPC) services over the past five years. While the bulk of computing facilities at CERN are running under Linux, a Windows cluster was dedicated for engineering simulations and analysis related to accelerator technology...

Descripción completa

Detalles Bibliográficos
Autores principales: Alandes Pradillo, Maria, Høimyr, Nils, Sanmillan, Pablo Llopis, Jylhänkangas, Markus Tapani
Lenguaje:eng
Publicado: 2020
Materias:
Acceso en línea:https://dx.doi.org/10.1051/epjconf/202024509016
http://cds.cern.ch/record/2752941
_version_ 1780969328202481664
author Alandes Pradillo, Maria
Høimyr, Nils
Sanmillan, Pablo Llopis
Jylhänkangas, Markus Tapani
author_facet Alandes Pradillo, Maria
Høimyr, Nils
Sanmillan, Pablo Llopis
Jylhänkangas, Markus Tapani
author_sort Alandes Pradillo, Maria
collection CERN
description The CERN IT department has been maintaining different High Performance Computing (HPC) services over the past five years. While the bulk of computing facilities at CERN are running under Linux, a Windows cluster was dedicated for engineering simulations and analysis related to accelerator technology development. The Windows cluster consisted of machines with powerful CPUs, big memory, and a low-latency interconnect. The Linux cluster resources are accessible through HTCondor, and are used for general purpose parallel but single-node type jobs, providing computing power to the CERN experiments and departments for tasks such as physics event reconstruction, data analysis, and simulation. For HPC workloads that require multi-node parallel environments for Message Passing Interface (MPI) based programs, there is another Linux-based HPC service that is comprised of several clusters running under the Slurm batch system, and consist of powerful hardware with low-latency interconnects.In 2018, it was decided to consolidate compute intensive jobs in Linux to make a better use of the existing resources. Moreover, this was also in line with CERN IT strategy to reduce its dependencies on Microsoft products. This paper focuses on the migration of Ansys [1], COMSOL [2] and CST [3] users from Windows HPC to Linux clusters. Ansys, COMSOL and CST are three engineering applications used at CERN for different domains, like multiphysics simulations and electromagnetic field problems. Users of these applications are in different departments, with different needs and levels of expertise. In most cases, the users have no prior knowledge of Linux. The paper will present the technical strategy to allow the engineering users to submit their simulations to the appropriate Linux cluster, depending on their simulation requirements. We also describe the technical solution to integrate their Windows workstations in order from them to be able to submit to Linux clusters. Finally, we discuss the challenges and lessons learnt during the migration.
id oai-inspirehep.net-1832151
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2020
record_format invenio
spelling oai-inspirehep.net-18321512021-03-01T20:16:23Zdoi:10.1051/epjconf/202024509016http://cds.cern.ch/record/2752941engAlandes Pradillo, MariaHøimyr, NilsSanmillan, Pablo LlopisJylhänkangas, Markus TapaniMigrating Engineering Windows HPC applications to Linux HTCondor and Slurm ClustersComputing and ComputersThe CERN IT department has been maintaining different High Performance Computing (HPC) services over the past five years. While the bulk of computing facilities at CERN are running under Linux, a Windows cluster was dedicated for engineering simulations and analysis related to accelerator technology development. The Windows cluster consisted of machines with powerful CPUs, big memory, and a low-latency interconnect. The Linux cluster resources are accessible through HTCondor, and are used for general purpose parallel but single-node type jobs, providing computing power to the CERN experiments and departments for tasks such as physics event reconstruction, data analysis, and simulation. For HPC workloads that require multi-node parallel environments for Message Passing Interface (MPI) based programs, there is another Linux-based HPC service that is comprised of several clusters running under the Slurm batch system, and consist of powerful hardware with low-latency interconnects.In 2018, it was decided to consolidate compute intensive jobs in Linux to make a better use of the existing resources. Moreover, this was also in line with CERN IT strategy to reduce its dependencies on Microsoft products. This paper focuses on the migration of Ansys [1], COMSOL [2] and CST [3] users from Windows HPC to Linux clusters. Ansys, COMSOL and CST are three engineering applications used at CERN for different domains, like multiphysics simulations and electromagnetic field problems. Users of these applications are in different departments, with different needs and levels of expertise. In most cases, the users have no prior knowledge of Linux. The paper will present the technical strategy to allow the engineering users to submit their simulations to the appropriate Linux cluster, depending on their simulation requirements. We also describe the technical solution to integrate their Windows workstations in order from them to be able to submit to Linux clusters. Finally, we discuss the challenges and lessons learnt during the migration.oai:inspirehep.net:18321512020
spellingShingle Computing and Computers
Alandes Pradillo, Maria
Høimyr, Nils
Sanmillan, Pablo Llopis
Jylhänkangas, Markus Tapani
Migrating Engineering Windows HPC applications to Linux HTCondor and Slurm Clusters
title Migrating Engineering Windows HPC applications to Linux HTCondor and Slurm Clusters
title_full Migrating Engineering Windows HPC applications to Linux HTCondor and Slurm Clusters
title_fullStr Migrating Engineering Windows HPC applications to Linux HTCondor and Slurm Clusters
title_full_unstemmed Migrating Engineering Windows HPC applications to Linux HTCondor and Slurm Clusters
title_short Migrating Engineering Windows HPC applications to Linux HTCondor and Slurm Clusters
title_sort migrating engineering windows hpc applications to linux htcondor and slurm clusters
topic Computing and Computers
url https://dx.doi.org/10.1051/epjconf/202024509016
http://cds.cern.ch/record/2752941
work_keys_str_mv AT alandespradillomaria migratingengineeringwindowshpcapplicationstolinuxhtcondorandslurmclusters
AT høimyrnils migratingengineeringwindowshpcapplicationstolinuxhtcondorandslurmclusters
AT sanmillanpablollopis migratingengineeringwindowshpcapplicationstolinuxhtcondorandslurmclusters
AT jylhankangasmarkustapani migratingengineeringwindowshpcapplicationstolinuxhtcondorandslurmclusters