Cargando…

Containers for Machine Learning in HEP

<!--HTML-->Physicists want to use modern open source machine learning tools developed by industry for machine learning projects and analyses in high energy physics. The software environment that a physicist prototypes, tests, and runs these projects in is ideally the same regardless of compute...

Descripción completa

Detalles Bibliográficos
Autor principal: Feickert, Matthew
Lenguaje:eng
Publicado: 2019
Materias:
Acceso en línea:http://cds.cern.ch/record/2672019
_version_ 1780962437042798592
author Feickert, Matthew
author_facet Feickert, Matthew
author_sort Feickert, Matthew
collection CERN
description <!--HTML-->Physicists want to use modern open source machine learning tools developed by industry for machine learning projects and analyses in high energy physics. The software environment that a physicist prototypes, tests, and runs these projects in is ideally the same regardless of compute site (be it their laptop or on the GRID). However, historically it has been difficult to find compute sites that have both the desired hardware resources for machine learning (i.e. GPUs) and a compatible software environment for the project, resulting in suboptimal use of resources and wasted researcher time tuning their software requirements to the imposed constraints. Container technologies, such as [Docker](https://www.docker.com/) and [Singularity](https://www.sylabs.io/singularity/), provide a scalable and robust solution to this problem. We present work by _Heinrich_ demonstrating the use of containers to run analysis jobs in reproducible compute environments at GRID endpoints with GPU resources that support Singularity. We additionally present complimentary work by _Feickert_ that provides publicly available "base" Docker images of a HEP orientated machine learning environment: the CentOS 7 file system with the ATLAS "standalone" analysis release AnalysisBase, HDF5 support and utilities, and modern Python 3 with pip with libraries such as [NumPy](https://github.com/numpy/numpy), [TensorFlow](https://github.com/tensorflow/tensorflow) and [uproot](https://github.com/scikit-hep/uproot) installed. We further present ongoing synergetic work to expand both these efforts.
id cern-2672019
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2019
record_format invenio
spelling cern-26720192022-11-02T22:33:38Zhttp://cds.cern.ch/record/2672019engFeickert, MatthewContainers for Machine Learning in HEP3rd IML Machine Learning WorkshopLPCC Workshops<!--HTML-->Physicists want to use modern open source machine learning tools developed by industry for machine learning projects and analyses in high energy physics. The software environment that a physicist prototypes, tests, and runs these projects in is ideally the same regardless of compute site (be it their laptop or on the GRID). However, historically it has been difficult to find compute sites that have both the desired hardware resources for machine learning (i.e. GPUs) and a compatible software environment for the project, resulting in suboptimal use of resources and wasted researcher time tuning their software requirements to the imposed constraints. Container technologies, such as [Docker](https://www.docker.com/) and [Singularity](https://www.sylabs.io/singularity/), provide a scalable and robust solution to this problem. We present work by _Heinrich_ demonstrating the use of containers to run analysis jobs in reproducible compute environments at GRID endpoints with GPU resources that support Singularity. We additionally present complimentary work by _Feickert_ that provides publicly available "base" Docker images of a HEP orientated machine learning environment: the CentOS 7 file system with the ATLAS "standalone" analysis release AnalysisBase, HDF5 support and utilities, and modern Python 3 with pip with libraries such as [NumPy](https://github.com/numpy/numpy), [TensorFlow](https://github.com/tensorflow/tensorflow) and [uproot](https://github.com/scikit-hep/uproot) installed. We further present ongoing synergetic work to expand both these efforts.oai:cds.cern.ch:26720192019
spellingShingle LPCC Workshops
Feickert, Matthew
Containers for Machine Learning in HEP
title Containers for Machine Learning in HEP
title_full Containers for Machine Learning in HEP
title_fullStr Containers for Machine Learning in HEP
title_full_unstemmed Containers for Machine Learning in HEP
title_short Containers for Machine Learning in HEP
title_sort containers for machine learning in hep
topic LPCC Workshops
url http://cds.cern.ch/record/2672019
work_keys_str_mv AT feickertmatthew containersformachinelearninginhep
AT feickertmatthew 3rdimlmachinelearningworkshop