Cargando…

Machine learning of network metrics in ATLAS Distributed Data Management

The increasing volume of physics data poses a critical challenge to the ATLAS experiment. In anticipation of high luminosity physics, automation of everyday data management tasks has become necessary. Previously many of these tasks required human decision-making and operation. Recent advances in har...

Descripción completa

Detalles Bibliográficos
Autores principales: Lassnig, Mario, Toler, Wesley, Vamosi, Ralf, Bogado Garcia, Joaquin Ignacio
Lenguaje:eng
Publicado: 2017
Materias:
Acceso en línea:https://dx.doi.org/10.1088/1742-6596/898/6/062009
http://cds.cern.ch/record/2243136
_version_ 1780953297589370880
author Lassnig, Mario
Toler, Wesley
Vamosi, Ralf
Bogado Garcia, Joaquin Ignacio
author_facet Lassnig, Mario
Toler, Wesley
Vamosi, Ralf
Bogado Garcia, Joaquin Ignacio
author_sort Lassnig, Mario
collection CERN
description The increasing volume of physics data poses a critical challenge to the ATLAS experiment. In anticipation of high luminosity physics, automation of everyday data management tasks has become necessary. Previously many of these tasks required human decision-making and operation. Recent advances in hardware and software have made it possible to entrust more complicated duties to automated systems using models trained by machine learning algorithms. In this contribution we show results from one of our ongoing automation efforts that focuses on network metrics. First, we describe our machine learning framework built atop the ATLAS Analytics Platform. This framework can automatically extract and aggregate data, train models with various machine learning algorithms, and eventually score the resulting models and parameters. Second, we use these models to forecast metrics relevant for network-aware job scheduling and data brokering. We show the characteristics of the data and evaluate the forecasting accuracy of our models.
id cern-2243136
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2017
record_format invenio
spelling cern-22431362019-10-15T15:17:23Zdoi:10.1088/1742-6596/898/6/062009http://cds.cern.ch/record/2243136engLassnig, MarioToler, WesleyVamosi, RalfBogado Garcia, Joaquin IgnacioMachine learning of network metrics in ATLAS Distributed Data ManagementParticle Physics - ExperimentThe increasing volume of physics data poses a critical challenge to the ATLAS experiment. In anticipation of high luminosity physics, automation of everyday data management tasks has become necessary. Previously many of these tasks required human decision-making and operation. Recent advances in hardware and software have made it possible to entrust more complicated duties to automated systems using models trained by machine learning algorithms. In this contribution we show results from one of our ongoing automation efforts that focuses on network metrics. First, we describe our machine learning framework built atop the ATLAS Analytics Platform. This framework can automatically extract and aggregate data, train models with various machine learning algorithms, and eventually score the resulting models and parameters. Second, we use these models to forecast metrics relevant for network-aware job scheduling and data brokering. We show the characteristics of the data and evaluate the forecasting accuracy of our models.ATL-SOFT-PROC-2017-027oai:cds.cern.ch:22431362017-01-27
spellingShingle Particle Physics - Experiment
Lassnig, Mario
Toler, Wesley
Vamosi, Ralf
Bogado Garcia, Joaquin Ignacio
Machine learning of network metrics in ATLAS Distributed Data Management
title Machine learning of network metrics in ATLAS Distributed Data Management
title_full Machine learning of network metrics in ATLAS Distributed Data Management
title_fullStr Machine learning of network metrics in ATLAS Distributed Data Management
title_full_unstemmed Machine learning of network metrics in ATLAS Distributed Data Management
title_short Machine learning of network metrics in ATLAS Distributed Data Management
title_sort machine learning of network metrics in atlas distributed data management
topic Particle Physics - Experiment
url https://dx.doi.org/10.1088/1742-6596/898/6/062009
http://cds.cern.ch/record/2243136
work_keys_str_mv AT lassnigmario machinelearningofnetworkmetricsinatlasdistributeddatamanagement
AT tolerwesley machinelearningofnetworkmetricsinatlasdistributeddatamanagement
AT vamosiralf machinelearningofnetworkmetricsinatlasdistributeddatamanagement
AT bogadogarciajoaquinignacio machinelearningofnetworkmetricsinatlasdistributeddatamanagement