Cargando…
Machine learning of network metrics in ATLAS Distributed Data Management
The increasing volume of physics data poses a critical challenge to the ATLAS experiment. In anticipation of high luminosity physics, automation of everyday data management tasks has become necessary. Previously many of these tasks required human decision-making and operation. Recent advances in har...
Autores principales: | , , , |
---|---|
Lenguaje: | eng |
Publicado: |
2017
|
Materias: | |
Acceso en línea: | https://dx.doi.org/10.1088/1742-6596/898/6/062009 http://cds.cern.ch/record/2243136 |
_version_ | 1780953297589370880 |
---|---|
author | Lassnig, Mario Toler, Wesley Vamosi, Ralf Bogado Garcia, Joaquin Ignacio |
author_facet | Lassnig, Mario Toler, Wesley Vamosi, Ralf Bogado Garcia, Joaquin Ignacio |
author_sort | Lassnig, Mario |
collection | CERN |
description | The increasing volume of physics data poses a critical challenge to the ATLAS experiment. In anticipation of high luminosity physics, automation of everyday data management tasks has become necessary. Previously many of these tasks required human decision-making and operation. Recent advances in hardware and software have made it possible to entrust more complicated duties to automated systems using models trained by machine learning algorithms. In this contribution we show results from one of our ongoing automation efforts that focuses on network metrics. First, we describe our machine learning framework built atop the ATLAS Analytics Platform. This framework can automatically extract and aggregate data, train models with various machine learning algorithms, and eventually score the resulting models and parameters. Second, we use these models to forecast metrics relevant for network-aware job scheduling and data brokering. We show the characteristics of the data and evaluate the forecasting accuracy of our models. |
id | cern-2243136 |
institution | Organización Europea para la Investigación Nuclear |
language | eng |
publishDate | 2017 |
record_format | invenio |
spelling | cern-22431362019-10-15T15:17:23Zdoi:10.1088/1742-6596/898/6/062009http://cds.cern.ch/record/2243136engLassnig, MarioToler, WesleyVamosi, RalfBogado Garcia, Joaquin IgnacioMachine learning of network metrics in ATLAS Distributed Data ManagementParticle Physics - ExperimentThe increasing volume of physics data poses a critical challenge to the ATLAS experiment. In anticipation of high luminosity physics, automation of everyday data management tasks has become necessary. Previously many of these tasks required human decision-making and operation. Recent advances in hardware and software have made it possible to entrust more complicated duties to automated systems using models trained by machine learning algorithms. In this contribution we show results from one of our ongoing automation efforts that focuses on network metrics. First, we describe our machine learning framework built atop the ATLAS Analytics Platform. This framework can automatically extract and aggregate data, train models with various machine learning algorithms, and eventually score the resulting models and parameters. Second, we use these models to forecast metrics relevant for network-aware job scheduling and data brokering. We show the characteristics of the data and evaluate the forecasting accuracy of our models.ATL-SOFT-PROC-2017-027oai:cds.cern.ch:22431362017-01-27 |
spellingShingle | Particle Physics - Experiment Lassnig, Mario Toler, Wesley Vamosi, Ralf Bogado Garcia, Joaquin Ignacio Machine learning of network metrics in ATLAS Distributed Data Management |
title | Machine learning of network metrics in ATLAS Distributed Data Management |
title_full | Machine learning of network metrics in ATLAS Distributed Data Management |
title_fullStr | Machine learning of network metrics in ATLAS Distributed Data Management |
title_full_unstemmed | Machine learning of network metrics in ATLAS Distributed Data Management |
title_short | Machine learning of network metrics in ATLAS Distributed Data Management |
title_sort | machine learning of network metrics in atlas distributed data management |
topic | Particle Physics - Experiment |
url | https://dx.doi.org/10.1088/1742-6596/898/6/062009 http://cds.cern.ch/record/2243136 |
work_keys_str_mv | AT lassnigmario machinelearningofnetworkmetricsinatlasdistributeddatamanagement AT tolerwesley machinelearningofnetworkmetricsinatlasdistributeddatamanagement AT vamosiralf machinelearningofnetworkmetricsinatlasdistributeddatamanagement AT bogadogarciajoaquinignacio machinelearningofnetworkmetricsinatlasdistributeddatamanagement |