Cargando…
Machine Learning for ATLAS DDM Network Metrics
The increasing volume of physics data is posing a critical challenge to the ATLAS experiment. In anticipation of high luminosity physics, automation of everyday data management tasks has become necessary. Previously many of these tasks required human decision-making and operation. Recent advances in...
Autores principales: | , , |
---|---|
Lenguaje: | eng |
Publicado: |
2016
|
Materias: | |
Acceso en línea: | http://cds.cern.ch/record/2220780 |
_version_ | 1780952209305894912 |
---|---|
author | Lassnig, Mario Toler, Wesley Vamosi, Ralf |
author_facet | Lassnig, Mario Toler, Wesley Vamosi, Ralf |
author_sort | Lassnig, Mario |
collection | CERN |
description | The increasing volume of physics data is posing a critical challenge to the ATLAS experiment. In anticipation of high luminosity physics, automation of everyday data management tasks has become necessary. Previously many of these tasks required human decision-making and operation. Recent advances in hardware and software have made it possible to entrust more complicated duties to automated systems using models trained by machine learning algorithms. In this contribution we show results from our ongoing automation efforts. First, we describe our framework for distributed data management and network metrics, automatically extract and aggregate data, train models with various machine learning algorithms, and eventually score the resulting models and parameters. Second, we use these models to forecast metrics relevant for network-aware job scheduling and data brokering. We show the characteristics of the data and evaluate the forecasting accuracy of our models. |
id | cern-2220780 |
institution | Organización Europea para la Investigación Nuclear |
language | eng |
publishDate | 2016 |
record_format | invenio |
spelling | cern-22207802019-09-30T06:29:59Zhttp://cds.cern.ch/record/2220780engLassnig, MarioToler, WesleyVamosi, RalfMachine Learning for ATLAS DDM Network MetricsParticle Physics - ExperimentThe increasing volume of physics data is posing a critical challenge to the ATLAS experiment. In anticipation of high luminosity physics, automation of everyday data management tasks has become necessary. Previously many of these tasks required human decision-making and operation. Recent advances in hardware and software have made it possible to entrust more complicated duties to automated systems using models trained by machine learning algorithms. In this contribution we show results from our ongoing automation efforts. First, we describe our framework for distributed data management and network metrics, automatically extract and aggregate data, train models with various machine learning algorithms, and eventually score the resulting models and parameters. Second, we use these models to forecast metrics relevant for network-aware job scheduling and data brokering. We show the characteristics of the data and evaluate the forecasting accuracy of our models.ATL-SOFT-SLIDE-2016-736oai:cds.cern.ch:22207802016-09-30 |
spellingShingle | Particle Physics - Experiment Lassnig, Mario Toler, Wesley Vamosi, Ralf Machine Learning for ATLAS DDM Network Metrics |
title | Machine Learning for ATLAS DDM Network Metrics |
title_full | Machine Learning for ATLAS DDM Network Metrics |
title_fullStr | Machine Learning for ATLAS DDM Network Metrics |
title_full_unstemmed | Machine Learning for ATLAS DDM Network Metrics |
title_short | Machine Learning for ATLAS DDM Network Metrics |
title_sort | machine learning for atlas ddm network metrics |
topic | Particle Physics - Experiment |
url | http://cds.cern.ch/record/2220780 |
work_keys_str_mv | AT lassnigmario machinelearningforatlasddmnetworkmetrics AT tolerwesley machinelearningforatlasddmnetworkmetrics AT vamosiralf machinelearningforatlasddmnetworkmetrics |