Cargando…

Machine Learning for ATLAS DDM Network Metrics

The increasing volume of physics data is posing a critical challenge to the ATLAS experiment. In anticipation of high luminosity physics, automation of everyday data management tasks has become necessary. Previously many of these tasks required human decision-making and operation. Recent advances in...

Descripción completa

Detalles Bibliográficos
Autores principales: Lassnig, Mario, Toler, Wesley, Vamosi, Ralf
Lenguaje:eng
Publicado: 2016
Materias:
Acceso en línea:http://cds.cern.ch/record/2220780
_version_ 1780952209305894912
author Lassnig, Mario
Toler, Wesley
Vamosi, Ralf
author_facet Lassnig, Mario
Toler, Wesley
Vamosi, Ralf
author_sort Lassnig, Mario
collection CERN
description The increasing volume of physics data is posing a critical challenge to the ATLAS experiment. In anticipation of high luminosity physics, automation of everyday data management tasks has become necessary. Previously many of these tasks required human decision-making and operation. Recent advances in hardware and software have made it possible to entrust more complicated duties to automated systems using models trained by machine learning algorithms. In this contribution we show results from our ongoing automation efforts. First, we describe our framework for distributed data management and network metrics, automatically extract and aggregate data, train models with various machine learning algorithms, and eventually score the resulting models and parameters. Second, we use these models to forecast metrics relevant for network-aware job scheduling and data brokering. We show the characteristics of the data and evaluate the forecasting accuracy of our models.
id cern-2220780
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2016
record_format invenio
spelling cern-22207802019-09-30T06:29:59Zhttp://cds.cern.ch/record/2220780engLassnig, MarioToler, WesleyVamosi, RalfMachine Learning for ATLAS DDM Network MetricsParticle Physics - ExperimentThe increasing volume of physics data is posing a critical challenge to the ATLAS experiment. In anticipation of high luminosity physics, automation of everyday data management tasks has become necessary. Previously many of these tasks required human decision-making and operation. Recent advances in hardware and software have made it possible to entrust more complicated duties to automated systems using models trained by machine learning algorithms. In this contribution we show results from our ongoing automation efforts. First, we describe our framework for distributed data management and network metrics, automatically extract and aggregate data, train models with various machine learning algorithms, and eventually score the resulting models and parameters. Second, we use these models to forecast metrics relevant for network-aware job scheduling and data brokering. We show the characteristics of the data and evaluate the forecasting accuracy of our models.ATL-SOFT-SLIDE-2016-736oai:cds.cern.ch:22207802016-09-30
spellingShingle Particle Physics - Experiment
Lassnig, Mario
Toler, Wesley
Vamosi, Ralf
Machine Learning for ATLAS DDM Network Metrics
title Machine Learning for ATLAS DDM Network Metrics
title_full Machine Learning for ATLAS DDM Network Metrics
title_fullStr Machine Learning for ATLAS DDM Network Metrics
title_full_unstemmed Machine Learning for ATLAS DDM Network Metrics
title_short Machine Learning for ATLAS DDM Network Metrics
title_sort machine learning for atlas ddm network metrics
topic Particle Physics - Experiment
url http://cds.cern.ch/record/2220780
work_keys_str_mv AT lassnigmario machinelearningforatlasddmnetworkmetrics
AT tolerwesley machinelearningforatlasddmnetworkmetrics
AT vamosiralf machinelearningforatlasddmnetworkmetrics