Cargando…
Analysing CMS transfers using Machine Learning techniques
LHC experiments transfer more than 10 PB/week between all grid sites using the FTS transfer service. In particular, CMS manages almost 5 PB/week of FTS transfers with PhEDEx (Physics Experiment Data Export). FTS sends metrics about each transfer (e.g. transfer rate, duration, size) to a central HDF...
Autor principal: | |
---|---|
Lenguaje: | eng |
Publicado: |
2016
|
Materias: | |
Acceso en línea: | http://cds.cern.ch/record/2218016 |
_version_ | 1780952133817860096 |
---|---|
author | Diotalevi, Tommaso |
author_facet | Diotalevi, Tommaso |
author_sort | Diotalevi, Tommaso |
collection | CERN |
description | LHC experiments transfer more than 10 PB/week between all grid sites using the FTS transfer service. In particular, CMS manages almost 5 PB/week of FTS transfers with PhEDEx (Physics Experiment Data Export). FTS sends metrics about each transfer (e.g. transfer rate, duration, size) to a central HDFS storage at CERN. The work done during these three months, here as a Summer Student, involved the usage of ML techniques, using a CMS framework called DCAFPilot, to process this new data and generate predictions of transfer latencies on all links between Grid sites. This analysis will provide, as a future service, the necessary information in order to proactively identify and maybe fix latency issued transfer over the WLCG. |
id | cern-2218016 |
institution | Organización Europea para la Investigación Nuclear |
language | eng |
publishDate | 2016 |
record_format | invenio |
spelling | cern-22180162019-09-30T06:29:59Zhttp://cds.cern.ch/record/2218016engDiotalevi, TommasoAnalysing CMS transfers using Machine Learning techniquesComputing and ComputersLHC experiments transfer more than 10 PB/week between all grid sites using the FTS transfer service. In particular, CMS manages almost 5 PB/week of FTS transfers with PhEDEx (Physics Experiment Data Export). FTS sends metrics about each transfer (e.g. transfer rate, duration, size) to a central HDFS storage at CERN. The work done during these three months, here as a Summer Student, involved the usage of ML techniques, using a CMS framework called DCAFPilot, to process this new data and generate predictions of transfer latencies on all links between Grid sites. This analysis will provide, as a future service, the necessary information in order to proactively identify and maybe fix latency issued transfer over the WLCG.CERN-STUDENTS-Note-2016-243oai:cds.cern.ch:22180162016-09-23 |
spellingShingle | Computing and Computers Diotalevi, Tommaso Analysing CMS transfers using Machine Learning techniques |
title | Analysing CMS transfers using Machine Learning techniques |
title_full | Analysing CMS transfers using Machine Learning techniques |
title_fullStr | Analysing CMS transfers using Machine Learning techniques |
title_full_unstemmed | Analysing CMS transfers using Machine Learning techniques |
title_short | Analysing CMS transfers using Machine Learning techniques |
title_sort | analysing cms transfers using machine learning techniques |
topic | Computing and Computers |
url | http://cds.cern.ch/record/2218016 |
work_keys_str_mv | AT diotalevitommaso analysingcmstransfersusingmachinelearningtechniques |