Cargando…

Analysing CMS transfers using Machine Learning techniques

LHC experiments transfer more than 10 PB/week between all grid sites using the FTS transfer service. In particular, CMS manages almost 5 PB/week of FTS transfers with PhEDEx (Physics Experiment Data Export). FTS sends metrics about each transfer (e.g. transfer rate, duration, size) to a central HDF...

Descripción completa

Detalles Bibliográficos
Autor principal: Diotalevi, Tommaso
Lenguaje:eng
Publicado: 2016
Materias:
Acceso en línea:http://cds.cern.ch/record/2218016
_version_ 1780952133817860096
author Diotalevi, Tommaso
author_facet Diotalevi, Tommaso
author_sort Diotalevi, Tommaso
collection CERN
description LHC experiments transfer more than 10 PB/week between all grid sites using the FTS transfer service. In particular, CMS manages almost 5 PB/week of FTS transfers with PhEDEx (Physics Experiment Data Export). FTS sends metrics about each transfer (e.g. transfer rate, duration, size) to a central HDFS storage at CERN. The work done during these three months, here as a Summer Student, involved the usage of ML techniques, using a CMS framework called DCAFPilot, to process this new data and generate predictions of transfer latencies on all links between Grid sites. This analysis will provide, as a future service, the necessary information in order to proactively identify and maybe fix latency issued transfer over the WLCG.
id cern-2218016
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2016
record_format invenio
spelling cern-22180162019-09-30T06:29:59Zhttp://cds.cern.ch/record/2218016engDiotalevi, TommasoAnalysing CMS transfers using Machine Learning techniquesComputing and ComputersLHC experiments transfer more than 10 PB/week between all grid sites using the FTS transfer service. In particular, CMS manages almost 5 PB/week of FTS transfers with PhEDEx (Physics Experiment Data Export). FTS sends metrics about each transfer (e.g. transfer rate, duration, size) to a central HDFS storage at CERN. The work done during these three months, here as a Summer Student, involved the usage of ML techniques, using a CMS framework called DCAFPilot, to process this new data and generate predictions of transfer latencies on all links between Grid sites. This analysis will provide, as a future service, the necessary information in order to proactively identify and maybe fix latency issued transfer over the WLCG.CERN-STUDENTS-Note-2016-243oai:cds.cern.ch:22180162016-09-23
spellingShingle Computing and Computers
Diotalevi, Tommaso
Analysing CMS transfers using Machine Learning techniques
title Analysing CMS transfers using Machine Learning techniques
title_full Analysing CMS transfers using Machine Learning techniques
title_fullStr Analysing CMS transfers using Machine Learning techniques
title_full_unstemmed Analysing CMS transfers using Machine Learning techniques
title_short Analysing CMS transfers using Machine Learning techniques
title_sort analysing cms transfers using machine learning techniques
topic Computing and Computers
url http://cds.cern.ch/record/2218016
work_keys_str_mv AT diotalevitommaso analysingcmstransfersusingmachinelearningtechniques