Cargando…

GRID Storage Optimization in Transparent and User-Friendly Way for LHCb Datasets

The LHCb collaboration is one of the four major experiments at the Large Hadron Collider at CERN. Many petabytes of data are produced by the detectors and Monte-Carlo simulations. The LHCb Grid interware LHCbDIRAC [1] is used to make data available to all collaboration members around the world. The...

Descripción completa

Detalles Bibliográficos
Autores principales: Hushchyn, Mikhail, Ustyuzhanin, Andrey, Charpentier, Philippe, Haen, Christophe
Lenguaje:eng
Publicado: 2017
Materias:
Acceso en línea:https://dx.doi.org/10.1088/1742-6596/898/6/062023
http://cds.cern.ch/record/2268242
_version_ 1780954665894019072
author Hushchyn, Mikhail
Ustyuzhanin, Andrey
Charpentier, Philippe
Haen, Christophe
author_facet Hushchyn, Mikhail
Ustyuzhanin, Andrey
Charpentier, Philippe
Haen, Christophe
author_sort Hushchyn, Mikhail
collection CERN
description The LHCb collaboration is one of the four major experiments at the Large Hadron Collider at CERN. Many petabytes of data are produced by the detectors and Monte-Carlo simulations. The LHCb Grid interware LHCbDIRAC [1] is used to make data available to all collaboration members around the world. The data is replicated to the Grid sites in different locations. However the Grid disk storage is limited and does not allow keeping replicas of each file at all sites. Thus it is essential to optimize number of replicas to achieve a better Grid performance. In this study, we present a new approach of data replication and distribution strategy based on data popularity prediction. The popularity is performed based on the data access history and metadata, and uses machine learning techniques and time series analysis methods.
id cern-2268242
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2017
record_format invenio
spelling cern-22682422023-03-14T17:50:46Zdoi:10.1088/1742-6596/898/6/062023http://cds.cern.ch/record/2268242engHushchyn, MikhailUstyuzhanin, AndreyCharpentier, PhilippeHaen, ChristopheGRID Storage Optimization in Transparent and User-Friendly Way for LHCb Datasetscs.DCComputing and ComputersThe LHCb collaboration is one of the four major experiments at the Large Hadron Collider at CERN. Many petabytes of data are produced by the detectors and Monte-Carlo simulations. The LHCb Grid interware LHCbDIRAC [1] is used to make data available to all collaboration members around the world. The data is replicated to the Grid sites in different locations. However the Grid disk storage is limited and does not allow keeping replicas of each file at all sites. Thus it is essential to optimize number of replicas to achieve a better Grid performance. In this study, we present a new approach of data replication and distribution strategy based on data popularity prediction. The popularity is performed based on the data access history and metadata, and uses machine learning techniques and time series analysis methods.The LHCb collaboration is one of the four major experiments at the Large Hadron Collider at CERN. Many petabytes of data are produced by the detectors and Monte-Carlo simulations. The LHCb Grid interware LHCbDIRAC is used to make data available to all collaboration members around the world. The data is replicated to the Grid sites in different locations. However the Grid disk storage is limited and does not allow keeping replicas of each file at all sites. Thus it is essential to optimize number of replicas to achieve a better Grid performance. In this study, we present a new approach of data replication and distribution strategy based on data popularity prediction. The popularity is performed based on the data access history and metadata, and uses machine learning techniques and time series analysis methods.arXiv:1705.04513oai:cds.cern.ch:22682422017-05-12
spellingShingle cs.DC
Computing and Computers
Hushchyn, Mikhail
Ustyuzhanin, Andrey
Charpentier, Philippe
Haen, Christophe
GRID Storage Optimization in Transparent and User-Friendly Way for LHCb Datasets
title GRID Storage Optimization in Transparent and User-Friendly Way for LHCb Datasets
title_full GRID Storage Optimization in Transparent and User-Friendly Way for LHCb Datasets
title_fullStr GRID Storage Optimization in Transparent and User-Friendly Way for LHCb Datasets
title_full_unstemmed GRID Storage Optimization in Transparent and User-Friendly Way for LHCb Datasets
title_short GRID Storage Optimization in Transparent and User-Friendly Way for LHCb Datasets
title_sort grid storage optimization in transparent and user-friendly way for lhcb datasets
topic cs.DC
Computing and Computers
url https://dx.doi.org/10.1088/1742-6596/898/6/062023
http://cds.cern.ch/record/2268242
work_keys_str_mv AT hushchynmikhail gridstorageoptimizationintransparentanduserfriendlywayforlhcbdatasets
AT ustyuzhaninandrey gridstorageoptimizationintransparentanduserfriendlywayforlhcbdatasets
AT charpentierphilippe gridstorageoptimizationintransparentanduserfriendlywayforlhcbdatasets
AT haenchristophe gridstorageoptimizationintransparentanduserfriendlywayforlhcbdatasets