Cargando…

Non-POSIX File System for LHCb Online Event Handling

LHCb aims to use its O(20000) CPU cores in the high level trigger (HLT) and its 120 TB Online storage system for data reprocessing during LHC shutdown periods. These periods can last a few days for technical maintenance or only a few hours during beam interfill gaps. These jobs run on files which ar...

Descripción completa

Detalles Bibliográficos
Autores principales: Garnier, J C, Neufeld, N, Cherukuwada, S S
Lenguaje:eng
Publicado: 2011
Materias:
Acceso en línea:https://dx.doi.org/10.1109/TNS.2011.2116804
https://dx.doi.org/10.1109/RTC.2010.5750457
http://cds.cern.ch/record/1399723
_version_ 1780923613866622976
author Garnier, J C
Neufeld, N
Cherukuwada, S S
author_facet Garnier, J C
Neufeld, N
Cherukuwada, S S
author_sort Garnier, J C
collection CERN
description LHCb aims to use its O(20000) CPU cores in the high level trigger (HLT) and its 120 TB Online storage system for data reprocessing during LHC shutdown periods. These periods can last a few days for technical maintenance or only a few hours during beam interfill gaps. These jobs run on files which are staged in from tape storage to the local storage buffer. The result are again one or more files. Efficient file writing and reading is essential for the performance of the system. Rather than using a traditional shared file-system such as NFS or CIFS we have implemented a custom, light-weight, non-Posix network file-system for the handling of these files. Streaming this file-system for the data-access allows to obtain high performance, while at the same time keep the resource consumption low and add nice features not found in NFS such as high-availability, transparent fail-over of the read and write service. The writing part of this streaming service is in successful use for the Online, real-time writing of the data during normal data acquisition operation. The network file system relies on a commercial file system which manages the data on persistent disk storage. The implementation together with performance figures are presented.
id cern-1399723
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2011
record_format invenio
spelling cern-13997232019-09-30T06:29:59Zdoi:10.1109/TNS.2011.2116804doi:10.1109/RTC.2010.5750457http://cds.cern.ch/record/1399723engGarnier, J CNeufeld, NCherukuwada, S SNon-POSIX File System for LHCb Online Event HandlingComputing and ComputersLHCb aims to use its O(20000) CPU cores in the high level trigger (HLT) and its 120 TB Online storage system for data reprocessing during LHC shutdown periods. These periods can last a few days for technical maintenance or only a few hours during beam interfill gaps. These jobs run on files which are staged in from tape storage to the local storage buffer. The result are again one or more files. Efficient file writing and reading is essential for the performance of the system. Rather than using a traditional shared file-system such as NFS or CIFS we have implemented a custom, light-weight, non-Posix network file-system for the handling of these files. Streaming this file-system for the data-access allows to obtain high performance, while at the same time keep the resource consumption low and add nice features not found in NFS such as high-availability, transparent fail-over of the read and write service. The writing part of this streaming service is in successful use for the Online, real-time writing of the data during normal data acquisition operation. The network file system relies on a commercial file system which manages the data on persistent disk storage. The implementation together with performance figures are presented.oai:cds.cern.ch:13997232011
spellingShingle Computing and Computers
Garnier, J C
Neufeld, N
Cherukuwada, S S
Non-POSIX File System for LHCb Online Event Handling
title Non-POSIX File System for LHCb Online Event Handling
title_full Non-POSIX File System for LHCb Online Event Handling
title_fullStr Non-POSIX File System for LHCb Online Event Handling
title_full_unstemmed Non-POSIX File System for LHCb Online Event Handling
title_short Non-POSIX File System for LHCb Online Event Handling
title_sort non-posix file system for lhcb online event handling
topic Computing and Computers
url https://dx.doi.org/10.1109/TNS.2011.2116804
https://dx.doi.org/10.1109/RTC.2010.5750457
http://cds.cern.ch/record/1399723
work_keys_str_mv AT garnierjc nonposixfilesystemforlhcbonlineeventhandling
AT neufeldn nonposixfilesystemforlhcbonlineeventhandling
AT cherukuwadass nonposixfilesystemforlhcbonlineeventhandling