Cargando…

Feasibility tests of RoCE for the cluster-based event building in LHCb

This paper evaluates the utilization of RDMA over Converged Ethernet (RoCE) for the Run3 LHCb event building at CERN. The acquisition system of the detector will collect partial data from approximately 1000 separate detector streams. Total estimated throughput equals 40 terabits per second. Full eve...

Descripción completa

Detalles Bibliográficos
Autor principal: Krawczyk, Rafal Dominik
Lenguaje:eng
Publicado: 2019
Acceso en línea:http://cds.cern.ch/record/2699799
_version_ 1780964454633046016
author Krawczyk, Rafal Dominik
author_facet Krawczyk, Rafal Dominik
author_sort Krawczyk, Rafal Dominik
collection CERN
description This paper evaluates the utilization of RDMA over Converged Ethernet (RoCE) for the Run3 LHCb event building at CERN. The acquisition system of the detector will collect partial data from approximately 1000 separate detector streams. Total estimated throughput equals 40 terabits per second. Full events will be assembled for subsequent processing and data selection in the filtering farm of the online trigger. As a result, inter-node large-throughput transmissions with a combination of 100 and 25 Gigabit-per-second will be essential features of the system. Therefore, the data exchange mechanism of the cluster must utilize memory-lightweight data transmission protocols. In this work, the RoCE high-throughput kernel bypass Ethernet-based protocol is benchmarked as an applicable technology for the event building network. CPU and memory bandwidth utilization for RoCE-based data transmissions is investigated and discussed. A comparison of RoCE with InfiniBand protocol is presented. Preliminary performance results are discussed with the selected network hardware supporting the protocol. Relevant utilization and interoperability issues are detailed along with lessons learned along the road.
id cern-2699799
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2019
record_format invenio
spelling cern-26997992019-11-08T21:20:53Zhttp://cds.cern.ch/record/2699799engKrawczyk, Rafal DominikFeasibility tests of RoCE for the cluster-based event building in LHCbThis paper evaluates the utilization of RDMA over Converged Ethernet (RoCE) for the Run3 LHCb event building at CERN. The acquisition system of the detector will collect partial data from approximately 1000 separate detector streams. Total estimated throughput equals 40 terabits per second. Full events will be assembled for subsequent processing and data selection in the filtering farm of the online trigger. As a result, inter-node large-throughput transmissions with a combination of 100 and 25 Gigabit-per-second will be essential features of the system. Therefore, the data exchange mechanism of the cluster must utilize memory-lightweight data transmission protocols. In this work, the RoCE high-throughput kernel bypass Ethernet-based protocol is benchmarked as an applicable technology for the event building network. CPU and memory bandwidth utilization for RoCE-based data transmissions is investigated and discussed. A comparison of RoCE with InfiniBand protocol is presented. Preliminary performance results are discussed with the selected network hardware supporting the protocol. Relevant utilization and interoperability issues are detailed along with lessons learned along the road.Poster-2019-1015oai:cds.cern.ch:26997992019-11-07
spellingShingle Krawczyk, Rafal Dominik
Feasibility tests of RoCE for the cluster-based event building in LHCb
title Feasibility tests of RoCE for the cluster-based event building in LHCb
title_full Feasibility tests of RoCE for the cluster-based event building in LHCb
title_fullStr Feasibility tests of RoCE for the cluster-based event building in LHCb
title_full_unstemmed Feasibility tests of RoCE for the cluster-based event building in LHCb
title_short Feasibility tests of RoCE for the cluster-based event building in LHCb
title_sort feasibility tests of roce for the cluster-based event building in lhcb
url http://cds.cern.ch/record/2699799
work_keys_str_mv AT krawczykrafaldominik feasibilitytestsofrocefortheclusterbasedeventbuildinginlhcb