Cargando…

Using S3 cloud storage with ROOT and CvmFS

Amazon S3 is a widely adopted web API for scalable cloud storage that could also fulfill storage requirements of the high-energy physics community. CERN has been evaluating this option using some key HEP applications such as ROOT and the CernVM filesystem (CvmFS) with S3 back-ends. In this contribut...

Descripción completa

Detalles Bibliográficos
Autores principales: Arsuaga-Ríos, Maria, Heikkilä, Seppo S, Duellmann, Dirk, Meusel, Ren, Blomer, Jakob, Couturier, Ben
Lenguaje:eng
Publicado: 2015
Materias:
Acceso en línea:https://dx.doi.org/10.1088/1742-6596/664/2/022001
http://cds.cern.ch/record/2134519
_version_ 1780949896857124864
author Arsuaga-Ríos, Maria
Heikkilä, Seppo S
Duellmann, Dirk
Meusel, Ren
Blomer, Jakob
Couturier, Ben
author_facet Arsuaga-Ríos, Maria
Heikkilä, Seppo S
Duellmann, Dirk
Meusel, Ren
Blomer, Jakob
Couturier, Ben
author_sort Arsuaga-Ríos, Maria
collection CERN
description Amazon S3 is a widely adopted web API for scalable cloud storage that could also fulfill storage requirements of the high-energy physics community. CERN has been evaluating this option using some key HEP applications such as ROOT and the CernVM filesystem (CvmFS) with S3 back-ends. In this contribution we present an evaluation of two versions of the Huawei UDS storage system stressed with a large number of clients executing HEP software applications. The performance of concurrently storing individual objects is presented alongside with more complex data access patterns as produced by the ROOT data analysis framework. Both Huawei UDS generations show a successful scalability by supporting multiple byte-range requests in contrast with Amazon S3 or Ceph which do not support these commonly used HEP operations. We further report the S3 integration with recent CvmFS versions and summarize the experience with CvmFS/S3 for publishing daily releases of the full LHCb experiment software stack.
id oai-inspirehep.net-1413174
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2015
record_format invenio
spelling oai-inspirehep.net-14131742022-08-10T13:00:46Zdoi:10.1088/1742-6596/664/2/022001http://cds.cern.ch/record/2134519engArsuaga-Ríos, MariaHeikkilä, Seppo SDuellmann, DirkMeusel, RenBlomer, JakobCouturier, BenUsing S3 cloud storage with ROOT and CvmFSComputing and ComputersAmazon S3 is a widely adopted web API for scalable cloud storage that could also fulfill storage requirements of the high-energy physics community. CERN has been evaluating this option using some key HEP applications such as ROOT and the CernVM filesystem (CvmFS) with S3 back-ends. In this contribution we present an evaluation of two versions of the Huawei UDS storage system stressed with a large number of clients executing HEP software applications. The performance of concurrently storing individual objects is presented alongside with more complex data access patterns as produced by the ROOT data analysis framework. Both Huawei UDS generations show a successful scalability by supporting multiple byte-range requests in contrast with Amazon S3 or Ceph which do not support these commonly used HEP operations. We further report the S3 integration with recent CvmFS versions and summarize the experience with CvmFS/S3 for publishing daily releases of the full LHCb experiment software stack.oai:inspirehep.net:14131742015
spellingShingle Computing and Computers
Arsuaga-Ríos, Maria
Heikkilä, Seppo S
Duellmann, Dirk
Meusel, Ren
Blomer, Jakob
Couturier, Ben
Using S3 cloud storage with ROOT and CvmFS
title Using S3 cloud storage with ROOT and CvmFS
title_full Using S3 cloud storage with ROOT and CvmFS
title_fullStr Using S3 cloud storage with ROOT and CvmFS
title_full_unstemmed Using S3 cloud storage with ROOT and CvmFS
title_short Using S3 cloud storage with ROOT and CvmFS
title_sort using s3 cloud storage with root and cvmfs
topic Computing and Computers
url https://dx.doi.org/10.1088/1742-6596/664/2/022001
http://cds.cern.ch/record/2134519
work_keys_str_mv AT arsuagariosmaria usings3cloudstoragewithrootandcvmfs
AT heikkilaseppos usings3cloudstoragewithrootandcvmfs
AT duellmanndirk usings3cloudstoragewithrootandcvmfs
AT meuselren usings3cloudstoragewithrootandcvmfs
AT blomerjakob usings3cloudstoragewithrootandcvmfs
AT couturierben usings3cloudstoragewithrootandcvmfs