Cargando…
Using S3 cloud storage with ROOT and CvmFS
Amazon S3 is a widely adopted web API for scalable cloud storage that could also fulfill storage requirements of the high-energy physics community. CERN has been evaluating this option using some key HEP applications such as ROOT and the CernVM filesystem (CvmFS) with S3 back-ends. In this contribut...
Autores principales: | , , , , , |
---|---|
Lenguaje: | eng |
Publicado: |
2015
|
Materias: | |
Acceso en línea: | https://dx.doi.org/10.1088/1742-6596/664/2/022001 http://cds.cern.ch/record/2134519 |
_version_ | 1780949896857124864 |
---|---|
author | Arsuaga-Ríos, Maria Heikkilä, Seppo S Duellmann, Dirk Meusel, Ren Blomer, Jakob Couturier, Ben |
author_facet | Arsuaga-Ríos, Maria Heikkilä, Seppo S Duellmann, Dirk Meusel, Ren Blomer, Jakob Couturier, Ben |
author_sort | Arsuaga-Ríos, Maria |
collection | CERN |
description | Amazon S3 is a widely adopted web API for scalable cloud storage that could also fulfill storage requirements of the high-energy physics community. CERN has been evaluating this option using some key HEP applications such as ROOT and the CernVM filesystem (CvmFS) with S3 back-ends. In this contribution we present an evaluation of two versions of the Huawei UDS storage system stressed with a large number of clients executing HEP software applications. The performance of concurrently storing individual objects is presented alongside with more complex data access patterns as produced by the ROOT data analysis framework. Both Huawei UDS generations show a successful scalability by supporting multiple byte-range requests in contrast with Amazon S3 or Ceph which do not support these commonly used HEP operations. We further report the S3 integration with recent CvmFS versions and summarize the experience with CvmFS/S3 for publishing daily releases of the full LHCb experiment software stack. |
id | oai-inspirehep.net-1413174 |
institution | Organización Europea para la Investigación Nuclear |
language | eng |
publishDate | 2015 |
record_format | invenio |
spelling | oai-inspirehep.net-14131742022-08-10T13:00:46Zdoi:10.1088/1742-6596/664/2/022001http://cds.cern.ch/record/2134519engArsuaga-Ríos, MariaHeikkilä, Seppo SDuellmann, DirkMeusel, RenBlomer, JakobCouturier, BenUsing S3 cloud storage with ROOT and CvmFSComputing and ComputersAmazon S3 is a widely adopted web API for scalable cloud storage that could also fulfill storage requirements of the high-energy physics community. CERN has been evaluating this option using some key HEP applications such as ROOT and the CernVM filesystem (CvmFS) with S3 back-ends. In this contribution we present an evaluation of two versions of the Huawei UDS storage system stressed with a large number of clients executing HEP software applications. The performance of concurrently storing individual objects is presented alongside with more complex data access patterns as produced by the ROOT data analysis framework. Both Huawei UDS generations show a successful scalability by supporting multiple byte-range requests in contrast with Amazon S3 or Ceph which do not support these commonly used HEP operations. We further report the S3 integration with recent CvmFS versions and summarize the experience with CvmFS/S3 for publishing daily releases of the full LHCb experiment software stack.oai:inspirehep.net:14131742015 |
spellingShingle | Computing and Computers Arsuaga-Ríos, Maria Heikkilä, Seppo S Duellmann, Dirk Meusel, Ren Blomer, Jakob Couturier, Ben Using S3 cloud storage with ROOT and CvmFS |
title | Using S3 cloud storage with ROOT and CvmFS |
title_full | Using S3 cloud storage with ROOT and CvmFS |
title_fullStr | Using S3 cloud storage with ROOT and CvmFS |
title_full_unstemmed | Using S3 cloud storage with ROOT and CvmFS |
title_short | Using S3 cloud storage with ROOT and CvmFS |
title_sort | using s3 cloud storage with root and cvmfs |
topic | Computing and Computers |
url | https://dx.doi.org/10.1088/1742-6596/664/2/022001 http://cds.cern.ch/record/2134519 |
work_keys_str_mv | AT arsuagariosmaria usings3cloudstoragewithrootandcvmfs AT heikkilaseppos usings3cloudstoragewithrootandcvmfs AT duellmanndirk usings3cloudstoragewithrootandcvmfs AT meuselren usings3cloudstoragewithrootandcvmfs AT blomerjakob usings3cloudstoragewithrootandcvmfs AT couturierben usings3cloudstoragewithrootandcvmfs |