Federating distributed storage for clouds in ATLAS

Mostrar otras versiones (1)

Input data for applications that run in cloud computing centres can be stored at distant repositories, often with multiple copies of the popular data stored at many sites. Locating and retrieving the remote data can be challenging, and we believe that federating the storage can address this problem....

Descripción completa

Detalles Bibliográficos
Autores principales: Berghaus, Frank, Casteels, Kevin, Di Girolamo, Alessandro, Ebert, Marcus, Furano, Fabrizio, Fernandez Galindo, Fernando, Lassnig, Mario, Leavett-brown, Colin Roy, Paterson, Michael, Serfon, Cedric, Seuster, Rolf, Sobie, Randall, Tafirout, Reda, Taylor, Ryan P.
Lenguaje:eng
Publicado: 2017
Materias:
Particle Physics - Experiment
Computing and Computers
Acceso en línea:https://dx.doi.org/10.1088/1742-6596/1085/3/032027
http://cds.cern.ch/record/2290502
_version_ 1780956349381738496
author Berghaus, Frank
Casteels, Kevin
Di Girolamo, Alessandro
Ebert, Marcus
Furano, Fabrizio
Fernandez Galindo, Fernando
Lassnig, Mario
Leavett-brown, Colin Roy
Paterson, Michael
Serfon, Cedric
Seuster, Rolf
Sobie, Randall
Tafirout, Reda
Taylor, Ryan P.
author_facet Berghaus, Frank
Casteels, Kevin
Di Girolamo, Alessandro
Ebert, Marcus
Furano, Fabrizio
Fernandez Galindo, Fernando
Lassnig, Mario
Leavett-brown, Colin Roy
Paterson, Michael
Serfon, Cedric
Seuster, Rolf
Sobie, Randall
Tafirout, Reda
Taylor, Ryan P.
author_sort Berghaus, Frank
collection CERN
description Input data for applications that run in cloud computing centres can be stored at distant repositories, often with multiple copies of the popular data stored at many sites. Locating and retrieving the remote data can be challenging, and we believe that federating the storage can address this problem. A federation would locate the closest copy of the data on the basis of GeoIP information. Currently we are using the dynamic data federation Dynafed, a software solution developed by CERN IT. Dynafed supports several industry standards for connection protocols like Amazon’s S3, Microsoft’s Azure, as well as WebDAV and HTTP. Dynafed functions as an abstraction layer under which protocol-dependent authentication details are hidden from the user, requiring the user to only provide an X509 certificate.
id cern-2290502
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2017
record_format invenio
spelling cern-22905022022-10-22T06:15:07Zdoi:10.1088/1742-6596/1085/3/032027http://cds.cern.ch/record/2290502engBerghaus, FrankCasteels, KevinDi Girolamo, AlessandroEbert, MarcusFurano, FabrizioFernandez Galindo, FernandoLassnig, MarioLeavett-brown, Colin RoyPaterson, MichaelSerfon, CedricSeuster, RolfSobie, RandallTafirout, RedaTaylor, Ryan P.Federating distributed storage for clouds in ATLASParticle Physics - ExperimentComputing and ComputersInput data for applications that run in cloud computing centres can be stored at distant repositories, often with multiple copies of the popular data stored at many sites. Locating and retrieving the remote data can be challenging, and we believe that federating the storage can address this problem. A federation would locate the closest copy of the data on the basis of GeoIP information. Currently we are using the dynamic data federation Dynafed, a software solution developed by CERN IT. Dynafed supports several industry standards for connection protocols like Amazon’s S3, Microsoft’s Azure, as well as WebDAV and HTTP. Dynafed functions as an abstraction layer under which protocol-dependent authentication details are hidden from the user, requiring the user to only provide an X509 certificate.Input data for applications that run in cloud computing centres can be stored at distant repositories, often with multiple copies of the popular data stored at many sites. Locating and retrieving the remote data can be challenging, and we believe that federating the storage can address this problem. A federation would locate the closest copy of the data on the basis of GeoIP information. Currently we are using the dynamic data federation Dynafed, a software solution developed by CERN IT. Dynafed supports several industry standards for connection protocols like Amazon’s S3, Microsoft’s Azure, as well as WebDAV and HTTP. Dynafed functions as an abstraction layer under which protocol-dependent authentication details are hidden from the user, requiring the user to only provide an X509 certificate. We have setup an instance of Dynafed and integrated it into the ATLAS data distribution management system. We report on the challenges faced during the installation and integration. We have tested ATLAS analysis jobs submitted by the PanDA production system and we report on our first experiences with its operation.Input data for applications that run in cloud computing centres can be stored at distant repositories, often with multiple copies of the popular data stored at many sites. Locating and retrieving the remote data can be challenging, and we believe that federating the storage can address this problem. A federation would locate the closest copy of the data on the basis of GeoIP information. Currently we are using the dynamic data federation Dynafed, a software solution developed by CERN IT. Dynafed supports several industry standards for connection protocols like Amazon's S3, Microsoft's Azure, as well as WebDAV and HTTP. Dynafed functions as an abstraction layer under which protocol-dependent authentication details are hidden from the user, requiring the user to only provide an X509 certificate. We have setup an instance of Dynafed and integrated it into the ATLAS data distribution management system. We report on the challenges faced during the installation and integration. We have tested ATLAS analysis jobs submitted by the PanDA production system and we report on our first experiences with its operation.arXiv:1810.11226ATL-SOFT-PROC-2017-061oai:cds.cern.ch:22905022017-10-25
spellingShingle Particle Physics - Experiment
Computing and Computers
Berghaus, Frank
Casteels, Kevin
Di Girolamo, Alessandro
Ebert, Marcus
Furano, Fabrizio
Fernandez Galindo, Fernando
Lassnig, Mario
Leavett-brown, Colin Roy
Paterson, Michael
Serfon, Cedric
Seuster, Rolf
Sobie, Randall
Tafirout, Reda
Taylor, Ryan P.
Federating distributed storage for clouds in ATLAS
title Federating distributed storage for clouds in ATLAS
title_full Federating distributed storage for clouds in ATLAS
title_fullStr Federating distributed storage for clouds in ATLAS
title_full_unstemmed Federating distributed storage for clouds in ATLAS
title_short Federating distributed storage for clouds in ATLAS
title_sort federating distributed storage for clouds in atlas
topic Particle Physics - Experiment
Computing and Computers
url https://dx.doi.org/10.1088/1742-6596/1085/3/032027
http://cds.cern.ch/record/2290502
work_keys_str_mv AT berghausfrank federatingdistributedstorageforcloudsinatlas
AT casteelskevin federatingdistributedstorageforcloudsinatlas
AT digirolamoalessandro federatingdistributedstorageforcloudsinatlas
AT ebertmarcus federatingdistributedstorageforcloudsinatlas
AT furanofabrizio federatingdistributedstorageforcloudsinatlas
AT fernandezgalindofernando federatingdistributedstorageforcloudsinatlas
AT lassnigmario federatingdistributedstorageforcloudsinatlas
AT leavettbrowncolinroy federatingdistributedstorageforcloudsinatlas
AT patersonmichael federatingdistributedstorageforcloudsinatlas
AT serfoncedric federatingdistributedstorageforcloudsinatlas
AT seusterrolf federatingdistributedstorageforcloudsinatlas
AT sobierandall federatingdistributedstorageforcloudsinatlas
AT tafiroutreda federatingdistributedstorageforcloudsinatlas
AT taylorryanp federatingdistributedstorageforcloudsinatlas

Ejemplares similares