Dynamo: Handling Scientific Data Across Sites and Storage Media

Dynamo is a full-stack software solution for scientific data management. Dynamo’s architecture is modular, extensible, and customizable, making the software suitable for managing data in a wide range of installation scales, from a few terabytes stored at a single location to hundreds of petabytes di...

Descripción completa

Detalles Bibliográficos
Autores principales: Iiyama, Yutaro, Maier, Benedikt, Abercrombie, Daniel, Goncharov, Maxim, Paus, Christoph
Lenguaje:eng
Publicado: 2020
Materias:
hep-ex
Particle Physics - Experiment
cs.DC
Computing and Computers
Acceso en línea:https://dx.doi.org/10.1007/s41781-021-00054-2
http://cds.cern.ch/record/2715327
_version_ 1780965426660900864
author Iiyama, Yutaro
Maier, Benedikt
Abercrombie, Daniel
Goncharov, Maxim
Paus, Christoph
author_facet Iiyama, Yutaro
Maier, Benedikt
Abercrombie, Daniel
Goncharov, Maxim
Paus, Christoph
author_sort Iiyama, Yutaro
collection CERN
description Dynamo is a full-stack software solution for scientific data management. Dynamo’s architecture is modular, extensible, and customizable, making the software suitable for managing data in a wide range of installation scales, from a few terabytes stored at a single location to hundreds of petabytes distributed across a worldwide computing grid. This article documents the core system design of Dynamo and describes the applications that implement various data management tasks. A brief report is also given on the operational experiences of the system at the CMS experiment at the CERN Large Hadron Collider and at a small-scale analysis facility.
id cern-2715327
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2020
record_format invenio
spelling cern-27153272022-01-07T07:13:21Zdoi:10.1007/s41781-021-00054-2http://cds.cern.ch/record/2715327engIiyama, YutaroMaier, BenediktAbercrombie, DanielGoncharov, MaximPaus, ChristophDynamo: Handling Scientific Data Across Sites and Storage Mediahep-exParticle Physics - Experimentcs.DCComputing and ComputersDynamo is a full-stack software solution for scientific data management. Dynamo’s architecture is modular, extensible, and customizable, making the software suitable for managing data in a wide range of installation scales, from a few terabytes stored at a single location to hundreds of petabytes distributed across a worldwide computing grid. This article documents the core system design of Dynamo and describes the applications that implement various data management tasks. A brief report is also given on the operational experiences of the system at the CMS experiment at the CERN Large Hadron Collider and at a small-scale analysis facility.Dynamo is a full-stack software solution for scientific data management. Dynamo's architecture is modular, extensible, and customizable, making the software suitable for managing data in a wide range of installation scales, from a few terabytes stored at a single location to hundreds of petabytes distributed across a worldwide computing grid. This article documents the core system design of Dynamo and describes the applications that implement various data management tasks. A brief report is also given on the operational experiences of the system at the CMS experiment at the CERN Large Hadron Collider and at a small scale analysis facility.arXiv:2003.11409oai:cds.cern.ch:27153272020-03-25
spellingShingle hep-ex
Particle Physics - Experiment
cs.DC
Computing and Computers
Iiyama, Yutaro
Maier, Benedikt
Abercrombie, Daniel
Goncharov, Maxim
Paus, Christoph
Dynamo: Handling Scientific Data Across Sites and Storage Media
title Dynamo: Handling Scientific Data Across Sites and Storage Media
title_full Dynamo: Handling Scientific Data Across Sites and Storage Media
title_fullStr Dynamo: Handling Scientific Data Across Sites and Storage Media
title_full_unstemmed Dynamo: Handling Scientific Data Across Sites and Storage Media
title_short Dynamo: Handling Scientific Data Across Sites and Storage Media
title_sort dynamo: handling scientific data across sites and storage media
topic hep-ex
Particle Physics - Experiment
cs.DC
Computing and Computers
url https://dx.doi.org/10.1007/s41781-021-00054-2
http://cds.cern.ch/record/2715327
work_keys_str_mv AT iiyamayutaro dynamohandlingscientificdataacrosssitesandstoragemedia
AT maierbenedikt dynamohandlingscientificdataacrosssitesandstoragemedia
AT abercrombiedaniel dynamohandlingscientificdataacrosssitesandstoragemedia
AT goncharovmaxim dynamohandlingscientificdataacrosssitesandstoragemedia
AT pauschristoph dynamohandlingscientificdataacrosssitesandstoragemedia

Ejemplares similares