Cargando…

Managing ATLAS data on a petabyte-scale with DQ2

The ATLAS detector at CERN's Large Hadron Collider presents data handling requirements on an unprecedented scale. From 2008 on the ATLAS distributed data management system, Don Quijote2 (DQ2), must manage tens of petabytes of experiment data per year, distributed globally via the LCG, OSG and N...

Descripción completa

Detalles Bibliográficos
Autores principales: Branco, M, Cameron, D, Gaidioz, B, Garonne, V, Koblitz, B, Lassnig, M, Rocha, R, Salgado, P, Wenaus, T
Lenguaje:eng
Publicado: 2008
Materias:
Acceso en línea:https://dx.doi.org/10.1088/1742-6596/119/6/062017
http://cds.cern.ch/record/1177372
_version_ 1780916271579136000
author Branco, M
Cameron, D
Gaidioz, B
Garonne, V
Koblitz, B
Lassnig, M
Rocha, R
Salgado, P
Wenaus, T
author_facet Branco, M
Cameron, D
Gaidioz, B
Garonne, V
Koblitz, B
Lassnig, M
Rocha, R
Salgado, P
Wenaus, T
author_sort Branco, M
collection CERN
description The ATLAS detector at CERN's Large Hadron Collider presents data handling requirements on an unprecedented scale. From 2008 on the ATLAS distributed data management system, Don Quijote2 (DQ2), must manage tens of petabytes of experiment data per year, distributed globally via the LCG, OSG and NDGF computing grids, now commonly known as the WLCG. Since its inception in 2005 DQ2 has continuously managed all experiment data for the ATLAS collaboration, which now comprises over 3000 scientists participating from more than 150 universities and laboratories in 34 countries. Fulfilling its primary requirement of providing a highly distributed, fault-tolerant and scalable architecture DQ2 was successfully upgraded from managing data on a terabyte-scale to managing data on a petabyte-scale. We present improvements and enhancements to DQ2 based on the increasing demands for ATLAS data management. We describe performance issues, architectural changes and implementation decisions, the current state of deployment in test and production as well as anticipated future improvements. Test results presented here show that DQ2 is capable of handling data up to and beyond the requirements of full-scale data-taking.
id cern-1177372
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2008
record_format invenio
spelling cern-11773722022-08-17T13:37:19Zdoi:10.1088/1742-6596/119/6/062017http://cds.cern.ch/record/1177372engBranco, MCameron, DGaidioz, BGaronne, VKoblitz, BLassnig, MRocha, RSalgado, PWenaus, TManaging ATLAS data on a petabyte-scale with DQ2Computing and ComputersThe ATLAS detector at CERN's Large Hadron Collider presents data handling requirements on an unprecedented scale. From 2008 on the ATLAS distributed data management system, Don Quijote2 (DQ2), must manage tens of petabytes of experiment data per year, distributed globally via the LCG, OSG and NDGF computing grids, now commonly known as the WLCG. Since its inception in 2005 DQ2 has continuously managed all experiment data for the ATLAS collaboration, which now comprises over 3000 scientists participating from more than 150 universities and laboratories in 34 countries. Fulfilling its primary requirement of providing a highly distributed, fault-tolerant and scalable architecture DQ2 was successfully upgraded from managing data on a terabyte-scale to managing data on a petabyte-scale. We present improvements and enhancements to DQ2 based on the increasing demands for ATLAS data management. We describe performance issues, architectural changes and implementation decisions, the current state of deployment in test and production as well as anticipated future improvements. Test results presented here show that DQ2 is capable of handling data up to and beyond the requirements of full-scale data-taking.oai:cds.cern.ch:11773722008
spellingShingle Computing and Computers
Branco, M
Cameron, D
Gaidioz, B
Garonne, V
Koblitz, B
Lassnig, M
Rocha, R
Salgado, P
Wenaus, T
Managing ATLAS data on a petabyte-scale with DQ2
title Managing ATLAS data on a petabyte-scale with DQ2
title_full Managing ATLAS data on a petabyte-scale with DQ2
title_fullStr Managing ATLAS data on a petabyte-scale with DQ2
title_full_unstemmed Managing ATLAS data on a petabyte-scale with DQ2
title_short Managing ATLAS data on a petabyte-scale with DQ2
title_sort managing atlas data on a petabyte-scale with dq2
topic Computing and Computers
url https://dx.doi.org/10.1088/1742-6596/119/6/062017
http://cds.cern.ch/record/1177372
work_keys_str_mv AT brancom managingatlasdataonapetabytescalewithdq2
AT camerond managingatlasdataonapetabytescalewithdq2
AT gaidiozb managingatlasdataonapetabytescalewithdq2
AT garonnev managingatlasdataonapetabytescalewithdq2
AT koblitzb managingatlasdataonapetabytescalewithdq2
AT lassnigm managingatlasdataonapetabytescalewithdq2
AT rochar managingatlasdataonapetabytescalewithdq2
AT salgadop managingatlasdataonapetabytescalewithdq2
AT wenaust managingatlasdataonapetabytescalewithdq2