Cargando…

The ATLAS TAGS database distribution and management: Operational challenges of a multi-terabyte distributed database

The TAG files store summary event quantities that allow a quick selection of interesting events. This data will be produced at a nominal rate of 200 Hz, and is uploaded into a relational database for access from websites and other tools. The estimated database volume is 6TB per year, making it the l...

Descripción completa

Detalles Bibliográficos
Autores principales: Viegas, F, Malon, D, Cranshaw, J, Dimitrov, G, Nowak, M, Nairz, A, Goossens, L, Gallas, E, Gamboa, C, Wong, A, Vinek, E
Lenguaje:eng
Publicado: 2010
Materias:
Acceso en línea:https://dx.doi.org/10.1088/1742-6596/219/7/072058
http://cds.cern.ch/record/1270560
_version_ 1780920201156493312
author Viegas, F
Malon, D
Cranshaw, J
Dimitrov, G
Nowak, M
Nairz, A
Goossens, L
Gallas, E
Gamboa, C
Wong, A
Vinek, E
author_facet Viegas, F
Malon, D
Cranshaw, J
Dimitrov, G
Nowak, M
Nairz, A
Goossens, L
Gallas, E
Gamboa, C
Wong, A
Vinek, E
author_sort Viegas, F
collection CERN
description The TAG files store summary event quantities that allow a quick selection of interesting events. This data will be produced at a nominal rate of 200 Hz, and is uploaded into a relational database for access from websites and other tools. The estimated database volume is 6TB per year, making it the largest application running on the ATLAS relational databases, at CERN and at other voluntary sites. The sheer volume and high rate of production makes this application a challenge to data and resource management, in many aspects. This paper will focus on the operational challenges of this system. These include: uploading the data from files to the CERN's and remote sites' databases; distributing the TAG metadata that is essential to guide the user through event selection; controlling resource usage of the database, from the user query load to the strategy of cleaning and archiving of old TAG data.
id cern-1270560
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2010
record_format invenio
spelling cern-12705602022-08-17T13:24:58Zdoi:10.1088/1742-6596/219/7/072058http://cds.cern.ch/record/1270560engViegas, FMalon, DCranshaw, JDimitrov, GNowak, MNairz, AGoossens, LGallas, EGamboa, CWong, AVinek, EThe ATLAS TAGS database distribution and management: Operational challenges of a multi-terabyte distributed databaseComputing and ComputersThe TAG files store summary event quantities that allow a quick selection of interesting events. This data will be produced at a nominal rate of 200 Hz, and is uploaded into a relational database for access from websites and other tools. The estimated database volume is 6TB per year, making it the largest application running on the ATLAS relational databases, at CERN and at other voluntary sites. The sheer volume and high rate of production makes this application a challenge to data and resource management, in many aspects. This paper will focus on the operational challenges of this system. These include: uploading the data from files to the CERN's and remote sites' databases; distributing the TAG metadata that is essential to guide the user through event selection; controlling resource usage of the database, from the user query load to the strategy of cleaning and archiving of old TAG data.oai:cds.cern.ch:12705602010
spellingShingle Computing and Computers
Viegas, F
Malon, D
Cranshaw, J
Dimitrov, G
Nowak, M
Nairz, A
Goossens, L
Gallas, E
Gamboa, C
Wong, A
Vinek, E
The ATLAS TAGS database distribution and management: Operational challenges of a multi-terabyte distributed database
title The ATLAS TAGS database distribution and management: Operational challenges of a multi-terabyte distributed database
title_full The ATLAS TAGS database distribution and management: Operational challenges of a multi-terabyte distributed database
title_fullStr The ATLAS TAGS database distribution and management: Operational challenges of a multi-terabyte distributed database
title_full_unstemmed The ATLAS TAGS database distribution and management: Operational challenges of a multi-terabyte distributed database
title_short The ATLAS TAGS database distribution and management: Operational challenges of a multi-terabyte distributed database
title_sort atlas tags database distribution and management: operational challenges of a multi-terabyte distributed database
topic Computing and Computers
url https://dx.doi.org/10.1088/1742-6596/219/7/072058
http://cds.cern.ch/record/1270560
work_keys_str_mv AT viegasf theatlastagsdatabasedistributionandmanagementoperationalchallengesofamultiterabytedistributeddatabase
AT malond theatlastagsdatabasedistributionandmanagementoperationalchallengesofamultiterabytedistributeddatabase
AT cranshawj theatlastagsdatabasedistributionandmanagementoperationalchallengesofamultiterabytedistributeddatabase
AT dimitrovg theatlastagsdatabasedistributionandmanagementoperationalchallengesofamultiterabytedistributeddatabase
AT nowakm theatlastagsdatabasedistributionandmanagementoperationalchallengesofamultiterabytedistributeddatabase
AT nairza theatlastagsdatabasedistributionandmanagementoperationalchallengesofamultiterabytedistributeddatabase
AT goossensl theatlastagsdatabasedistributionandmanagementoperationalchallengesofamultiterabytedistributeddatabase
AT gallase theatlastagsdatabasedistributionandmanagementoperationalchallengesofamultiterabytedistributeddatabase
AT gamboac theatlastagsdatabasedistributionandmanagementoperationalchallengesofamultiterabytedistributeddatabase
AT wonga theatlastagsdatabasedistributionandmanagementoperationalchallengesofamultiterabytedistributeddatabase
AT vineke theatlastagsdatabasedistributionandmanagementoperationalchallengesofamultiterabytedistributeddatabase
AT viegasf atlastagsdatabasedistributionandmanagementoperationalchallengesofamultiterabytedistributeddatabase
AT malond atlastagsdatabasedistributionandmanagementoperationalchallengesofamultiterabytedistributeddatabase
AT cranshawj atlastagsdatabasedistributionandmanagementoperationalchallengesofamultiterabytedistributeddatabase
AT dimitrovg atlastagsdatabasedistributionandmanagementoperationalchallengesofamultiterabytedistributeddatabase
AT nowakm atlastagsdatabasedistributionandmanagementoperationalchallengesofamultiterabytedistributeddatabase
AT nairza atlastagsdatabasedistributionandmanagementoperationalchallengesofamultiterabytedistributeddatabase
AT goossensl atlastagsdatabasedistributionandmanagementoperationalchallengesofamultiterabytedistributeddatabase
AT gallase atlastagsdatabasedistributionandmanagementoperationalchallengesofamultiterabytedistributeddatabase
AT gamboac atlastagsdatabasedistributionandmanagementoperationalchallengesofamultiterabytedistributeddatabase
AT wonga atlastagsdatabasedistributionandmanagementoperationalchallengesofamultiterabytedistributeddatabase
AT vineke atlastagsdatabasedistributionandmanagementoperationalchallengesofamultiterabytedistributeddatabase