Cargando…

The ATLAS EventIndex and its evolution towards Run 3

The ATLAS experiment produced so far hundreds of petabytes of data and expects to have one order of magnitude more in the future. This data are spread among hundreds of computing Grid sites around the world. The EventIndex is the complete catalogue of all ATLAS events, real and simulated, keeping th...

Descripción completa

Detalles Bibliográficos
Autores principales: Villaplana Perez, Miguel, Alexandrov, Evgeny, Aleksandrov, Igor, Baranowski, Zbigniew, Barberis, Dario, Dimitrov, Gancho, Fernandez Casani, Alvaro, Gallas, Elizabeth, Garcia Montoro, Carlos, Gonzalez de la Hoz, Santiago, Hrivnac, Julius, Iakovlev, Alexander, Kazymov, Andrei, Mineev, Mikhail, Prokoshin, Fedor, Rybkin, Grigori, Sánchez, Javier, Salt, José, Vasileva, Petya Tsvetanova
Lenguaje:eng
Publicado: 2019
Materias:
Acceso en línea:https://dx.doi.org/10.1088/1742-6596/1525/1/012056
http://cds.cern.ch/record/2675026
_version_ 1780962594140454912
author Villaplana Perez, Miguel
Alexandrov, Evgeny
Aleksandrov, Igor
Baranowski, Zbigniew
Barberis, Dario
Dimitrov, Gancho
Fernandez Casani, Alvaro
Gallas, Elizabeth
Garcia Montoro, Carlos
Gonzalez de la Hoz, Santiago
Hrivnac, Julius
Iakovlev, Alexander
Kazymov, Andrei
Mineev, Mikhail
Prokoshin, Fedor
Rybkin, Grigori
Sánchez, Javier
Salt, José
Vasileva, Petya Tsvetanova
author_facet Villaplana Perez, Miguel
Alexandrov, Evgeny
Aleksandrov, Igor
Baranowski, Zbigniew
Barberis, Dario
Dimitrov, Gancho
Fernandez Casani, Alvaro
Gallas, Elizabeth
Garcia Montoro, Carlos
Gonzalez de la Hoz, Santiago
Hrivnac, Julius
Iakovlev, Alexander
Kazymov, Andrei
Mineev, Mikhail
Prokoshin, Fedor
Rybkin, Grigori
Sánchez, Javier
Salt, José
Vasileva, Petya Tsvetanova
author_sort Villaplana Perez, Miguel
collection CERN
description The ATLAS experiment produced so far hundreds of petabytes of data and expects to have one order of magnitude more in the future. This data are spread among hundreds of computing Grid sites around the world. The EventIndex is the complete catalogue of all ATLAS events, real and simulated, keeping the references to all permanent files that contain a given event in any processing stage. It provides the means to select and access event data in the ATLAS distributed storage system, and provides support for completeness and consistency checks and trigger and offline selection overlap studies. The EventIndex employs various data handling technologies like Hadoop and Oracle databases, and is integrated with other systems of the ATLAS distributed computing infrastructure, including those for data, metadata, and production management. The project is in operation since the start of LHC Run 2 in 2015, and is in permanent development in order to fit the production and analysis demands and follow technology evolutions. The main data store in Hadoop, based on MapFiles and HBase, has worked well during Run 2 but new solutions are explored for the future. This paper reports on the current system performance and on the studies of a new data storage prototype that can carry the EventIndex through Run 3.
id cern-2675026
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2019
record_format invenio
spelling cern-26750262022-01-14T14:54:28Zdoi:10.1088/1742-6596/1525/1/012056http://cds.cern.ch/record/2675026engVillaplana Perez, MiguelAlexandrov, EvgenyAleksandrov, IgorBaranowski, ZbigniewBarberis, DarioDimitrov, GanchoFernandez Casani, AlvaroGallas, ElizabethGarcia Montoro, CarlosGonzalez de la Hoz, SantiagoHrivnac, JuliusIakovlev, AlexanderKazymov, AndreiMineev, MikhailProkoshin, FedorRybkin, GrigoriSánchez, JavierSalt, JoséVasileva, Petya TsvetanovaThe ATLAS EventIndex and its evolution towards Run 3Particle Physics - ExperimentThe ATLAS experiment produced so far hundreds of petabytes of data and expects to have one order of magnitude more in the future. This data are spread among hundreds of computing Grid sites around the world. The EventIndex is the complete catalogue of all ATLAS events, real and simulated, keeping the references to all permanent files that contain a given event in any processing stage. It provides the means to select and access event data in the ATLAS distributed storage system, and provides support for completeness and consistency checks and trigger and offline selection overlap studies. The EventIndex employs various data handling technologies like Hadoop and Oracle databases, and is integrated with other systems of the ATLAS distributed computing infrastructure, including those for data, metadata, and production management. The project is in operation since the start of LHC Run 2 in 2015, and is in permanent development in order to fit the production and analysis demands and follow technology evolutions. The main data store in Hadoop, based on MapFiles and HBase, has worked well during Run 2 but new solutions are explored for the future. This paper reports on the current system performance and on the studies of a new data storage prototype that can carry the EventIndex through Run 3.ATL-SOFT-PROC-2019-003oai:cds.cern.ch:26750262019-05-19
spellingShingle Particle Physics - Experiment
Villaplana Perez, Miguel
Alexandrov, Evgeny
Aleksandrov, Igor
Baranowski, Zbigniew
Barberis, Dario
Dimitrov, Gancho
Fernandez Casani, Alvaro
Gallas, Elizabeth
Garcia Montoro, Carlos
Gonzalez de la Hoz, Santiago
Hrivnac, Julius
Iakovlev, Alexander
Kazymov, Andrei
Mineev, Mikhail
Prokoshin, Fedor
Rybkin, Grigori
Sánchez, Javier
Salt, José
Vasileva, Petya Tsvetanova
The ATLAS EventIndex and its evolution towards Run 3
title The ATLAS EventIndex and its evolution towards Run 3
title_full The ATLAS EventIndex and its evolution towards Run 3
title_fullStr The ATLAS EventIndex and its evolution towards Run 3
title_full_unstemmed The ATLAS EventIndex and its evolution towards Run 3
title_short The ATLAS EventIndex and its evolution towards Run 3
title_sort atlas eventindex and its evolution towards run 3
topic Particle Physics - Experiment
url https://dx.doi.org/10.1088/1742-6596/1525/1/012056
http://cds.cern.ch/record/2675026
work_keys_str_mv AT villaplanaperezmiguel theatlaseventindexanditsevolutiontowardsrun3
AT alexandrovevgeny theatlaseventindexanditsevolutiontowardsrun3
AT aleksandrovigor theatlaseventindexanditsevolutiontowardsrun3
AT baranowskizbigniew theatlaseventindexanditsevolutiontowardsrun3
AT barberisdario theatlaseventindexanditsevolutiontowardsrun3
AT dimitrovgancho theatlaseventindexanditsevolutiontowardsrun3
AT fernandezcasanialvaro theatlaseventindexanditsevolutiontowardsrun3
AT gallaselizabeth theatlaseventindexanditsevolutiontowardsrun3
AT garciamontorocarlos theatlaseventindexanditsevolutiontowardsrun3
AT gonzalezdelahozsantiago theatlaseventindexanditsevolutiontowardsrun3
AT hrivnacjulius theatlaseventindexanditsevolutiontowardsrun3
AT iakovlevalexander theatlaseventindexanditsevolutiontowardsrun3
AT kazymovandrei theatlaseventindexanditsevolutiontowardsrun3
AT mineevmikhail theatlaseventindexanditsevolutiontowardsrun3
AT prokoshinfedor theatlaseventindexanditsevolutiontowardsrun3
AT rybkingrigori theatlaseventindexanditsevolutiontowardsrun3
AT sanchezjavier theatlaseventindexanditsevolutiontowardsrun3
AT saltjose theatlaseventindexanditsevolutiontowardsrun3
AT vasilevapetyatsvetanova theatlaseventindexanditsevolutiontowardsrun3
AT villaplanaperezmiguel atlaseventindexanditsevolutiontowardsrun3
AT alexandrovevgeny atlaseventindexanditsevolutiontowardsrun3
AT aleksandrovigor atlaseventindexanditsevolutiontowardsrun3
AT baranowskizbigniew atlaseventindexanditsevolutiontowardsrun3
AT barberisdario atlaseventindexanditsevolutiontowardsrun3
AT dimitrovgancho atlaseventindexanditsevolutiontowardsrun3
AT fernandezcasanialvaro atlaseventindexanditsevolutiontowardsrun3
AT gallaselizabeth atlaseventindexanditsevolutiontowardsrun3
AT garciamontorocarlos atlaseventindexanditsevolutiontowardsrun3
AT gonzalezdelahozsantiago atlaseventindexanditsevolutiontowardsrun3
AT hrivnacjulius atlaseventindexanditsevolutiontowardsrun3
AT iakovlevalexander atlaseventindexanditsevolutiontowardsrun3
AT kazymovandrei atlaseventindexanditsevolutiontowardsrun3
AT mineevmikhail atlaseventindexanditsevolutiontowardsrun3
AT prokoshinfedor atlaseventindexanditsevolutiontowardsrun3
AT rybkingrigori atlaseventindexanditsevolutiontowardsrun3
AT sanchezjavier atlaseventindexanditsevolutiontowardsrun3
AT saltjose atlaseventindexanditsevolutiontowardsrun3
AT vasilevapetyatsvetanova atlaseventindexanditsevolutiontowardsrun3