Cargando…
The ATLAS EventIndex and its evolution towards Run 3
The ATLAS experiment produced so far hundreds of petabytes of data and expects to have one order of magnitude more in the future. This data are spread among hundreds of computing Grid sites around the world. The EventIndex is the complete catalogue of all ATLAS events, real and simulated, keeping th...
Autores principales: | , , , , , , , , , , , , , , , , , , |
---|---|
Lenguaje: | eng |
Publicado: |
2019
|
Materias: | |
Acceso en línea: | https://dx.doi.org/10.1088/1742-6596/1525/1/012056 http://cds.cern.ch/record/2675026 |
_version_ | 1780962594140454912 |
---|---|
author | Villaplana Perez, Miguel Alexandrov, Evgeny Aleksandrov, Igor Baranowski, Zbigniew Barberis, Dario Dimitrov, Gancho Fernandez Casani, Alvaro Gallas, Elizabeth Garcia Montoro, Carlos Gonzalez de la Hoz, Santiago Hrivnac, Julius Iakovlev, Alexander Kazymov, Andrei Mineev, Mikhail Prokoshin, Fedor Rybkin, Grigori Sánchez, Javier Salt, José Vasileva, Petya Tsvetanova |
author_facet | Villaplana Perez, Miguel Alexandrov, Evgeny Aleksandrov, Igor Baranowski, Zbigniew Barberis, Dario Dimitrov, Gancho Fernandez Casani, Alvaro Gallas, Elizabeth Garcia Montoro, Carlos Gonzalez de la Hoz, Santiago Hrivnac, Julius Iakovlev, Alexander Kazymov, Andrei Mineev, Mikhail Prokoshin, Fedor Rybkin, Grigori Sánchez, Javier Salt, José Vasileva, Petya Tsvetanova |
author_sort | Villaplana Perez, Miguel |
collection | CERN |
description | The ATLAS experiment produced so far hundreds of petabytes of data and expects to have one order of magnitude more in the future. This data are spread among hundreds of computing Grid sites around the world. The EventIndex is the complete catalogue of all ATLAS events, real and simulated, keeping the references to all permanent files that contain a given event in any processing stage. It provides the means to select and access event data in the ATLAS distributed storage system, and provides support for completeness and consistency checks and trigger and offline selection overlap studies. The EventIndex employs various data handling technologies like Hadoop and Oracle databases, and is integrated with other systems of the ATLAS distributed computing infrastructure, including those for data, metadata, and production management. The project is in operation since the start of LHC Run 2 in 2015, and is in permanent development in order to fit the production and analysis demands and follow technology evolutions. The main data store in Hadoop, based on MapFiles and HBase, has worked well during Run 2 but new solutions are explored for the future. This paper reports on the current system performance and on the studies of a new data storage prototype that can carry the EventIndex through Run 3. |
id | cern-2675026 |
institution | Organización Europea para la Investigación Nuclear |
language | eng |
publishDate | 2019 |
record_format | invenio |
spelling | cern-26750262022-01-14T14:54:28Zdoi:10.1088/1742-6596/1525/1/012056http://cds.cern.ch/record/2675026engVillaplana Perez, MiguelAlexandrov, EvgenyAleksandrov, IgorBaranowski, ZbigniewBarberis, DarioDimitrov, GanchoFernandez Casani, AlvaroGallas, ElizabethGarcia Montoro, CarlosGonzalez de la Hoz, SantiagoHrivnac, JuliusIakovlev, AlexanderKazymov, AndreiMineev, MikhailProkoshin, FedorRybkin, GrigoriSánchez, JavierSalt, JoséVasileva, Petya TsvetanovaThe ATLAS EventIndex and its evolution towards Run 3Particle Physics - ExperimentThe ATLAS experiment produced so far hundreds of petabytes of data and expects to have one order of magnitude more in the future. This data are spread among hundreds of computing Grid sites around the world. The EventIndex is the complete catalogue of all ATLAS events, real and simulated, keeping the references to all permanent files that contain a given event in any processing stage. It provides the means to select and access event data in the ATLAS distributed storage system, and provides support for completeness and consistency checks and trigger and offline selection overlap studies. The EventIndex employs various data handling technologies like Hadoop and Oracle databases, and is integrated with other systems of the ATLAS distributed computing infrastructure, including those for data, metadata, and production management. The project is in operation since the start of LHC Run 2 in 2015, and is in permanent development in order to fit the production and analysis demands and follow technology evolutions. The main data store in Hadoop, based on MapFiles and HBase, has worked well during Run 2 but new solutions are explored for the future. This paper reports on the current system performance and on the studies of a new data storage prototype that can carry the EventIndex through Run 3.ATL-SOFT-PROC-2019-003oai:cds.cern.ch:26750262019-05-19 |
spellingShingle | Particle Physics - Experiment Villaplana Perez, Miguel Alexandrov, Evgeny Aleksandrov, Igor Baranowski, Zbigniew Barberis, Dario Dimitrov, Gancho Fernandez Casani, Alvaro Gallas, Elizabeth Garcia Montoro, Carlos Gonzalez de la Hoz, Santiago Hrivnac, Julius Iakovlev, Alexander Kazymov, Andrei Mineev, Mikhail Prokoshin, Fedor Rybkin, Grigori Sánchez, Javier Salt, José Vasileva, Petya Tsvetanova The ATLAS EventIndex and its evolution towards Run 3 |
title | The ATLAS EventIndex and its evolution towards Run 3 |
title_full | The ATLAS EventIndex and its evolution towards Run 3 |
title_fullStr | The ATLAS EventIndex and its evolution towards Run 3 |
title_full_unstemmed | The ATLAS EventIndex and its evolution towards Run 3 |
title_short | The ATLAS EventIndex and its evolution towards Run 3 |
title_sort | atlas eventindex and its evolution towards run 3 |
topic | Particle Physics - Experiment |
url | https://dx.doi.org/10.1088/1742-6596/1525/1/012056 http://cds.cern.ch/record/2675026 |
work_keys_str_mv | AT villaplanaperezmiguel theatlaseventindexanditsevolutiontowardsrun3 AT alexandrovevgeny theatlaseventindexanditsevolutiontowardsrun3 AT aleksandrovigor theatlaseventindexanditsevolutiontowardsrun3 AT baranowskizbigniew theatlaseventindexanditsevolutiontowardsrun3 AT barberisdario theatlaseventindexanditsevolutiontowardsrun3 AT dimitrovgancho theatlaseventindexanditsevolutiontowardsrun3 AT fernandezcasanialvaro theatlaseventindexanditsevolutiontowardsrun3 AT gallaselizabeth theatlaseventindexanditsevolutiontowardsrun3 AT garciamontorocarlos theatlaseventindexanditsevolutiontowardsrun3 AT gonzalezdelahozsantiago theatlaseventindexanditsevolutiontowardsrun3 AT hrivnacjulius theatlaseventindexanditsevolutiontowardsrun3 AT iakovlevalexander theatlaseventindexanditsevolutiontowardsrun3 AT kazymovandrei theatlaseventindexanditsevolutiontowardsrun3 AT mineevmikhail theatlaseventindexanditsevolutiontowardsrun3 AT prokoshinfedor theatlaseventindexanditsevolutiontowardsrun3 AT rybkingrigori theatlaseventindexanditsevolutiontowardsrun3 AT sanchezjavier theatlaseventindexanditsevolutiontowardsrun3 AT saltjose theatlaseventindexanditsevolutiontowardsrun3 AT vasilevapetyatsvetanova theatlaseventindexanditsevolutiontowardsrun3 AT villaplanaperezmiguel atlaseventindexanditsevolutiontowardsrun3 AT alexandrovevgeny atlaseventindexanditsevolutiontowardsrun3 AT aleksandrovigor atlaseventindexanditsevolutiontowardsrun3 AT baranowskizbigniew atlaseventindexanditsevolutiontowardsrun3 AT barberisdario atlaseventindexanditsevolutiontowardsrun3 AT dimitrovgancho atlaseventindexanditsevolutiontowardsrun3 AT fernandezcasanialvaro atlaseventindexanditsevolutiontowardsrun3 AT gallaselizabeth atlaseventindexanditsevolutiontowardsrun3 AT garciamontorocarlos atlaseventindexanditsevolutiontowardsrun3 AT gonzalezdelahozsantiago atlaseventindexanditsevolutiontowardsrun3 AT hrivnacjulius atlaseventindexanditsevolutiontowardsrun3 AT iakovlevalexander atlaseventindexanditsevolutiontowardsrun3 AT kazymovandrei atlaseventindexanditsevolutiontowardsrun3 AT mineevmikhail atlaseventindexanditsevolutiontowardsrun3 AT prokoshinfedor atlaseventindexanditsevolutiontowardsrun3 AT rybkingrigori atlaseventindexanditsevolutiontowardsrun3 AT sanchezjavier atlaseventindexanditsevolutiontowardsrun3 AT saltjose atlaseventindexanditsevolutiontowardsrun3 AT vasilevapetyatsvetanova atlaseventindexanditsevolutiontowardsrun3 |