The ATLAS EventIndex: an event catalogue for experiments collecting large amounts of data

Mostrar otras versiones (3)

Modern scientific experiments collect vast amounts of data that must be catalogued to meet multiple use cases and search criteria. In particular, high-energy physics experiments currently in operation produce several billion events per year. A database with the references to the files including each...

Descripción completa

Detalles Bibliográficos
Autores principales: Barberis, D, Cranshaw, J, Favareto, A, Fernández Casaní, Á, Gallas, E J, González de la Hoz, S, Hřivnáč, J, Malon, D, Nowak, M, Prokoshin, F, Salt Cairols, J, Sánchez, J, Töebbicke, R, Yuan, R
Lenguaje:eng
Publicado: 2014
Materias:
Detectors and Experimental Techniques
Acceso en línea:http://cds.cern.ch/record/1690609
_version_ 1780935610547044352
author Barberis, D
Cranshaw, J
Favareto, A
Fernández Casaní, Á
Gallas, E J
González de la Hoz, S
Hřivnáč, J
Malon, D
Nowak, M
Prokoshin, F
Salt Cairols, J
Sánchez, J
Töebbicke, R
Yuan, R
author_facet Barberis, D
Cranshaw, J
Favareto, A
Fernández Casaní, Á
Gallas, E J
González de la Hoz, S
Hřivnáč, J
Malon, D
Nowak, M
Prokoshin, F
Salt Cairols, J
Sánchez, J
Töebbicke, R
Yuan, R
author_sort Barberis, D
collection CERN
description Modern scientific experiments collect vast amounts of data that must be catalogued to meet multiple use cases and search criteria. In particular, high-energy physics experiments currently in operation produce several billion events per year. A database with the references to the files including each event in every stage of processing is necessary in order to retrieve the selected events from data storage systems. The ATLAS EventIndex project is developing a way to store the necessary information using modern data storage technologies (Hadoop, HBase etc.) that allow saving in memory key-value pairs and select the best tools to support this application from the point of view of performance, robustness and ease of use. This paper describes the initial design and performance tests and the project evolution towards deployment and operation during 2014.
id cern-1690609
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2014
record_format invenio
spelling cern-16906092019-09-30T06:29:59Zhttp://cds.cern.ch/record/1690609engBarberis, DCranshaw, JFavareto, AFernández Casaní, ÁGallas, E JGonzález de la Hoz, SHřivnáč, JMalon, DNowak, MProkoshin, FSalt Cairols, JSánchez, JTöebbicke, RYuan, RThe ATLAS EventIndex: an event catalogue for experiments collecting large amounts of dataDetectors and Experimental TechniquesModern scientific experiments collect vast amounts of data that must be catalogued to meet multiple use cases and search criteria. In particular, high-energy physics experiments currently in operation produce several billion events per year. A database with the references to the files including each event in every stage of processing is necessary in order to retrieve the selected events from data storage systems. The ATLAS EventIndex project is developing a way to store the necessary information using modern data storage technologies (Hadoop, HBase etc.) that allow saving in memory key-value pairs and select the best tools to support this application from the point of view of performance, robustness and ease of use. This paper describes the initial design and performance tests and the project evolution towards deployment and operation during 2014.ATL-SOFT-PROC-2014-002oai:cds.cern.ch:16906092014-03-28
spellingShingle Detectors and Experimental Techniques
Barberis, D
Cranshaw, J
Favareto, A
Fernández Casaní, Á
Gallas, E J
González de la Hoz, S
Hřivnáč, J
Malon, D
Nowak, M
Prokoshin, F
Salt Cairols, J
Sánchez, J
Töebbicke, R
Yuan, R
The ATLAS EventIndex: an event catalogue for experiments collecting large amounts of data
title The ATLAS EventIndex: an event catalogue for experiments collecting large amounts of data
title_full The ATLAS EventIndex: an event catalogue for experiments collecting large amounts of data
title_fullStr The ATLAS EventIndex: an event catalogue for experiments collecting large amounts of data
title_full_unstemmed The ATLAS EventIndex: an event catalogue for experiments collecting large amounts of data
title_short The ATLAS EventIndex: an event catalogue for experiments collecting large amounts of data
title_sort atlas eventindex: an event catalogue for experiments collecting large amounts of data
topic Detectors and Experimental Techniques
url http://cds.cern.ch/record/1690609
work_keys_str_mv AT barberisd theatlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT cranshawj theatlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT favaretoa theatlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT fernandezcasania theatlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT gallasej theatlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT gonzalezdelahozs theatlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT hrivnacj theatlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT malond theatlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT nowakm theatlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT prokoshinf theatlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT saltcairolsj theatlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT sanchezj theatlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT toebbicker theatlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT yuanr theatlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT barberisd atlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT cranshawj atlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT favaretoa atlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT fernandezcasania atlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT gallasej atlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT gonzalezdelahozs atlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT hrivnacj atlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT malond atlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT nowakm atlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT prokoshinf atlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT saltcairolsj atlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT sanchezj atlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT toebbicker atlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata
AT yuanr atlaseventindexaneventcatalogueforexperimentscollectinglargeamountsofdata

Ejemplares similares