CMS DAQ monitoring data storage technology evaluation

Up until the summer of 2019 the CMS DAQ system monitoring data have been stored in compressed JSON files in so called 'snapshots' taken during 2-3 second intervals. This storage method has made analysis of the stored data an inconvenient process. To address this issue, an evaluation of som...

Descripción completa

Detalles Bibliográficos
Autor principal: More, Oskar Tomas
Lenguaje:eng
Publicado: 2019
Materias:
Computing and Computers
Acceso en línea:http://cds.cern.ch/record/2688090
_version_ 1780963648895713280
author More, Oskar Tomas
author_facet More, Oskar Tomas
author_sort More, Oskar Tomas
collection CERN
description Up until the summer of 2019 the CMS DAQ system monitoring data have been stored in compressed JSON files in so called 'snapshots' taken during 2-3 second intervals. This storage method has made analysis of the stored data an inconvenient process. To address this issue, an evaluation of some contemporary storage systems have been made and benchmarked with our monitoring data. The two most promising technologies, Elasticsearch and PostgreSQL with Timescale has been selected and evaluated on data for a two-month period.
id cern-2688090
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2019
record_format invenio
spelling cern-26880902019-09-30T06:29:59Zhttp://cds.cern.ch/record/2688090engMore, Oskar TomasCMS DAQ monitoring data storage technology evaluationComputing and ComputersComputing and ComputersUp until the summer of 2019 the CMS DAQ system monitoring data have been stored in compressed JSON files in so called 'snapshots' taken during 2-3 second intervals. This storage method has made analysis of the stored data an inconvenient process. To address this issue, an evaluation of some contemporary storage systems have been made and benchmarked with our monitoring data. The two most promising technologies, Elasticsearch and PostgreSQL with Timescale has been selected and evaluated on data for a two-month period.CERN-STUDENTS-Note-2019-169oai:cds.cern.ch:26880902019-08-30
spellingShingle Computing and Computers
Computing and Computers
More, Oskar Tomas
CMS DAQ monitoring data storage technology evaluation
title CMS DAQ monitoring data storage technology evaluation
title_full CMS DAQ monitoring data storage technology evaluation
title_fullStr CMS DAQ monitoring data storage technology evaluation
title_full_unstemmed CMS DAQ monitoring data storage technology evaluation
title_short CMS DAQ monitoring data storage technology evaluation
title_sort cms daq monitoring data storage technology evaluation
topic Computing and Computers
Computing and Computers
url http://cds.cern.ch/record/2688090
work_keys_str_mv AT moreoskartomas cmsdaqmonitoringdatastoragetechnologyevaluation

Ejemplares similares