Development of noSQL data storage for the ATLAS PanDA Monitoring System

Mostrar otras versiones (1)

For several years the PanDA Workload Management System has been the basis for distributed production and analysis for the ATLAS experiment at the LHC. Since the start of data taking PanDA usage has ramped up steadily, typically exceeding 500k completed jobs/day by June 2011. The associated monitorin...

Descripción completa

Detalles Bibliográficos
Autores principales: Ito, H, Potekhin, M, Wenaus, T
Lenguaje:eng
Publicado: 2012
Materias:
Detectors and Experimental Techniques
Acceso en línea:http://cds.cern.ch/record/1446543
_version_ 1780924791818027008
author Ito, H
Potekhin, M
Wenaus, T
author_facet Ito, H
Potekhin, M
Wenaus, T
author_sort Ito, H
collection CERN
description For several years the PanDA Workload Management System has been the basis for distributed production and analysis for the ATLAS experiment at the LHC. Since the start of data taking PanDA usage has ramped up steadily, typically exceeding 500k completed jobs/day by June 2011. The associated monitoring data volume has been rising as well, to levels that present a new set of challenges in the areas of database scalability and monitoring system performance and efficiency. These challenges have being met with a R&D and development effort aimed at implementing a scalable and efficient monitoring data storage based on a noSQL solution (Cassandra). We present the data design and indexing strategies for efficient queries, as well as our experience of operating a Cassandra cluster and interfacing it with a Web service.
id cern-1446543
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2012
record_format invenio
spelling cern-14465432019-09-30T06:29:59Zhttp://cds.cern.ch/record/1446543engIto, HPotekhin, MWenaus, TDevelopment of noSQL data storage for the ATLAS PanDA Monitoring SystemDetectors and Experimental TechniquesFor several years the PanDA Workload Management System has been the basis for distributed production and analysis for the ATLAS experiment at the LHC. Since the start of data taking PanDA usage has ramped up steadily, typically exceeding 500k completed jobs/day by June 2011. The associated monitoring data volume has been rising as well, to levels that present a new set of challenges in the areas of database scalability and monitoring system performance and efficiency. These challenges have being met with a R&D and development effort aimed at implementing a scalable and efficient monitoring data storage based on a noSQL solution (Cassandra). We present the data design and indexing strategies for efficient queries, as well as our experience of operating a Cassandra cluster and interfacing it with a Web service.ATL-SOFT-SLIDE-2012-181oai:cds.cern.ch:14465432012-05-07
spellingShingle Detectors and Experimental Techniques
Ito, H
Potekhin, M
Wenaus, T
Development of noSQL data storage for the ATLAS PanDA Monitoring System
title Development of noSQL data storage for the ATLAS PanDA Monitoring System
title_full Development of noSQL data storage for the ATLAS PanDA Monitoring System
title_fullStr Development of noSQL data storage for the ATLAS PanDA Monitoring System
title_full_unstemmed Development of noSQL data storage for the ATLAS PanDA Monitoring System
title_short Development of noSQL data storage for the ATLAS PanDA Monitoring System
title_sort development of nosql data storage for the atlas panda monitoring system
topic Detectors and Experimental Techniques
url http://cds.cern.ch/record/1446543
work_keys_str_mv AT itoh developmentofnosqldatastoragefortheatlaspandamonitoringsystem
AT potekhinm developmentofnosqldatastoragefortheatlaspandamonitoringsystem
AT wenaust developmentofnosqldatastoragefortheatlaspandamonitoringsystem

Ejemplares similares