Cargando…

Development of noSQL data storage for the ATLAS PanDA Monitoring System

For several years the PanDA Workload Management System has been the basis for distributed production and analysis for the ATLAS experiment at the LHC. Since the start of data taking PanDA usage has ramped up steadily, typically exceeding 500k completed jobs/day by June 2011. The associated monitorin...

Descripción completa

Detalles Bibliográficos
Autores principales: Ito, H, Potekhin, M, Wenaus, T
Lenguaje:eng
Publicado: 2012
Materias:
Acceso en línea:http://cds.cern.ch/record/1446543
Descripción
Sumario:For several years the PanDA Workload Management System has been the basis for distributed production and analysis for the ATLAS experiment at the LHC. Since the start of data taking PanDA usage has ramped up steadily, typically exceeding 500k completed jobs/day by June 2011. The associated monitoring data volume has been rising as well, to levels that present a new set of challenges in the areas of database scalability and monitoring system performance and efficiency. These challenges have being met with a R&D and development effort aimed at implementing a scalable and efficient monitoring data storage based on a noSQL solution (Cassandra). We present the data design and indexing strategies for efficient queries, as well as our experience of operating a Cassandra cluster and interfacing it with a Web service.