Sharing server nodes for storage and compute Batch on EOS Extra Resources

Based on the observation of low average CPU utilisation of several hundred disk servers in the EOS storage system at CERN, the Batch on EOS Extra Resources (BEER) project developed an approach to utilise these resources for batch processing. After initial proof of concept tests, showing almost no in...

Descripción completa

Detalles Bibliográficos
Autores principales: Schulz, Markus, Jones, Ben, Kiryanov, Andrey, Lamanna, Massimo, Mascetti, Luca, McCance, Gavin, Rousseau, Herve, Smith, David, Tollefsen, Havard
Lenguaje:eng
Publicado: 2018
Materias:
Talk
Acceso en línea:http://cds.cern.ch/record/2653016
Descripción
Sumario:Based on the observation of low average CPU utilisation of several hundred disk servers in the EOS storage system at CERN, the Batch on EOS Extra Resources (BEER) project developed an approach to utilise these resources for batch processing. After initial proof of concept tests, showing almost no interference between the batch and storage services, a model for production has been developed and implemented. This has been deployed on part the CERN EOS preproduction instance. The implementation and test results will be presented. The potential for additional resources at the CERN T0 centre is of the order of ten thousand hardware threads, as well as being a step towards a hyper-converged infrastructure.

Ejemplares similares