Sharing server nodes for storage and compute Batch on EOS Extra Resources

Based on the observation of low average CPU utilisation of several hundred disk servers in the EOS storage system at CERN, the Batch on EOS Extra Resources (BEER) project developed an approach to utilise these resources for batch processing. After initial proof of concept tests, showing almost no in...

Descripción completa

Detalles Bibliográficos
Autores principales: Schulz, Markus, Jones, Ben, Kiryanov, Andrey, Lamanna, Massimo, Mascetti, Luca, McCance, Gavin, Rousseau, Herve, Smith, David, Tollefsen, Havard
Lenguaje:eng
Publicado: 2018
Materias:
Talk
Acceso en línea:http://cds.cern.ch/record/2653016
_version_ 1780961012926644224
author Schulz, Markus
Jones, Ben
Kiryanov, Andrey
Lamanna, Massimo
Mascetti, Luca
McCance, Gavin
Rousseau, Herve
Smith, David
Tollefsen, Havard
author_facet Schulz, Markus
Jones, Ben
Kiryanov, Andrey
Lamanna, Massimo
Mascetti, Luca
McCance, Gavin
Rousseau, Herve
Smith, David
Tollefsen, Havard
author_sort Schulz, Markus
collection CERN
description Based on the observation of low average CPU utilisation of several hundred disk servers in the EOS storage system at CERN, the Batch on EOS Extra Resources (BEER) project developed an approach to utilise these resources for batch processing. After initial proof of concept tests, showing almost no interference between the batch and storage services, a model for production has been developed and implemented. This has been deployed on part the CERN EOS preproduction instance. The implementation and test results will be presented. The potential for additional resources at the CERN T0 centre is of the order of ten thousand hardware threads, as well as being a step towards a hyper-converged infrastructure.
id cern-2653016
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2018
record_format invenio
spelling cern-26530162019-11-04T15:16:48Zhttp://cds.cern.ch/record/2653016engSchulz, MarkusJones, BenKiryanov, AndreyLamanna, MassimoMascetti, LucaMcCance, GavinRousseau, HerveSmith, DavidTollefsen, HavardSharing server nodes for storage and compute Batch on EOS Extra ResourcesTalkBased on the observation of low average CPU utilisation of several hundred disk servers in the EOS storage system at CERN, the Batch on EOS Extra Resources (BEER) project developed an approach to utilise these resources for batch processing. After initial proof of concept tests, showing almost no interference between the batch and storage services, a model for production has been developed and implemented. This has been deployed on part the CERN EOS preproduction instance. The implementation and test results will be presented. The potential for additional resources at the CERN T0 centre is of the order of ten thousand hardware threads, as well as being a step towards a hyper-converged infrastructure.IT-TALK-2018-011oai:cds.cern.ch:26530162018
spellingShingle Talk
Schulz, Markus
Jones, Ben
Kiryanov, Andrey
Lamanna, Massimo
Mascetti, Luca
McCance, Gavin
Rousseau, Herve
Smith, David
Tollefsen, Havard
Sharing server nodes for storage and compute Batch on EOS Extra Resources
title Sharing server nodes for storage and compute Batch on EOS Extra Resources
title_full Sharing server nodes for storage and compute Batch on EOS Extra Resources
title_fullStr Sharing server nodes for storage and compute Batch on EOS Extra Resources
title_full_unstemmed Sharing server nodes for storage and compute Batch on EOS Extra Resources
title_short Sharing server nodes for storage and compute Batch on EOS Extra Resources
title_sort sharing server nodes for storage and compute batch on eos extra resources
topic Talk
url http://cds.cern.ch/record/2653016
work_keys_str_mv AT schulzmarkus sharingservernodesforstorageandcomputebatchoneosextraresources
AT jonesben sharingservernodesforstorageandcomputebatchoneosextraresources
AT kiryanovandrey sharingservernodesforstorageandcomputebatchoneosextraresources
AT lamannamassimo sharingservernodesforstorageandcomputebatchoneosextraresources
AT mascettiluca sharingservernodesforstorageandcomputebatchoneosextraresources
AT mccancegavin sharingservernodesforstorageandcomputebatchoneosextraresources
AT rousseauherve sharingservernodesforstorageandcomputebatchoneosextraresources
AT smithdavid sharingservernodesforstorageandcomputebatchoneosextraresources
AT tollefsenhavard sharingservernodesforstorageandcomputebatchoneosextraresources

Ejemplares similares