Cargando…
Dimensioning storage and computing clusters for efficient High Throughput Computing
<!--HTML-->Scientific experiments are producing huge amounts of data, and they continue increasing the size of their datasets and the total volume of data. These data are then processed by researchers belonging to large scientific collaborations, with the Large Hadron Collider being a good exa...
Autor principal: | |
---|---|
Lenguaje: | eng |
Publicado: |
2012
|
Materias: | |
Acceso en línea: | http://cds.cern.ch/record/1460665 |
_version_ | 1780925250990505984 |
---|---|
author | Espinal Curull, Xavier |
author_facet | Espinal Curull, Xavier |
author_sort | Espinal Curull, Xavier |
collection | CERN |
description | <!--HTML-->Scientific experiments are producing huge amounts of data, and they continue increasing the size of their datasets and the total volume of data. These data are then processed by researchers belonging to large scientific collaborations, with the Large Hadron Collider being a good example. The focal point of Scientific Data Centres has shifted from coping efficiently with PetaByte scale storage to deliver quality data processing throughput. The dimensioning of the internal components in High Throughput Computing (HTC) data centers is of crucial importance to cope with all the activities demanded by the experiments, both the online (data acceptance) and the offline (data processing, simulation and user analysis). This requires a precise setup involving disk and tape storage services, a computing cluster and the internal networking to prevent bottlenecks, overloads and undesired slowness that lead to losses cpu cycles and batch jobs failures. In this paper we point out relevant features for running a successful storage setup in an intensive HTC environment |
id | cern-1460665 |
institution | Organización Europea para la Investigación Nuclear |
language | eng |
publishDate | 2012 |
record_format | invenio |
spelling | cern-14606652022-11-02T22:23:35Zhttp://cds.cern.ch/record/1460665engEspinal Curull, XavierDimensioning storage and computing clusters for efficient High Throughput ComputingComputing in High Energy and Nuclear Physics (CHEP) 2012Conferences<!--HTML-->Scientific experiments are producing huge amounts of data, and they continue increasing the size of their datasets and the total volume of data. These data are then processed by researchers belonging to large scientific collaborations, with the Large Hadron Collider being a good example. The focal point of Scientific Data Centres has shifted from coping efficiently with PetaByte scale storage to deliver quality data processing throughput. The dimensioning of the internal components in High Throughput Computing (HTC) data centers is of crucial importance to cope with all the activities demanded by the experiments, both the online (data acceptance) and the offline (data processing, simulation and user analysis). This requires a precise setup involving disk and tape storage services, a computing cluster and the internal networking to prevent bottlenecks, overloads and undesired slowness that lead to losses cpu cycles and batch jobs failures. In this paper we point out relevant features for running a successful storage setup in an intensive HTC environmentoai:cds.cern.ch:14606652012 |
spellingShingle | Conferences Espinal Curull, Xavier Dimensioning storage and computing clusters for efficient High Throughput Computing |
title | Dimensioning storage and computing clusters for efficient High Throughput Computing |
title_full | Dimensioning storage and computing clusters for efficient High Throughput Computing |
title_fullStr | Dimensioning storage and computing clusters for efficient High Throughput Computing |
title_full_unstemmed | Dimensioning storage and computing clusters for efficient High Throughput Computing |
title_short | Dimensioning storage and computing clusters for efficient High Throughput Computing |
title_sort | dimensioning storage and computing clusters for efficient high throughput computing |
topic | Conferences |
url | http://cds.cern.ch/record/1460665 |
work_keys_str_mv | AT espinalcurullxavier dimensioningstorageandcomputingclustersforefficienthighthroughputcomputing AT espinalcurullxavier computinginhighenergyandnuclearphysicschep2012 |