Cargando…

Analysis of empty ATLAS pilot jobs

The pilot model used by the ATLAS production system has been in use for many years. The model has proven to be a success with many advantages over push models. However one of the negative side-effects of using a pilot model is the presence of 'empty pilots' running on sites which consume a...

Descripción completa

Detalles Bibliográficos
Autor principal: Love, Peter
Lenguaje:eng
Publicado: 2016
Materias:
Acceso en línea:http://cds.cern.ch/record/2218074
_version_ 1780952138694787072
author Love, Peter
author_facet Love, Peter
author_sort Love, Peter
collection CERN
description The pilot model used by the ATLAS production system has been in use for many years. The model has proven to be a success with many advantages over push models. However one of the negative side-effects of using a pilot model is the presence of 'empty pilots' running on sites which consume a small amount of walltime and not running a useful payload job. The impact on a site can be significant with previous studies showing a total 0.5% walltime usage with no benefit to either the site or to ATLAS. Another impact is the number of empty pilots being processed by a site's Compute Element and batch system which can be 5% of the total number of pilots being handled. In this paper we review the latest statistics using both ATLAS and site data and highlight edge cases where the number of empty pilots dominate. We also study the effect of tuning the pilot factories to reduce the number of empty pilots.
id cern-2218074
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2016
record_format invenio
spelling cern-22180742019-09-30T06:29:59Zhttp://cds.cern.ch/record/2218074engLove, PeterAnalysis of empty ATLAS pilot jobsParticle Physics - ExperimentThe pilot model used by the ATLAS production system has been in use for many years. The model has proven to be a success with many advantages over push models. However one of the negative side-effects of using a pilot model is the presence of 'empty pilots' running on sites which consume a small amount of walltime and not running a useful payload job. The impact on a site can be significant with previous studies showing a total 0.5% walltime usage with no benefit to either the site or to ATLAS. Another impact is the number of empty pilots being processed by a site's Compute Element and batch system which can be 5% of the total number of pilots being handled. In this paper we review the latest statistics using both ATLAS and site data and highlight edge cases where the number of empty pilots dominate. We also study the effect of tuning the pilot factories to reduce the number of empty pilots.ATL-SOFT-SLIDE-2016-698oai:cds.cern.ch:22180742016-09-24
spellingShingle Particle Physics - Experiment
Love, Peter
Analysis of empty ATLAS pilot jobs
title Analysis of empty ATLAS pilot jobs
title_full Analysis of empty ATLAS pilot jobs
title_fullStr Analysis of empty ATLAS pilot jobs
title_full_unstemmed Analysis of empty ATLAS pilot jobs
title_short Analysis of empty ATLAS pilot jobs
title_sort analysis of empty atlas pilot jobs
topic Particle Physics - Experiment
url http://cds.cern.ch/record/2218074
work_keys_str_mv AT lovepeter analysisofemptyatlaspilotjobs