Cargando…

PD2P : PanDA Dynamic Data Placement for ATLAS

<!--HTML-->The PanDA Production and Distributed Analysis System is the ATLAS workload management system for processing user analysis, group analysis and production jobs. In 2011 more than 1400 users have submitted jobs through PanDA to the ATLAS grid infrastructure. The system processes more...

Descripción completa

Detalles Bibliográficos
Autor principal: Maeno, Tadashi
Lenguaje:eng
Publicado: 2012
Materias:
Acceso en línea:http://cds.cern.ch/record/1460907
_version_ 1780925267477266432
author Maeno, Tadashi
author_facet Maeno, Tadashi
author_sort Maeno, Tadashi
collection CERN
description <!--HTML-->The PanDA Production and Distributed Analysis System is the ATLAS workload management system for processing user analysis, group analysis and production jobs. In 2011 more than 1400 users have submitted jobs through PanDA to the ATLAS grid infrastructure. The system processes more than 2 million analysis jobs per week. Analysis jobs are routed to sites based on the availability of relevant data and processing resources, taking account of the nonuniform distribution of CPU and storage resources in the ATLAS grid. The data distribution has to be optimized to fit the resource distribution, and also has to be dynamically changed to meet rapidly evolving requirements for analysis use cases. The PanDA Dynamic Data Placement (PD2P) system has been developed to cope with difficulties of data placement for ATLAS. PD2P is an intelligent subsystem of PanDA to distribute data by taking the following factors into account: popularity, locality, the usage pattern of the data, the distribution of CPU and storage resources, network topology between sites, site operation downtime and reliability, and so on. We will describe the design of the new system, its performance during the past year of data taking, dramatic improvements it has brought about in the efficient use of storage and processing resources, associated reductions in average wait time for user analysis jobs, and plans for the future.
id cern-1460907
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2012
record_format invenio
spelling cern-14609072022-11-02T22:23:31Zhttp://cds.cern.ch/record/1460907engMaeno, TadashiPD2P : PanDA Dynamic Data Placement for ATLASComputing in High Energy and Nuclear Physics (CHEP) 2012Conferences<!--HTML-->The PanDA Production and Distributed Analysis System is the ATLAS workload management system for processing user analysis, group analysis and production jobs. In 2011 more than 1400 users have submitted jobs through PanDA to the ATLAS grid infrastructure. The system processes more than 2 million analysis jobs per week. Analysis jobs are routed to sites based on the availability of relevant data and processing resources, taking account of the nonuniform distribution of CPU and storage resources in the ATLAS grid. The data distribution has to be optimized to fit the resource distribution, and also has to be dynamically changed to meet rapidly evolving requirements for analysis use cases. The PanDA Dynamic Data Placement (PD2P) system has been developed to cope with difficulties of data placement for ATLAS. PD2P is an intelligent subsystem of PanDA to distribute data by taking the following factors into account: popularity, locality, the usage pattern of the data, the distribution of CPU and storage resources, network topology between sites, site operation downtime and reliability, and so on. We will describe the design of the new system, its performance during the past year of data taking, dramatic improvements it has brought about in the efficient use of storage and processing resources, associated reductions in average wait time for user analysis jobs, and plans for the future.oai:cds.cern.ch:14609072012
spellingShingle Conferences
Maeno, Tadashi
PD2P : PanDA Dynamic Data Placement for ATLAS
title PD2P : PanDA Dynamic Data Placement for ATLAS
title_full PD2P : PanDA Dynamic Data Placement for ATLAS
title_fullStr PD2P : PanDA Dynamic Data Placement for ATLAS
title_full_unstemmed PD2P : PanDA Dynamic Data Placement for ATLAS
title_short PD2P : PanDA Dynamic Data Placement for ATLAS
title_sort pd2p : panda dynamic data placement for atlas
topic Conferences
url http://cds.cern.ch/record/1460907
work_keys_str_mv AT maenotadashi pd2ppandadynamicdataplacementforatlas
AT maenotadashi computinginhighenergyandnuclearphysicschep2012