Experiences on Grid Production for GEANT4

Geant4 is a worldwide collaboration of scientists and software engineers whose goal is to develop, maintain and provide support for the Geant4 package, representing a general purpose Monte Carlo toolkit for simulating the propagation of particles through matter. Geant4 is currently employed in a row of particle physics experiments (BaBar, HARP, ATLAS, CMS, LHCb), and is also applied in other areas like space science, medical applications, and radiation studies. Due to the complexity of the Geant4 code and to the broad spectrum of possible configurations - involving a variety of physical processes for different source particles, particle energies and target or shielding materials - an intensive testing of new release candidates is mandatory to thoroughly test each of its components, especially before major releases, generally twice a year. Regression tests are required to be performed by the Geant4 team within a short period of time (2 to 3 weeks), basically demanding vast computational resources (equivalent to approximately 4 CPU years). In order to cope with this abrupt rise in CPU power, EGEE Grid resources are utilized (several EGEE sites provide resources for these Geant4 production series). The first Geant4 production on the Grid was performed in December 2004, and since December 2005 Geant4 is officially recognized as wLCG/EGEE VO. Basically, the test series compare a reference and candidate version of the Geant4 toolkit to detect any significant difference between them, by looking at calorimeter observables. In order to guarantee the successful testing of the release candidates within the required period of time, a minimum amount of dedicated computational resources (of the order of 100 CPU's) and robust dedicated services are vital. This includes in particular RB's and batch systems being in completely stable throughout the entire period. Each site is required to provide a shared file system across the worker nodes to prevent time-consuming multiple reinstallation of the required software. Geant4 production series encompasses the need to submit a vast number of Grid jobs within 2 to 3 weeks, where delays are not acceptable due to fixed release dates. Thus, particular attention has to be given to provide a stable computing environment and to ensure an uncompromising performance of basic services. This includes an instantaneous service support, since a missing response to performance problems might cause significant interruptions of the regression testing.