The Process Controller for the LHCb On-LIne Farm

The Process Controller is a tool of the LHCb FMC (Farm Monitoring and Control System) in charge of keeping a list of applications up and running on the farm nodes. It tipically runs on a few control PCs each one watching ~200 farm nodes and performs its task by maintaining the list of scheduled applications for each controlled farm node and by interacting with the Task Manager Servers running on the farm nodes to start processes, to obtain the notification of process termination, to re-spawn the terminated processes (if requested) and to stop processes. Processes can be added to or removed from the scheduled application list for one or more nodes by means of DIM commands, while DIM services provide the list of scheduled applications for each controlled farm node together with their properties, the number of re-spawns and the re-spawn times.