Control and monitoring of On-line Trigger Algorithms using gaucho

In the LHCb experiment, the trigger decisions are computed by Gaudi (the LHCb software framework) algorithms running on an event filter farm of around 2000 PCs. The control and monitoring of these algorithms has to be integrated in the overall experiment control system (ECS). To enable and facilitate this integration Gaucho, the GAUdi Component Helping Online, was developed. Gaucho consists of three parts: a C++ package integrated with Gaudi, the communications package DIM, and a set of PVSS panels and libraries. PVSS is a commercial SCADA system chosen as toolkit and framework for the LHCb controls system. The C++ package implements monitor service interface (IMonitorSvc) following the Gaudi specifications, with methods to declare variables and histograms for monitoring. Algorithms writers use them to indicate which quantities should be monitored. Since the interface resides in the GaudiKernel the code does not need changing if the monitoring services are not present. The Gaudi main job implements a state machine with states (stopped, configured, running and paused). The Gaudi job starts a DIM server. The Gaucho monitor service implements DIM services and commands to publish the monitored objects and receive commands, such as requests for state changes. PVSS panels provide the user interface. Using a DIM client written in the PVSS scripting language commands can be sent to the running job (configure, start, pause, stop) and to the algorithm to send data. 1D histograms are displayed using the PVSS tranding tool and 2D histograms are displayed using ROOT. Any histogram on the Gaudi Transient Store can also be dynamically published. Some configuration of algorithms is possible on the fly, and a future objective is to initialize algorithms using the data stored int the LHCb online configuration database. We give some user experience of monitoring a large number of software processes.