Visual Cluster Analysis for Computing Tasks at Workflow Management System of the ATLAS Experiment

Hundreds of petabytes of experimental data in high energy and nuclear physics (HENP) have already been obtained by unique scientific facilities such as LHC, RHIC, KEK. As the accelerators are being modernized (energy and luminosity were increased), data volumes are rapidly growing and have reached the exabyte scale, that also affects the increasing the number of analysis and data processing tasks, that are competing continuously for computational resources. The increase of processing tasks causes an increase in the performance of the computing environment by the involvement of high-performance computing resources, and forming a heterogeneous distributed computing environment (hundreds of distributed computing centers). In addition, errors happen to occur while executing tasks for data analysis and processing, which are caused by software and hardware failures. With a distributed model of data processing and analysis, the optimization of data management and workload systems becomes a fundamental task, and the lack of its timely solutions leads to economic, functional and time losses. This work describes the first stage of the study aiming at solving the task to increase the stability and efficiency of workflow management systems for mega-science experiments by using visual analytics methods. Using the case of the ATLAS experiment at LHC the visual methods for cluster analysis of the workload management system computing tasks/jobs will be applied. The interdependencies and correlations between various tasks/jobs parameters will be investigated and graphically interpreted in N-dimensional space using 3D projections. Visual analysis allows to identify the similar jobs, as well as anomaly jobs, and to determine by means of which parameters this anomaly is taking place. A further evolvement of the work in this direction will be focused on the increasing the amount of analysed computing jobs and the development of the appropriate infrastructure.