A Novel Metric to Evaluate In Situ Workflows

Performance evaluation is crucial to understanding the behavior of scientific workflows and efficiently utilizing resources on high-performance computing architectures. In this study, we target an emerging type of workflow, called in situ workflows. Through an analysis of the state-of-the-art research on in situ workflows, we model a theoretical framework that helps characterize such workflows. We further propose a lightweight metric for assessing resource usage efficiency of an in situ workflow execution. By applying this metric to a simple, yet representative, synthetic workflow, we explore two possible scenarios (Idle Simulation and Idle Analyzer) for the execution of real in situ workflows. Experimental results show that there is no substantial difference in the performance of both the in transit placement (analytics on dedicated nodes) and the helper-core configuration (analytics co-allocated with simulation) on our target system.