Usability testing of radiotherapy systems as a medical device evaluation tool to inform hospital procurement decision-making

PURPOSE: Poor usability designs of radiotherapy systems can contribute to use errors and adverse events. Therefore, we evaluated the usability of two radiotherapy systems through radiation therapists’ performance, workload, and experience that can inform hospital procurement decision-making about the selection of appropriate radiotherapy system for radiation therapist use. METHODS: We performed a comparative usability study for two radiotherapy systems through user testing. Thirty radiation therapists participated in our study, in which four typical operational tasks were performed in two tested radiotherapy systems. User performance was measured by task completion time and completion difficulty level. User workloads were measured by perceived and physiological workload using NASA-TLX questionnaires and eye motion data. User experience was measured by the USE questionnaire. RESULTS: Significantly less task completion time and an easier task completion difficulty level were shown with the Varian Trilogy than with the XHA600E. The study results suggest that higher perceived and physiological workloads were experienced with the XHA600E than with the Varian Trilogy. Radiation therapists reported better user experience with the Varian Trilogy than with the XHA600E. Five paired t-tests regarding user performance, user workload, and user experience between the Varian Trilogy and the XHA600E were performed, showing that the Varian Trilogy radiotherapy system has a better usability design than the XHA600E radiotherapy system. CONCLUSIONS: Based on study results, we confirmed that the Varian Trilogy radiotherapy system has a better usability design than the XHA600E radiotherapy system. Furthermore, the study results provide valuable evidence for hospital procurement decision-making regarding the selection of a suitable radiotherapy system for radiation therapists to use.