Meaningful assessment method for laparoscopic suturing training in augmented reality

BACKGROUND: To be an effective training tool, a laparoscopic simulator has to provide metrics that are meaningful and informative to the trainee. Time, path length and smoothness are often used parameters, but are not very informative on the quality of the performance. This study aims to validate a newly developed assessment method for laparoscopic suturing on the ProMIS augmented reality simulator, and compares it with scores of objective observers. METHODS: Twenty-four participants practised their suturing skills on the augmented reality suturing module: experienced participants (n = 10), >50 clinical laparoscopic suturing experience; and novice participants (n = 14), without laparoscopic experience. The performances were recorded and assessed by two unrelated observers and compared with the assessment scores. The assessment score was a calculation of time spent in the correct area and quality (strength) of the knot. To test the accuracy of the individual assessment parameters, we compared these with each other. RESULTS: The experienced participants had significantly higher performance scores than the novice participants in the beginner-level mode (mean 95.73 vs. 60.89, standard deviation 2.63 vs. 17.09, p < 0.001, independent t-test). The performance scores of the assessment method (n = 43) correlated significantly with the scorings of the objective observers (Spearman’s rho 0.672; p < 0.001). The parameter time spent in correct area had a calculated significant correlation with the strength of the knot (n = 229, Spearman’s rho 0.257, p < 0.001), but this was clinically irrelevant. CONCLUSION: This assessment method is a valid tool for objectively assessing laparoscopic suturing skills. Although assessment parameters can correlate, to provide informative feedback it is important to combine meaningful measurements in the assessment of suturing skills.