RAS-NOTECHS: validity and reliability of a tool for measuring non-technical skills in robotic-assisted surgery settings

BACKGROUND: Non-technical skills (NTS) are essential for safe surgical practice as they impact workflow and patient outcomes. Observational tools to measure operating room (OR) teams’ NTS have been introduced. However, there are none that account for the specific teamwork challenges introduced by robotic-assisted surgery (RAS). We set out to develop and content-validate a tool to assess multidisciplinary NTS in RAS. METHODOLOGY: Stepwise, multi-method procedure. Observations in different surgical departments and a scoping literature review were first used to compile a set of RAS-specific teamwork behaviours. This list was refined and expert validated using a Delphi consensus approach consisting of qualitative interviews and a quantitative survey. Then, RAS-specific behaviours were merged with a well-established assessment tool on OR teamwork (NOTECHS II). Finally, the new tool—RAS-NOTECHS—was applied in standardized observations of real-world procedures to test its reliability (inter-rater agreement via intra-class correlations). RESULTS: Our scoping review revealed 5242 articles, of which 21 were included based on pre-established inclusion criteria. We elicited 16 RAS-specific behaviours from the literature base. These were synthesized with further 18 behavioural markers (obtained from 12 OR-observations) into a list of 26 behavioural markers. This list was reviewed by seven RAS experts and condensed to 15 expert-validated RAS-specific behavioural markers which were then merged into NOTECHS II. For five observations of urologic RAS procedures (duration: 13 h and 41 min), inter-rater agreement for identification of behavioural markers was strong. Agreement of RAS-NOTECHS scores indicated moderate to strong agreement. CONCLUSIONS: RAS-NOTECHS is the first observational tool for multidisciplinary NTS in RAS. In preliminary application, it has been shown to be reliable. Since RAS is rapidly increasing and challenges for effective and safe teamwork remain at the forefront of quality and safety of surgical care, RAS-NOTECHS may contribute to training and improvement efforts in technology-facilitated surgeries. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s00464-021-08474-2.