Objective assessment of surgical operative performance by observational clinical human reliability analysis (OCHRA): a systematic review

BACKGROUND: Both morbidity and mortality data (MMD) and learning curves (LCs) do not provide information on the nature of intraoperative errors and their mechanisms when these adversely impact on patient outcome. OCHRA was developed specifically to address the unmet surgical need for an objective assessment technique of the quality of technical execution of operations at individual operator level. The aim of this systematic review was to review of OCHRA as a method of objective assessment of surgical operative performance. METHODS: Systematic review based on searching 4 databases for articles published from January 1998 to January 2019. The review complies with Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines and includes original publications on surgical task performance based on technical errors during operations across several surgical specialties. RESULTS: Only 26 published studies met the search criteria, indicating that the uptake of OCHRA during the study period has been low. In 31% of reported studies, the operations were performed by fully qualified consultant/attending surgeons and by surgical trainees in 69% in approved training programs. OCHRA identified 7869 consequential errors (CE) during the conduct of 719 clinical operations (mean = 11 CEs). It also identified ‘hazard zones’ of operations and proficiency–gain curves (P-GCs) that confirm attainment of persistent competent execution of specific operations by individual trainee surgeons. P-GCs are both surgeon and operation specific. CONCLUSIONS: Increased OCHRA use has the potential to improve patient outcome after surgery, but this is a contingent progress towards automatic assessment of unedited videos of operations. The low uptake of OCHRA is attributed to its labor-intensive nature involving human factors (cognitive engineering) expertise. Aside from faster and more objective peer-based assessment, this development should accelerate increased clinical uptake and use of the technique in both routine surgical practice and surgical training.