Actor feedback and rigorous monitoring: Essential quality assurance tools for testing behavioral interventions with simulation

INTRODUCTION: Simulation is a powerful tool for training and evaluating clinicians. However, few studies have examined the consistency of actor performances during simulation based medical education (SBME). The Simulated Communication with ICU Proxies trial (ClinicalTrials.gov NCT02721810) used simulation to evaluate the effect of a behavioral intervention on physician communication. The purpose of this secondary analysis of data generated by the quality assurance team during the trial was to assess how quality assurance monitoring procedures impacted rates of actor errors during simulations. METHODS: The trial used rigorous quality assurance to train actors, evaluate performances, and ensure the intervention was delivered within a standardized environment. The quality assurance team evaluated video recordings and documented errors. Actors received both timely, formative feedback and participated in group feedback sessions. RESULTS: Error rates varied significantly across three actors (H(2) = 8.22, p = 0.02). In adjusted analyses, there was a decrease in the incidence of actor error over time, and errors decreased sharply after the first group feedback session (Incidence Rate Ratio = 0.25, 95% confidence interval 0.14–0.42). CONCLUSIONS: Rigorous quality assurance procedures may help ensure consistent actor performances during SBME.