Objective Assessment of Checklist Fidelity Using Digital Audio Recording and a Standardized Scoring System Audit

The use of the World Health Organization Surgical Safety Checklist (SSC) has been reported to significantly reduce operative morbidity and mortality rates. Recent findings have cast doubt on the efficacy of such checklists in improving patient safety. The effectiveness of surgical safety checklists cannot be fully measured or understood without an accurate assessment of implementation fidelity, most effectively through direct observations of the checklist process. Here, we describe the use of a secure audio recording protocol in conjunction with a novel standardized scoring system to assess checklist compliance rates. METHODS: We used a black box digital audio recording protocol to observe the execution of SSCs in real time. A novel checklist scoring system was used to quantify the implementation fidelity of a modified version of the SSC. Physician and staff perception of patient safety was also surveyed before and after implementation. RESULTS: Audio-recorded audits revealed a precisely executed checklist 73.6% of the time compared with a previously reported compliance rate of 97.6%. Implementation fidelity was highest during preanesthesia and preincision checklist sections, whereas postprocedure checklist compliance and fidelity was consistently the lowest. Positive attitudes on patient safety by surgical staff increased by 11% from baseline. CONCLUSIONS: The use of a secure digital audio recording protocol is a simple yet effective tool for observing checklist performance. Moreover, the implementation of a standardized scoring system allows for the objective evaluation of checklist fidelity. Together, they provide a powerful auditing tool for identifying improvement.