Evaluation of the quality of COVID-19 guidance documents in anaesthesia using the Appraisal of Guidelines for Research and Evaluation II instrument

BACKGROUND: Guidance documents are a valuable resource to clinicians to guide evidenced-based decision making. The quality of guidelines in anaesthesia and across other specialties has been demonstrated to be poor. COVID-19 presented an urgent need for immediate guidance for anaesthetists as frontline clinicians. The aim of this study was to evaluate the quality of COVID-19 guidance documents using the internationally validated Appraisal of Guidelines for Research & Evaluation (AGREE) II tool. METHODS: A search was conducted in Ovid EMBASE and Ovid MEDLINE to identify all COVID-19 anaesthesia guidance documents from 2020-2021. Thirty-eight guidance documents were selected for analysis by 4 independent appraisers using the AGREE II instrument, across its 6 domains and 23 items. A scoring threshold for high quality was agreed by the working group via consensus. RESULTS: Overall, the body of COVID-19 guidance documents achieved poor scores using AGREE II. Only 5% of documents met the high-quality criteria. Markers of quality included international and multi-institutional collaboration. Document title (‘guideline’ vs ‘consensus statement’/ ‘recommendations’) did not yield any differences in domain scores and overall quality ratings. Compared with recent general anaesthesia guidelines, COVID-19 guidelines performed significantly worse. CONCLUSIONS: COVID-19 guidance documents published during the first two years of the pandemic lacked rigour and appropriate quality. This raises concern about their trustworthiness for use in clinical practice. Enhanced systems are required to ensure the integrity of rapidly formulated guidance.