Empirical Assessment of U.S. Coronavirus Disease 2019 Crisis Standards of Care Guidelines

OBJECTIVES: To establish the feasibility of empirically testing crisis standards of care guidelines. DESIGN: Retrospective single-center study. SETTING: ICUs at a large academic medical center in the United States. SUBJECTS: Adult, critically ill patients admitted to ICU, with 27 patients admitted for acute respiratory failure due to coronavirus disease 2019 and 37 patients admitted for diagnoses other than coronavirus disease 2019. INTERVENTIONS: Review of electronic health record. MEASUREMENTS AND MAIN RESULTS: Many U.S. states released crisis standards of care guidelines with algorithms to allocate scarce healthcare resources during the coronavirus disease 2019 pandemic. We compared state guidelines that represent different approaches to incorporating disease severity and comorbidities: New York, Maryland, Pennsylvania, and Colorado. Following each algorithm, we calculated priority scores at the time of ICU admission for a cohort of patients with primary diagnoses of coronavirus disease 2019 and diseases other than coronavirus disease 2019 (n = 64). We assessed discrimination of 28-day mortality by area under the receiver operating characteristic curve. We simulated real-time decision-making by applying the triage algorithms to groups of two, five, or 10 patients. For prediction of 28-day mortality by priority scores, area under the receiver operating characteristic curve was 0.56, 0.49, 0.53, 0.66, and 0.69 for New York, Maryland, Pennsylvania, Colorado, and raw Sequential Organ Failure Assessment score algorithms, respectively. For groups of five patients, the percentage of decisions made without deferring to a lottery were 1%, 57%, 80%, 88%, and 95% for New York, Maryland, Pennsylvania, Colorado, and raw Sequential Organ Failure Assessment score algorithms, respectively. The percentage of decisions made without lottery was higher in the subcohort without coronavirus disease 2019, compared with the subcohort with coronavirus disease 2019. CONCLUSIONS: Inclusion of comorbidities does not consistently improve an algorithm’s performance in predicting 28-day mortality. Crisis standards of care algorithms result in a substantial percentage of tied priority scores. Crisis standards of care algorithms operate differently in cohorts with and without coronavirus disease 2019. This proof-of-principle study demonstrates the feasibility and importance of empirical testing of crisis standards of care guidelines to understand whether they meet their goals.