External validation and update of prediction models for unfavorable outcomes in hospitalized patients with COVID-19 in Japan

INTRODUCTION: Most of the currently used prognostic models for COVID-19 are based on Western cohorts, but it is unknown whether any are applicable to patients with COVID-19 in Japan. METHODS: This retrospective cohort study included 160 patients with COVID-19 who were admitted to the National Center for Global Health and Medicine between January 26, 2020 and July 25, 2020. We searched PubMed for prognostic models for COVID-19. The predicted outcome was initiation of respiratory support or death. Performance of the candidate models was evaluated according to discrimination and calibration. We recalibrated the intercept of each model with our data. We also updated each model by adding β2-microglobulin (β2MG) to the model and recalculating the intercept and the coefficient of β2MG. RESULTS: Mean patient age was 49.8 years, 68% were male, 88.7% were Japanese. The study outcomes occurred in 15 patients, including two deaths. Two-hundred sixty-nine papers were screened, and four candidate prognostic models were assessed. The model of Bartoletti et al. had the highest area under receiver operating characteristic curve (AUC) (0.88; 95% confidence interval 0.81–0.96). All four models overestimated the probability of occurrence of the outcome. None of the four models showed statistically significant improvement in AUCs by adding β2MG. CONCLUSIONS: Our results suggest that the existing prediction models for COVID-19 overestimate the probability of occurrence of unfavorable outcomes in a Japanese cohort. When applying a prediction model to a different cohort, it is desirable to evaluate its performance according to the prevalent health situation in that region.