Can medical practitioners rely on prediction models for COVID-19? A systematic review

Aim This systematic review sought to assess and scrutinise the validity and practicality of published and preprint reports of prediction models for the diagnosis of coronavirus disease 2019 (COVID-19) in patients with suspected infection, for prognosis of patients with COVID-19, and for identifying individuals in the general population at increased risk of infection with COVID-19 or being hospitalised with the illness. Data sources A systematic, online search was conducted in PubMed and Embase. In order to do so, the authors used Ovid as the host platform for these two databases and also investigated bioRxiv, medRxiv and arXiv as repositories for the preprints of studies. A public living systematic review list of COVID-19-related studies was used as the baseline searching platform (Institute of Social and Preventive Medicine's repository for living evidence on COVID-19). Study selection Studies which developed or validated a multivariable prediction model related to COVID-19 patients' data (individual level data) were included. The authors did not put any restrictions on the models included in their study regarding the model setting, prediction horizon or outcomes. Data extraction and synthesis Checklists of critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS) and prediction model risk of bias assessment tool (PROBAST) were used to guide developing of a standardised data extraction form. Each model's predictive performance was extracted by using any summaries of discrimination and calibration. All these steps were done according to the aspects of the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) and preferred reporting items for systematic reviews and meta-analyses (PRISMA). Results One hundred and forty-five prediction models (107 studies) were selected for data extraction and critical appraisal. The most common predictors of diagnosis and prognosis of COVID-19 were age, body temperature, lymphocyte count and lung imaging characteristics. Influenza-like symptoms and neutrophil count were regularly predictive in diagnostic models, while comorbidities, sex, C-reactive protein and creatinine were common prognostic items. C-indices (a measure of discrimination for models) ranged from 0.73 to 0.81 in prediction models for the general population, from 0.65 to more than 0.99 in diagnostic models, and from 0.68 to 0.99 in the prognostic models. All the included studies were reported to have high risks of bias. Conclusions Overall, this study did not recommend applying any of the predictive models in clinical practice yet. High risk of bias, reporting problems and (probably) optimistic reported performances are all among the reasons for the previous conclusion. Prompt actions regarding accurate data sharing and international collaborations are required to achieve more rigorous prediction models for COVID-19.