Mathematical modelling of COVID-19: a systematic review and quality assessment in the early epidemic response phase

PURPOSE: Epidemiological models have played a key role in informing national response strategies for the current COVID-19 pandemic. We aimed to identify how mathematical models were employed in the early phase of the pandemic, at a time of great epidemiological uncertainty, as well as to formally assess the quality of models used. Hence we aimed to identify areas for improvement in model-based decision-making for future unknown disease threats. METHODS & MATERIALS: A systematic review of mathematical modelling studies estimating the epidemiological impact of COVID-19 (risk of importation/spread) and non-pharmaceutical interventions (NPI) was conducted. We systematically searched PubMed, Embase, and preprints in ARxiv, MedRxiv and bioRxiv. We adopted two published quality assessment frameworks to formally assess the extent in which modelling studies met minimal requirements for incorporation of uncertainty and good modelling practice. RESULTS: In total, 166 articles met our eligibility criteria. The vast majority (129 studies, 78%) of models evaluated the effectiveness NPIs. NPI effectiveness was predominantly modelled in China and Italy, but varied by global region. Asian studies largely evaluated the impact of quarantine and isolation (64 studies), whereas European modelling studies modelled the impact of containment (15 studies), quarantine of travellers, and the isolation of cases. Early models primarily concerned compartmental, deterministic frameworks using SEIR or variants of SEIR compartments (93 studies, 56%) assuming homogenous, symptomatic transmission. Incorporation of parameter uncertainty through model calibration (inference of unknown parameters by fitting models to data) and sensitivity analyses were relatively common (66% and 56% of studies respectively), the former mainly using Chinese data. In contrast, inclusion of structural uncertainty (uncertainty in disease characteristics) was relatively uncommon, as was validation of model output to external data. CONCLUSION: This work allows for the identification of existing challenges in the mathematical modelling of emerging diseases, and emphasises minimal criteria for enhancing reliable model estimation and reporting. Limited availability of epidemiological data in the early phase of a new disease treat challenges model calibration to local, and validation to external data, emphasising the critical importance of enforcing standardised protocols for early epi-data collection, and raising awareness among modellers and decision-makers alike in handling uncertainty.