Identifying best modelling practices for tobacco control policy simulations: a systematic review and a novel quality assessment framework

BACKGROUND: Policy simulation models (PSMs) have been used extensively to shape health policies before real-world implementation and evaluate post-implementation impact. This systematic review aimed to examine best practices, identify common pitfalls in tobacco control PSMs and propose a modelling quality assessment framework. METHODS: We searched five databases to identify eligible publications from July 2013 to August 2019. We additionally included papers from Feirman et al for studies before July 2013. Tobacco control PSMs that project tobacco use and tobacco-related outcomes from smoking policies were included. We extracted model inputs, structure and outputs data for models used in two or more included papers. Using our proposed quality assessment framework, we scored these models on population representativeness, policy effectiveness evidence, simulated smoking histories, included smoking-related diseases, exposure-outcome lag time, transparency, sensitivity analysis, validation and equity. FINDINGS: We found 146 eligible papers and 25 distinct models. Most models used population data from public or administrative registries, and all performed sensitivity analysis. However, smoking behaviour was commonly modelled into crude categories of smoking status. Eight models only presented overall changes in mortality rather than explicitly considering smoking-related diseases. Only four models reported impacts on health inequalities, and none offered the source code. Overall, the higher scored models achieved higher citation rates. CONCLUSIONS: While fragments of good practices were widespread across the reviewed PSMs, only a few included a ‘critical mass’ of the good practices specified in our quality assessment framework. This framework might, therefore, potentially serve as a benchmark and support sharing of good modelling practices.