A review on the relation between simulation and improvement in hospitals

BACKGROUND: Simulation applications on operations management in hospitals are frequently published and claim to support decision-making on operations management subjects. However, the reported implementation rates of recommendations are low and the actual impact of the changes recommended by the modeler has hardly been examined. This paper examines: 1) the execution rate of simulation study recommendations, 2) the research methods used to evaluate implementation of recommendations, 3) factors contributing to implementation, and 4) the differences regarding implementation between literature and practice. RESULTS: Altogether 16 hospitals executed the recommendations (at least partially). Implementation results were hardly reported upon; 1 study described a before-and-after design, 2 a partial before and after design. Factors that help implementation were grouped according to 1) technical quality, of which data availability, validation/verification with historic data/expert opinion, and the development of the conceptual model were mentioned most frequently 2) process quality, with client involvement and 3) outcome quality with, presentation of results. The survey response rate of traceable authors was 61%, 18 authors implemented the results at least partially. Among these responses, evaluation methods were relatively better with 3 time series designs and 2 before-and-after designs. CONCLUSIONS: Although underreported in literature, implementation of recommendations seems limited; this review provides recommendations on project design, implementation conditions and evaluation methods to increase implementation. METHODS: A literature review in PubMed and Business Source Elite on stochastic simulation applications on operations management in individual hospitals published between 1997 and 2008. From those reporting implementation, cross references were added. In total, 89 papers were included. A scoring list was used for data extraction. Two reviewers evaluated each paper separately; in case of discrepancies, they jointly determined the scores. The findings were validated with a survey to the original authors.