Background: Simulation models are increasingly used to inform health policy. We provide an overview of applications of simulation models in health policy, analyze the use of best reporting practices, and assess the reproducibility of existing studies.
Method: Studies that used simulation modeling as the core method to address any health policy questions were included. Health policy domain distribution and changes in quality over time were well-characterized using MeSH terms and model characteristics, respectively. Reproducibility was assessed using predefined, categorical criteria.
Findings: 1,613 studies were analyzed. We found an exponential growth in the number of studies over the past half century, with the highest growth in dynamic modeling approaches. The largest subset of studies is focused on disease policy models (70%), within which pathological conditions, viral diseases, neoplasms, and cardiovascular diseases account for one-third of the articles. Nearly half of the studies do not report the details of their models. A subset of 100 articles (50 highly cited and 50 random) were selected to analyze in-depth criteria for reporting quality and reproducibility. Significant gaps between best modeling practices could be found in both the random and highly cited samples; only seven of 26 in-depth evaluation criteria were satisfied by more than 80% of samples. We found no evidence that the highly cited samples adhered better to the modeling best practices.
Interpretation: Our results suggest crucial areas for increased applications of simulation modeling, and opportunities to enhance the rigor and documentation in the conduct and reporting of simulation modeling in health policy.