VOI for Clinicians and Funders: An Accessible Approach to Prioritize Research Aims
with Laura Hatfield
Value of information (VOI) analyses can optimize research priorities and investments. Recent methods developments have decreased computational burden and extended VOI to new areas, for example, value of implementation and value of personalized information. Here, we propose a simpler alternative to VOI designed to be accessible to researchers, clinicians, and funders who lack access to sophisticated disease models. We apply the method to prioritize research into heterogeneous treatment effects. The question is whether to invest in new randomized controlled trials that will estimate treatment effects in specific patient subpopulations.
We provide general conclusions for choosing candidate effect modifiers for future research, with inputs that include population prevalence, the underlying true magnitude of effect modification, and the current strength of existing evidence. We find that a study that is optimal based on societal value may not be powered based on frequentist minimum required sample sizes. Further we show that clinical decision making benchmarked on only ‘positive’ results with a p-value below 0.05, reduces the value of additional research. Interestingly, defining value as health versus net monetary benefit can both create superior value depending on the subpopulation differential health and cost outcomes of each treatment.
For all the advances in VOI, the method still relies on substantial investment in sophisticated clinical models. Important gatekeepers of research and funding outside of decision science may find this a barrier to implementation of VOI. Our strategy is simple and flexible across clinical contexts while still providing guidance on the relative value of potential research priorities.
Value of (Biased) Information
with Laura Hatifeld and Ariel Stern
Value of information typically assumes that current and future observed patient outcomes are random samples from the same underlying population. However, in many contexts, that assumption may be violated. For example, in the Food and Drug Administration’s accelerated approval pathway, drugs are initially approved on proxy outcomes (like tumor size). Proxy outcomes are typically easier to demonstrate a drug-response compared to patient survival. Importantly, regulators have the authority to stipulate outcomes in post-market studies for accelerated approval drugs. Here we ask: What is the expected difference in the value of sample information when an optimal follow-up study reports a proxy vs. true outcome of interest?
We conducted a VOI analysis to estimate the expected net monetary benefit of a future study measuring a proxy outcome compared to a true outcome of interest. The proxy is biased in favor of a treatment effect. In our simulated scenarios, we consider cases where initial and future evidence is both unbiased and biased.
The majority of post-market studies of accelerated approval drugs report proxy outcomes. Our results suggest this is a sub-optimal regulatory strategy. Although this work is framed in terms of data collection on proxy versus true outcomes, the results have implications across VOI analyses. In any context where study results are potentially biased population samples, VOI results that do not account for information bias may be inaccurate.
Health Benefit Package Design: Integrating Multiple Objectives with Mathematical Optimization
with David Watkins, Solomon T. Memirie, Joshua Salomon, and Stéphane Verguet
Policymakers grapple with what health interventions to publicly fund across health systems globally. Here, we propose mathematical optimization as a tool to prioritize health services for inclusion in a publicly financed essential health benefits package. Our approach can formally and flexibly balance tradeoffs across dimensions like the expected health and financial risk protection of each candidate health service.
We consider 23 health interventions across 40 disease and age categories in the context of Ethiopia, an illustrative country setting. The optimal subset of interventions to include in an essential benefits package is determined with bi-criterion integer programming conditional on a budget constraint. To approximate health and financial risk protection benefits we estimate the number of deaths and cases of catastrophic health expenditure averted when an intervention is included in the essential health benefits package.
Rarely are the same health interventions the ‘best buys’ across the dimensions of population health benefit and financial risk protection. We address this tension by incorporating both deaths and catastrophic health expenditures averted as joint optimization objectives. The optimal essential health benefit package depends on the relative importance of deaths averted compared to catastrophic health expenditures averted. We demonstrate the implications of differential objective weights to simulate real-world policy decision making.