Working Paper
Barnard-Mayers RM, Childs E, Corlin L, Caniglia E, Fox M, Donnelly JP, Murray EJ. Assessing Knowledge, Attitudes, and Practices towards Causal Directed Acyclic Graphs among Epidemiologists and Medical Researchers: a qualitative research project. medRxiv. Working Paper. Publisher's VersionAbstract
Background: Estimating the strength of causal effects is an important component of epidemiologic research, and causal graphs provide a key tool for optimizing the validity of these effect estimates. Although a large literature exists on the mathematical theory underlying the use of causal graphs, including directed acyclic graphs, to assess and describe causal assumptions, and translate these assumptions into appropriate statistical analysis plans, less literature exists to aid applied researchers in understanding how best to develop and use causal graphs in their research projects. Objective We sought to understand this gap by surveying practicing epidemiologists and medical researchers on their knowledge, level of interest, attitudes, and practices towards the use of causal graphs in applied epidemiology and health research. Methods We conducted an anonymous survey of self-identified epidemiology and health researchers via Twitter and via the Society of Epidemiologic Research membership listserv. The survey was conducted using Qualtrics and asked a series of multiple choice and open-ended questions about causal graphs. Results In total, 439 responses were collected. Overall, 62% reported being comfortable with using causal graphs, and 60% reported using them 9sometimes9, 9often9, or 9always9 in their research. About 70% of respondents had received formal training on causal graphs (typically causal directed acyclic graphs). Having received any training appeared to improve comprehension of the underlying assumptions of causal graphs. Forty percent of respondents who did not use causal graphs reported lack of knowledge …
Murray EJ, Swanson SS, Hernan MA. Guidelines for estimating causal effects in pragmatic randomized trials. arXiv preprint arXiv:1911.06030. Working Paper. Publisher's VersionAbstract
Pragmatic randomized trials are designed to provide evidence for clinical decision-making rather than regulatory approval. Common features of these trials include the inclusion of heterogeneous or diverse patient populations in a wide range of care settings, the use of active treatment strategies as comparators, unblinded treatment assignment, and the study of long-term, clinically relevant outcomes. These features can greatly increase the usefulness of the trial results for patients, clinicians, and other stakeholders. However, these features also introduce an increased risk of non-adherence, which reduces the value of the intention-to-treat effect as a patient-centered measure of causal effect. In these settings, the per-protocol effect provides useful complementary information for decision making. Unfortunately, there is little guidance for valid estimation of the per-protocol effect. Here, we present our full guidelines for analyses of pragmatic trials that will result in more informative causal inferences for both the intention-to-treat effect and the per-protocol effect.
Caniglia EC, Murray EJ, Hernan MA, Shahn Z. A Note on Estimating Optimal Dynamic Treatment Strategies Under Resource Constraints Using Dynamic Marginal Structural Models. arXiv preprint arXiv:1903.06488. Working Paper. Publisher's VersionAbstract
Existing strategies for determining the optimal treatment or monitoring strategy typically assume unlimited access to resources. However, when a health system has resource constraints, such as limited funds, access to medication, or monitoring capabilities, medical decisions must balance impacts on both individual and population health outcomes. That is, decisions should account for competition between individuals in resource usage. One simple solution is to estimate the (counterfactual) resource usage under the possible interventions and choose the optimal strategy for which resource usage is within acceptable limits. We propose a method to identify the optimal dynamic intervention strategy that leads to the best expected health outcome accounting for a health system's resource constraints. We then apply this method to determine the optimal dynamic monitoring strategy for people living with HIV when resource limits on monitoring exist using observational data from the HIV-CAUSAL Collaboration.
Tennant PWG, Harrison WJ, Murray EJ, Arnold KF, Berrie L, Fox MP, Gadd SC, Keeble C, Ranker LR, Textor J, et al. Use of directed acyclic graphs (DAGs) in applied health research: review and recommendations. medRxiv. Working Paper. Publisher's VersionAbstract
BACKGROUND: Directed acyclic graphs (DAGs) are an increasingly popular approach for identifying confounding variables that require adjustment when estimating causal effects. This review examined the use of DAGs in applied health research to inform recommendations for improving their transparency and utility in future research. METHODS: Original health research articles published during 1999-2017 mentioning "directed acyclic graphs" or similar or citing DAGitty were identified from Scopus, Web of Science, Medline, and Embase. Data were extracted on the reporting of: estimands, DAGs, and adjustment sets, alongside the characteristics of each article's largest DAG. RESULTS: A total of 234 articles were identified that reported using DAGs. A fifth (n=48, 21%) reported their target estimand(s) and half (n=115, 48%) reported the adjustment set(s) implied by their DAG(s). Two-thirds of the articles (n=144, 62%) made at least one DAG available. Diagrams varied in size but averaged 12 nodes (IQR: 9-16, range: 3-28) and 29 arcs (IQR: 19-42, range: 3-99). The median saturation (i.e. percentage of total possible arcs) was 46% (IQR: 31-67, range: 12-100). 37% (n=53) of the DAGs included unobserved variables, 17% (n=25) included super-nodes (i.e. nodes containing more than one variable, and a 34% (n=49) were arranged so the constituent arcs flowed in a consistent direction. CONCLUSIONS: There is substantial variation in the use and reporting of DAGs in applied health research. Although this partly reflects their flexibility, it also highlight some potential areas for improvement. This review hence offers several recommendations to improve the reporting and use of DAGs in future research.
In Press
Murray EJ, Claggett BL, Granger B, Solomon SD, Hernan MA. Adherence-adjustment in placebo-controlled randomized trials: An application to the candesartan in heart failure randomized trial. Contemporary Clinical Trials. In Press. Publisher's VersionAbstract


The per-protocol effect provides important information in randomized trials with incomplete adherence. Yet, because valid estimation typically requires adjustment for prognostic factors that predict adherence, per-protocol effect estimates are often met with skepticism. In placebo-controlled trials, however, the validity of adjustment can be indirectly verified by demonstrating no association between adherence and the outcome among the placebo arm. Here, we describe a two-stage procedure in which we first adjust for time-varying adherence in the placebo arm and then use a similar procedure to estimate the per-protocol effect.


We use the Candesartan in Heart Failure: Assessment of Reduction in Mortality and Morbidity (CHARM) randomized trial. First, we compare adherers versus non-adherers in the placebo arm, adjusting for pre- and post-randomization variables. Second, we use models validated in the placebo arm to estimate the per-protocol effect of adherence to candesartan versus placebo in the full trial.


We successfully estimated no association between adherence and mortality in the placebo arm; hazard ratio: 0∙91 (95% CI: 0∙51, 2∙52). We then estimated the per-protocol effect under two sets of protocol-defined stopping criteria after adjustment for post-randomization confounders. The mortality hazard ratio estimates ranged from 0.91 to 0.93 for the per-protocol effect estimates, similar to the intention-to-treat effect estimates.


Adherence adjustment in the CHARM trial is feasible when appropriate assumptions about missing data and confounding are made. These assumptions cannot be verified but can be supported through the use of placebo-arm adherence assessment.
Murray EJ, Farland LV, Caniglia EC, Dorans KS, DuPre NC, Hughes KC, Kim IY, Pernar CH, Tanz LJ, Zack RM. Is this a Portrait of John Graunt? An Art History Mystery. American Journal of Epidemiology. In Press. Publisher's Version
Murray EJ, Robins JM, III GSR, Freedberg KA, Hernan MA. The Challenges of Parameterizing Direct Effects in Individual-Level Simulation Models. Medical Decision Making. 2020;40 (1) :106-111. Publisher's Version
Kuehne F, Jahn B, Conrads-Frank A, Bundo M, Arvandi M, Endel F, Popper N, Endel G, Urach C, Gyimesi M, et al. Guidance for a causal comparative effectiveness analysis emulating a target trial based on big real world evidence: when to start statin treatment. J Comp Eff Res. 2019;8 (12) :1013-1025.Abstract
The aim of this project is to describe a causal (counterfactual) approach for analyzing when to start statin treatment to prevent cardiovascular disease using real-world evidence. We use directed acyclic graphs to operationalize and visualize the causal research question considering selection bias, potential time-independent and time-dependent confounding. We provide a study protocol following the 'target trial' approach and describe the data structure needed for the causal assessment. The study protocol can be applied to real-world data, in general. However, the structure and quality of the database play an essential role for the validity of the results, and database-specific potential for bias needs to be explicitly considered.
Young JG, Vatsa R, Murray EJ, Hernán MA. Interval-cohort designs and bias in the estimation of per-protocol effects: a simulation study. Trials. 2019;20 (1) :552.Abstract
BACKGROUND: Randomized trials are considered the gold standard for making inferences about the causal effects of treatments. However, when protocol deviations occur, the baseline randomization of the trial is no longer sufficient to ensure unbiased estimation of the per-protocol effect: post-randomization, time-varying confounders must be sufficiently measured and adjusted for in the analysis. Given the historical emphasis on intention-to-treat effects in randomized trials, measurement of post-randomization confounders is typically infrequent. This may induce bias in estimates of the per-protocol effect, even using methods such as inverse probability weighting, which appropriately account for time-varying confounders affected by past treatment. METHODS/DESIGN: In order to concretely illustrate the potential magnitude of bias due to infrequent measurement of time-varying covariates, we simulated data from a very large trial with a survival outcome and time-varying confounding affected by past treatment. We generated the data such that the true underlying per-protocol effect is null and under varying degrees of confounding (strong, moderate, weak). In the simulated data, we estimated per-protocol survival curves and associated contrasts using inverse probability weighting under monthly measurement of the time-varying covariates (which constituted complete measurement in our simulation), yearly measurement, as well as 3- and 6-month intervals. RESULTS: Using inverse probability weighting, we were able to recover the true null under the complete measurement scenario no matter the strength of confounding. Under yearly measurement intervals, the estimate of the per-protocol effect diverged from the null; inverse probability weighted estimates of the per-protocol 5-year risk ratio based on yearly measurement were 1.19, 1.12, and 1.03 under strong, moderate, and weak confounding, respectively. Bias decreased with measurement interval length. Under all scenarios, inverse probability weighted estimators were considerably less biased than a naive estimator that ignored time-varying confounding completely. CONCLUSIONS: Bias that arises from interval measurement designs highlights the need for planning in the design of randomized trials for collection of time-varying covariate data. This may come from more frequent in-person measurement or external sources (e.g., electronic medical record data). Such planning will provide improved estimates of the per-protocol effect through the use of methods that appropriately adjust for time-varying confounders.
Caniglia EC, Zash R, Swanson SA, Wirth KE, Diseko M, Mayondi G, Lockman S, Mmalane M, Makhema J, Dryden-Peterson S, et al. Methodological Challenges When Studying Distance to Care as an Exposure in Health Research. Am J Epidemiol. 2019;188 (9) :1674-1681. Publisher's VersionAbstract
Distance to care is a common exposure and proposed instrumental variable in health research, but it is vulnerable to violations of fundamental identifiability conditions for causal inference. We used data collected from the Botswana Birth Outcomes Surveillance study between 2014 and 2016 to outline 4 challenges and potential biases when using distance to care as an exposure and as a proposed instrument: selection bias, unmeasured confounding, lack of sufficiently well-defined interventions, and measurement error. We describe how these issues can arise, and we propose sensitivity analyses for estimating the degree of bias.
Murray EJ, Caniglia EC, Swanson SA, Hernández-Díaz S, Hernán MA. Patients and investigators preferred measures of absolute risk in subgroups for pragmatic randomized trials. J Clin Epidemiol. 2018. Publisher's VersionAbstract
OBJECTIVES: Pragmatic randomized trials are important tools for shared decision-making, but no guidance exists on patients' preferences for types of causal information. We aimed to assess preferences of patients and investigators towards causal effects in pragmatic randomized trials. STUDY DESIGN AND SETTING: We: (a) held 3 focus groups with patients (n=23) in Boston, MA; (b) surveyed (n=12) and interviewed (n=5) investigators with experience conducting pragmatic trials; and (c) conducted a systematic literature review of pragmatic trials (n=63). RESULTS: Patients were distrustful of new-to-market medications unless substantially more effective than existing choices, preferred stratified absolute risks, and valued adherence-adjusted analyses when they expected to adhere. Investigators wanted both intention-to-treat and per-protocol effects, but felt methods for estimating per-protocol effects were lacking. When estimating per-protocol effects, many pragmatic trials used inappropriate methods to adjust for adherence and loss to follow-up. CONCLUSION: We make 4 recommendations for pragmatic trials to improve patient centeredness: (1) focus on superiority in effectiveness or safety, rather than noninferiority; (2) involve patients in specifying a priori subgroups; (3) report absolute measures of risk, and (4) complement intention-to-treat effect estimates with valid per-protocol effect estimates.
Murray EJ, Hernan MA. Improved adherence adjustment in the Coronary Drug Project. Trials. 2018;19 :158. Publisher's VersionAbstract


The survival difference between adherers and non-adherers to placebo in the Coronary Drug Project has been used to support the thesis that adherence adjustment in randomized trials is not generally possible and, therefore, that only intention-to-treat analyses should be trusted. We previously demonstrated that adherence adjustment can be validly conducted in the Coronary Drug Project using a simplistic approach. Here, we re-analyze the data using an approach that takes full advantage of recent methodological developments.


We used inverse-probability weighted hazards models to estimate the 5-year survival and mortality risk when individuals in the placebo arm of the Coronary Drug Project adhere to at least 80% of the drug continuously or never during the 5-year follow-up period.


Adjustment for post-randomization covariates resulted in 5-year mortality risk difference estimates ranging from − 0.7 (95% confidence intervals (CI), − 12.2, 10.7) to 4.5 (95% CI, − 6.3, 15.3) percentage points.


Our analysis confirms that appropriate adjustment for post-randomization predictors of adherence largely removes the association between adherence to placebo and mortality originally described in this trial.

Murray EJ, Robins JM, Seage GR, Lodi S, Hyle EP, Reddy KP, Freedberg KA, Hernán MA. Using observational data to calibrate simulation models. Med Decis Making. 2018;38 (2) :212-24. Publisher's VersionAbstract
BACKGROUND: Individual-level simulation models are valuable tools for comparing the impact of clinical or public health interventions on population health and cost outcomes over time. However, a key challenge is ensuring that outcome estimates correctly reflect real-world impacts. Calibration to targets obtained from randomized trials may be insufficient if trials do not exist for populations, time periods, or interventions of interest. Observational data can provide a wider range of calibration targets but requires methods to adjust for treatment-confounder feedback. We propose the use of the parametric g-formula to estimate calibration targets and present a case-study to demonstrate its application. METHODS: We used the parametric g-formula applied to data from the HIV-CAUSAL Collaboration to estimate calibration targets for 7-y risks of AIDS and/or death (AIDS/death), as defined by the Center for Disease Control and Prevention under 3 treatment initiation strategies. We compared these targets to projections from the Cost-Effectiveness of Preventing AIDS Complications (CEPAC) model for treatment-naïve individuals presenting to care in the following year ranges: 1996 to 1999, 2000 to 2002, or 2003 onwards. RESULTS: The parametric g-formula estimated a decreased risk of AIDS/death over time and with earlier treatment. The uncalibrated CEPAC model successfully reproduced targets obtained via the g-formula for baseline 1996 to 1999, but over-estimated calibration targets in contemporary populations and failed to reproduce time trends in AIDS/death risk. Calibration to g-formula targets improved CEPAC model fit for contemporary populations. CONCLUSION: Individual-level simulation models are developed based on best available information about disease processes in one or more populations of interest, but these processes can change over time or between populations. The parametric g-formula provides a method for using observational data to obtain valid calibration targets and enables updating of simulation model inputs when randomized trials are not available.
Murray EJ, Robins JM, George R. Seage III, Freedberg KA, Hernan MA. A comparison of agent-based models and the parametric g-formula for causal inference. American Journal of Epidemiology. 2017;186 (2) :131-42. Publisher's VersionAbstract
Decision-making requires choosing from treatments on the basis of correctly estimated outcome distributions under each treatment. In the absence of randomized trials, 2 possible approaches are the parametric g-formula and agent-based models (ABMs). The g-formula has been used exclusively to estimate effects in the population from which data were collected, whereas ABMs are commonly used to estimate effects in multiple populations, necessitating stronger assumptions. Here, we describe potential biases that arise when ABM assumptions do not hold. To do so, we estimated 12-month mortality risk in simulated populations differing in prevalence of an unknown common cause of mortality and a time-varying confounder. The ABM and g-formula correctly estimated mortality and causal effects when all inputs were from the target population. However, whenever any inputs came from another population, the ABM gave biased estimates of mortality—and often of causal effects even when the true effect was null. In the absence of unmeasured confounding and model misspecification, both methods produce valid causal inferences for a given population when all inputs are from that population. However, ABMs may result in bias when extrapolated to populations that differ on the distribution of unmeasured outcome determinants, even when the causal network linking variables is identical.
Murray EJ, Hernán MA. Adherence adjustment in the Coronary Drug Project: A call for better per-protocol effect estimates in randomized trials. Clinical Trials. 2016;13 (4) :372-8. Publisher's VersionAbstract
Background: In many randomized controlled trials, patients and doctors are more interested in the per-protocol effect than in the intention-to-treat effect. However, valid estimation of the per-protocol effect generally requires adjustment for prognostic factors associated with adherence. These adherence adjustments have been strongly questioned in the clinical trials community, especially after 1980 when the Coronary Drug Project team found that adherers to placebo had lower 5-year mortality than non-adherers to placebo.Methods: We replicated the original Coronary Drug Project findings from 1980 and re-analyzed the Coronary Drug Project data using technical and conceptual developments that have become established since 1980. Specifically, we used logistic models for binary outcomes, decoupled the definition of adherence from loss to follow-up, and adjusted for pre-randomization covariates via standardization and for post-randomization covariates via inverse probability weighting.Results: The original Coronary Drug Project analysis reported a difference in 5-year mortality between adherers and non-adherers in the placebo arm of 9.4 percentage points. Using modern approaches, we found that this difference was reduced to 2.5 (95% confidence interval: −2.1 to 7.0).Conclusion: Valid estimation of per-protocol effects may be possible in randomized clinical trials when analysts use appropriate methods to adjust for post-randomization variables.
White MI, Wagner SL, Schultz IZ, Murray E, Bradley SM, Hsu V, McGuire L, Schulz W. Non-modifiable worker and workplace risk factors contributing to workplace absence: A stakeholder-centred synthesis of systematic reviews. Work. 2015;52 (2) :353-73.Abstract
BACKGROUND: Workplace stakeholders report the identification and translation of relevant high quality research to inform workplace disability policy and practice is a challenge. The present study engaged academic and community stakeholders in conducting a best evidence-synthesis to identify non-modifiable risk and protective worker and workplace factors impacting work-related absence across a variety of health conditions. OBJECTIVE: To identify non-modifiable worker and workplace disability risk and protective factors impacting work-related absence across common health conditions. METHODS: The research team searched Medline, Embase, CINAHL, The Cochrane Library, PsycINFO, BusinessSource-Complete, and ABI/Inform from 2000 to 2011. Quantitative, qualitative, or mixed methods systematic reviews of work-focused population were considered for inclusion. Two or more reviewers independently reviewed articles for inclusion and methodological screening. RESULTS: The search strategy, including expert input and grey literature, led to the identification of 2,467 unique records. From this initial search, 2325 were eliminated by title or abstract review, 142 articles underwent comprehensive review to assess for inclusion, 26 systematic reviews met eligibility criteria for this synthesis. For non-modifiable worker and workplace factors we found consistent evidence across two or more health conditions for increased risk of disability in situations where workers experience lower education, older age, emotional distress, poor personal functioning, decreased physical functioning, psychological symptoms, overweight status, and greater sick leave history. LIMITATIONS: Heterogeneity of existing literature due to differences in outcome measures, definitions and research designs limited ability to assess effect size and results reflect findings limited to English-language papers.
Wagner S, White M, Schultz I, Murray E, Bradley SM, Hsu V, McGuire L, Schulz W. Modifiable worker risk factors contributing to workplace absence: a stakeholder-centred best-evidence synthesis of systematic reviews. Work. 2014;49 (4) :541-58.Abstract
BACKGROUND: A challenge facing stakeholders is the identification and translation of relevant high quality research to inform policy and practice. This study engaged academic and community stakeholders in conducting a best evidence-synthesis to identify modifiable risk and protective worker factors across health conditions impacting work-related absence. OBJECTIVES: To identify modifiable worker disability risk and protective factors across common health conditions impacting work-related absence. METHODS: We searched Medline, Embase, CINHAL, The Cochrane Library, PsycINFO, BusinessSourceComplete, and ABI/Inform from 2000 to 2011. Quantitative, qualitative, or mixed methods systematic reviews of work-focused population were considered for inclusion. Two or more reviewers independently reviewed articles for inclusion and methodological screening. RESULTS: The search strategy, expert input and grey literature identified 2,467 unique records. One hundred and forty-two full text articles underwent comprehensive review. Twenty-four systematic reviews met eligibility criteria. Modifiable worker factors found to have consistent evidence across two or more health conditions included emotional distress, negative enduring psychology/personality factors, negative health and disability perception, decreased physical activity, lack of family support, poor general health, increased functional disability, increased pain, increased fatigue and lack of motivation to return to work. CONCLUSIONS: Systematic reviews are limited by availability of high quality studies, lack of consistency of methodological screening and reporting, and variability of outcome measures used.
Astrakianakis G, Murray E. Conflicting Effects of Occupational Endotoxin Exposure on Lung Health-A Hypothesis-Generating Review of Cancer and COPD Risk. Journal of Environmental Immunology and Toxicology. 2013;1 :128–139.
Murray E, Franche R-L, Ibrahim S, Smith P, Carnide N, Côté P, Gibson J, Guzman J, Koehoorn M, Mustard C. Pain-related work interference is a key factor in a worker/workplace model of work absence duration due to musculoskeletal conditions in Canadian nurses. J Occup Rehabil. 2013;23 (4) :585-96.Abstract
OBJECTIVE: To examine the role of pain experiences in relation to work absence, within the context of other worker health factors and workplace factors among Canadian nurses with work-related musculoskeletal (MSK) injury. METHODS: Structural equation modeling was used on a sample of 941 employed, female, direct care nurses with at least one day of work absence due to a work-related MSK injury, from the cross-sectional 2005 National Survey of the Work and Health of Nurses. RESULTS: The final model suggests that pain severity and pain-related work interference mediate the impact of the following worker health and workplace factors on work absence duration: depression, back problems, age, unionization, workplace physical demands and low job control. The model accounted for 14 % of the variance in work absence duration and 46.6 % of the variance in pain-related work interference. CONCLUSIONS: Our findings support a key role for pain severity and pain-related work interference in mediating the effects of workplace factors and worker health factors on work absence duration. Future interventions should explore reducing pain-related work interference through addressing workplace issues, such as providing modified work, reducing physical demands, and increasing job control.
White M, Wagner S, Schultz IZ, Murray E, Bradley SM, Hsu V, McGuire L, Schulz W. Modifiable workplace risk factors contributing to workplace absence across health conditions: A stakeholder-centered best-evidence synthesis of systematic reviews. Work. 2013;45 (4) :475-92.Abstract
BACKGROUND: A challenge facing stakeholders is the identification and translation of relevant high quality research to inform policy and practice. This study engaged academic and community stakeholders in conducting a best evidence-synthesis to enhance knowledge use. OBJECTIVES: To identify modifiable workplace disability risk and protective factors across common health conditions impacting work-related absence. METHODS: We searched MEDLINE, Embase, CINHAL, The Cochrane Library, PsycINFO, BusinessSourceComplete, and ABI/Inform from 2000 to 2011. Systematic reviews that employed quantitative, qualitative, or mixed methods of work-focused population were considered for inclusion. Two or more independent reviewers reviewed titles only, titles and abstracts, and/or full articles when assessing eligibility for inclusion. Selected articles underwent methodological screening. RESULTS: The search strategy, expert input and grey literature identified 2,467 unique records from which 142 full text articles underwent comprehensive review. Twenty-seven systematic reviews met eligibility criteria. Modifiable work factors found to have consistent evidence across two or more health conditions included lack of social support, increased physical demands at work, job strain, lack of supervisory support, increased psychological demands, low job satisfaction, low worker control of job, and poor leadership quality. CONCLUSIONS: The active engagement of stakeholders led to greater understanding of relevance of the study findings for community stakeholders and appreciation of the mutual benefits of collaboration.