Working Paper
Stafford C, Marrero W, Naumann RB, Lich KH, Wakeman S, Jalali MS. Predictors of Premature Discontinuation of Opioid Use Disorder Treatment in the United States. Working Paper;Abstract

Importance: Over the last few decades, opioid use, use disorder, and overdose have increased. In 2019, nearly 50,000 people died in the U.S. due to an opioid overdose, and 1.6 million people had an opioid use disorder (OUD). Despite investments and decreases in barriers to treatment, retention in OUD treatment remains a challenge. Given increases in substance use and misuse, especially during the COVID-19 pandemic, understanding risk factors for OUD treatment discontinuation remains a critical priority.

Objective: To identify key risk factors of premature treatment exit and develop a predictive model.

Design, Setting, Participants: In this prognostic study, we built a machine learning model using the Treatment Episode Data Set – Discharge (TEDS-D). Included were 2,446,710 separate treatment episodes for individuals in the U.S. discharged between January 1, 2015 (the first year with desired factors) and December 31, 2018 (the most recent available data).

Exposures: 32 potential risk factors, including treatment characteristics, substance use history, socioeconomic status, and demographic characteristics.

Main Outcome and Measure: Premature OUD treatment discontinuation, defined as a reason for discharge of “dropped out of treatment (left against professional advice).” Models were assessed using the area under the receiver operating characteristic curve (AUC).

Results: The AUC of our machine learning model was 75% with an accuracy of 74%. The most influential risk factors included characteristics of service setting, geographic region, primary source of payment, referral source, and health insurance status. Importantly, several factors previously reported as influential predictors, such as age, living situation, age of first substance use, race and ethnicity, and sex had far weaker predictive impacts.

Conclusions and Relevance: The risk of premature OUD treatment discontinuation was impacted by several factors that may be used to rethink treatment models for OUD.

Social Vulnerability Increases Drug Poisoning Mortality in the United States: 2014-2018. Working Paper;
Garcia G-GP, Dehghanpoor R, Stringfellow EJ, Gupta M, Rochelle J, Mason E, Pujol TA, Jalali MS. Identifying Advice-Seekers for Buprenorphine-Naloxone Use on Reddit via Social Network Characteristics and Medication Use Stage. Submitted;Abstract

Online communities can provide social support for those recovering from opioid use disorder. However, advice-seekers on these platforms risk exposure to uncurated medical advice, potentially harming their health or recovery efforts. To identify advice-seekers on an online platform for buprenorphine-naloxone use, we combined text annotation, social network analysis, and statistical modeling. We collected 5,258 posts and their comments from Reddit between 2014 and 2019. Among 202 posts which met our inclusion criteria, we annotated each post to determine which were advice-seeking (n=137) and not advice-seeking (n=65). We also annotated each posting user’s medication use stage and quantified their connectedness using social network analysis. In order to analyze the relationship between advice-seeking with a user’s social connectivity and medication use stage, we constructed four models which varied in explanatory variables. The stepwise model (containing “total degree” (P=0.002), “using: inducting/tapering” (P<0.001), and “using: other” (P=0.01) as significant explanatory variables) outperformed all other models. We found that users with fewer connections and who are currently using buprenorphine-naloxone are more likely to seek advice than users who are well-connected and no longer using the medication, respectively. Hence, clinicians should be especially attentive (e.g., through frequent follow-up) to patients who are inducting or tapering buprenorphine-naloxone or signal limited social support. Moreover, advice-seeking behavior is most accurately predicted using a combination of network characteristics and buprenorphine-naloxone use status, rather than either factor alone. These findings provide insights for the clinical care of people who use buprenorphine-naloxone and the nature of online medical advice-seeking overall.

Leerapan B, Kaewkamjornchai PS, Atun RA, Jalali MS. How Systems Respond to Policies: Unintended Consequences of COVID-19 Lockdown Policies in Thailand. Submitted;
Decouttere C, Banzimana S, Davidsen Pål, Riet CV, Vandermeulen C, Mason E, Jalali MS, Vandaele N. Behavioural Dynamics of Vaccine Hesitancy: The Case of sub-Saharan African Communities. Submitted;
Beaulieu E, Naumann RB, Deveaux G, Wang L, Stringfellow E, Lich KH, Jalali MS. Alcohol and Opioid Use Impacts on Road Safety-related Outcomes: Systematic Review. Submitted;Abstract

Background and Aims: Connections between substance use, impairment, and road safety have been frequently researched. Yet, little is known about how simultaneous use of opioids and alcohol affects road safety outcomes, which is an increasingly critical link to improve health outcomes for people with alcohol, opioid, or heroin use disorders as well as reduce wider driving-related risk for the general public. Given the increasing rate of opioid use and high prevalence of alcohol use, we aim to synthesize literature on the prevalence and impact of this polysubstance combination on road safety-related outcomes.

Methods: We performed a systematic review of studies published between 1974 and 2020 that examined opioid and alcohol use exposures and road safety-related outcomes.

Results: Twenty studies were identified. Studies utilized randomized control trial (n=2), cross-sectional (n=15), and case-control designs (n=3) and were of moderate methodological quality. Outcomes included motor vehicle crash injuries, deaths, or driver culpability; suspected driving under the influence; and simulated driving performance. There was a dearth of studies that isolated findings for simultaneous opioid and alcohol use, making the ability to draw strong conclusions on their relationship challenging, and presenting an opportunity for further research. Per available results, evidence pointed to elevated risks in road safety outcomes for simultaneous use of opioids and alcohol, compared to when neither or only one substance was present.

Conclusions: Research indicates that alcohol and opioid use is common and increasing among people involved in adverse driving events and that simultaneous use may further elevate risk. Future research can improve estimates of associations with road traffic-related outcomes, potentially using linked data sets or other novel data sources.

Cerdá M, Jalali MS, Hamilton A, Digennaro C, Hyder A, Santaella-Tenorio J, Kaur N, Wang C, Keyes KM. The promise of simulation models to track and address the opioid crisis: a systematic review. Submitted;Abstract

The opioid overdose crisis is driven by an intersecting set of social, structural, and economic forces. Simulation models offer a tool to help us understand and address this complex, dynamic, nonlinear, social phenomenon. We conducted a systematic review of the literature on simulation models of opioid use and overdose up to September 2019. We extracted modeling types, target populations, interventions, and findings. Further, we created a database of model parameters used for model calibration, and evaluated study transparency and reproducibility. Of the 1,381 articles screened, we identified 72 eligible articles. The most frequent types of models were Markov (26%), compartmental (25%), system dynamics (19%), and Agent-Based models (18%). Almost half (46%) evaluated intervention cost-effectiveness, while 29% of studies focused on treatment and harm reduction services for people with opioid use disorder (OUD). More than half (57%) calibrated their models to empirical data, and 31% discussed validation approaches used in their modeling process. From the 51 studies that provided data on model parameters, we mapped out the data sources for parameters on opioid use, OUD, OUD treatment, cessation/relapse, emergency medical services, and mortality. This database offers a tool that future modelers can use to identify model inputs, and to evaluate comparability of their models to prior work.  Future applications of simulation models to this field should actively tackle key methodological challenges, including the potential for bias in the choice of parameter inputs, investment in model calibration and validation, and transparency in the assumptions and mechanics of simulation models to facilitate reproducibility.

Li P, Luo Y, Yu X, Zeng Z, Wen J, Mason E, Li W, Jalali MS. Readiness of Healthcare Providers for e-Hospitals: Cross-sectional Analysis in China. Submitted; eHospitals_healthcare providers.pdf
Xu R, Rahmandad H, Gupta M, DiGennaro C, Ghaffarzadegan N, Jalali MS. Weather Conditions and COVID-19 Transmission: Estimates and Projections. Lancet Planetary Health Forthcoming;Abstract

Background: Understanding and projecting the spread of COVID-19 requires reliable estimates of how weather components are associated with the transmission of the virus. Prior research on this topic has been inconclusive. Identifying key challenges to reliable estimation of weather impact on transmission we study this question using one of the largest assembled databases of COVID-19 infections and weather.
Methods: We assemble a dataset that includes virus transmission and weather data across 3,739 locations from December 12, 2019 to April 22, 2020. Using simulation, we identify key challenges to reliable estimation of weather impacts on transmission, design a statistical method to overcome these challenges, and validate it in a blinded simulation study. Using this method and controlling for location-specific response trends we estimate how different weather variables are associated with the reproduction number for COVID-19. We then use the estimates to project the relative weather-related risk of COVID-19 transmission across the world and in large cities.
Results: We show that the delay between exposure and detection of infection complicates the estimation of weather impact on COVID-19 transmission, potentially explaining significant variability in results to-date. Correcting for that distributed delay and offering conservative estimates, we find a negative relationship between temperatures above 25 degrees Celsius and estimated reproduction number (R ̂), with each degree Celsius associated with a 3.1% (95% CI, 1.5% to 4.8%) reduction in R ̂. Higher levels of relative humidity strengthen the negative effect of temperature above 25 degrees. Moreover, one millibar of additional pressure increases R ̂ by approximately 0.8 percent (95% CI, 0.6% to 1%) at the median pressure (1016 millibars) in our sample. We also find significant positive effects for wind speed, precipitation, and diurnal temperature on R ̂. Sensitivity analysis and simulations show that results are robust to multiple assumptions. Despite conservative estimates, weather effects are associated with a 43% change in R ̂ between the 5th and 95th percentile of weather conditions in our sample.
Conclusions: These results provide evidence for the relationship between several weather variables and the spread of COVID-19. However, the (conservatively) estimated relationships are not strong enough to seasonally control the epidemic in most locations.


Online simulator:

Evolution and Reproducibility of Simulation Modeling in Epidemiology and Health Policy over Half a Century
Jalali MS, DiGennaro C, Guitar A, Lew K, Rahmandad H. Evolution and Reproducibility of Simulation Modeling in Epidemiology and Health Policy over Half a Century. Epidemiologic Reviews Forthcoming;Abstract

Background: Simulation models are increasingly used to inform health policy. We provide an overview of applications of simulation models in health policy, analyze the use of best reporting practices, and assess the reproducibility of existing studies.

Method: Studies that used simulation modeling as the core method to address any health policy questions were included. Health policy domain distribution and changes in quality over time were well-characterized using MeSH terms and model characteristics, respectively. Reproducibility was assessed using predefined, categorical criteria.

Findings: 1,613 studies were analyzed. We found an exponential growth in the number of studies over the past half century, with the highest growth in dynamic modeling approaches. The largest subset of studies is focused on disease policy models (70%), within which pathological conditions, viral diseases, neoplasms, and cardiovascular diseases account for one-third of the articles. Nearly half of the studies do not report the details of their models. A subset of 100 articles (50 highly cited and 50 random) were selected to analyze in-depth criteria for reporting quality and reproducibility. Significant gaps between best modeling practices could be found in both the random and highly cited samples; only seven of 26 in-depth evaluation criteria were satisfied by more than 80% of samples. We found no evidence that the highly cited samples adhered better to the modeling best practices.

Interpretation: Our results suggest crucial areas for increased applications of simulation modeling, and opportunities to enhance the rigor and documentation in the conduct and reporting of simulation modeling in health policy.

Evaluation criteria

Removing The X-Waiver Is One Small Step Toward Increasing Treatment Of Opioid Use Disorder, But Great Leaps Are Needed
Stringfellow EJ, Humphreys K, Jalali MS. Removing The X-Waiver Is One Small Step Toward Increasing Treatment Of Opioid Use Disorder, But Great Leaps Are Needed [Internet]. Health Affairs Blog 2021; Publisher's Version
Impact of Florida’s Prescription Drug Monitoring Program on Fatal Vehicle Crashes: A Difference-in-Differences Approach
Tatar M, Jalali MS, Tak HJ, Chen L-W, Araz OM, Wilson FA. Impact of Florida’s Prescription Drug Monitoring Program on Fatal Vehicle Crashes: A Difference-in-Differences Approach. Injury Prevention 2021;Abstract

Objective: To examine the impact of Florida's implementation of a mandatory PDMP on drug-related motor vehicle crashes occurring on public roads.

Methods: We employed a difference-in-differences approach to estimate the difference in prescription drug-related fatal crashes in Florida associated with its 2011 PDMP implementation relative to those in Georgia, which did not utilize PDMPs during the same period (2009-2013).

Results: In Florida, there was a significant decline in drug-related vehicle crashes during the 22 months post-PDMP. PDMP implementation was associated with approximately two (-2.21; 95%-CI: [-4.04,-0.37]; P<0.05) fewer prescribed opioid-related fatal crashes every month, indicating 25% reduction in the number of monthly crashes. We also conducted sensitivity analyses to investigate the impact of PDMP implementation on CNS depressants and stimulants as well as cocaine and marijuana-related fatal crashes but found no robust significant reductions.

Conclusions: The implementation of PDMPs in Florida provided important benefits for traffic safety, reducing the rates of prescription opioid-related vehicle crashes.

Policy implications: Mandatory PDMP implementation policies could decrease drug-related fatal vehicle crashes.

Changes in Characteristics of Drug Overdose Death Trends during the COVID-19 Pandemic
DiGennaro C, Garcia G-GP, Stringfellow EJ, Wakeman S, Jalali MS. Changes in Characteristics of Drug Overdose Death Trends during the COVID-19 Pandemic. International Journal of Drug Policy 2021;Abstract


Background: Reports analyzing drug overdose (OD) mortality data during the COVID-19 pandemic are limited. Outcomes across states are heterogenous, necessitating assessments of associations between COVID-19 and OD deaths on a state-by-state level. This report aims to analyze trends in OD deaths in Massachusetts during COVID-19.

Methods: Analyzing 3,924 death records, we characterize opioid-, cocaine-, and amphetamine-involved OD mortality and substance co-presence trends from March 24-November 8 in 2020 as compared to 2018 and 2019.

Results: OD deaths involving amphetamines increased by 85% from 2019 to 2020 (61 vs. 113; P<0.001) but were steady from 2018 to 2019. Heroin’s presence continued to decrease (341 in 2018, 247 in 2019, 157 in 2020; P<0.001); however, fentanyl was present in more than 85% of all OD deaths across all periods. Among OD deaths, alcohol involvement consistently increased, present in 250 deaths in 2018, 299 in 2019 (P=0.02), and 350 in 2020 (P=0.04). In 2019, 78% of OD decedents were White and 7% were Black, versus 73% and 10% in 2020 (P=0.02).

Conclusion: Increased deaths involving stimulants, alcohol, and fentanyl reflect concerning trends in the era of COVID-19. Rising OD death rates among Black residents underscore that interventions focused on racial equity are necessary.


Data Needs in Opioid Systems Modeling: Challenges and Future Directions
Jalali MS, Ewing E, Bannister CB, Glos L, Eggers S, Lim TY, Stringfellow E, Stafford C, Pacula RL, Jalal H, Kazemi-Tabriz R. Data Needs in Opioid Systems Modeling: Challenges and Future Directions. American Journal of Preventive Medicine 2021;60(2):e95-e105.Abstract

Background: The opioid crisis is a pervasive public health threat in the U.S. Simulation modeling approaches that integrate a systems perspective are used to understand the complexity of this crisis and to analyze what policy interventions can best address it. However, limitations in currently available data sources can hamper the quantification of these models.

Methods: To understand and discuss data needs and challenges for opioid systems modeling, a meeting of federal partners, modeling teams, and data experts was held at the U.S. Food and Drug Administration in April 2019. This paper synthesizes the meeting discussions and interprets them in the context of ongoing simulation modeling work.

Results: The current landscape of national-level quantitative data sources of potential use in opioid systems modeling is identified, and significant issues within data sources are discussed. Major recommendations on how to improve data sources are to: maintain close collaboration among modeling teams; enhance data collection to better fit modeling needs; focus on bridging the most crucial information gaps; engage in direct and regular interaction between modelers and data experts; and gain a clearer definition of policymakers’ research questions and policy goals.

Conclusions: This article provides an important step in identifying and discussing data challenges in opioid research in general and opioid systems modeling in particular. It also identifies opportunities for systems modelers and government agencies to improve opioid systems models.

slides_data_challenges_in_opioid_systems_modeling.pdf data_needs_opioids_modeling.pdf
Whether the Weather Will Help Us Weather the COVID-19 Pandemic: Using Machine Learning to Measure Twitter Users’ Perceptions
Gupta M, Bansal A, Jain B, Rochelle J, Oak A, Jalali MS. Whether the Weather Will Help Us Weather the COVID-19 Pandemic: Using Machine Learning to Measure Twitter Users’ Perceptions. International Journal of Medical Informatics 2021;145:104340.Abstract

Objective: The potential ability for weather to affect SARS-CoV-2 transmission has been an area of controversial discussion during the COVID-19 pandemic. Individuals’ perceptions of the impact of weather can inform their adherence to public health guidelines; however, there is no measure of their perceptions. We quantified Twitter users’ perceptions of the effect of weather and analyzed how they evolved with respect to real-world events and time.

Materials and Methods: We collected 166,005 tweets posted between January 23 and June 22, 2020 and employed machine learning/natural language processing techniques to filter for relevant tweets, classify them by the type of effect they claimed, and identify topics of discussion.

Results: We identified 28,555 relevant tweets and estimate that 40.4% indicate uncertainty about weather’s impact, 33.5% indicate no effect, and 26.1% indicate some effect. We tracked changes in these proportions over time. Topic modeling revealed major latent areas of discussion.

Discussion: There is no consensus among the public for weather’s potential impact. Earlier months were characterized by tweets that were uncertain of weather’s effect or claimed no effect; later, the portion of tweets claiming some effect of weather increased. Tweets claiming no effect of weather comprised the largest class by June. Major topics of discussion included comparisons to influenza’s seasonality, President Trump’s comments on weather’s effect, and social distancing.

Conclusion: There is a major gap between scientific evidence and public opinion of weather’s impacts on COVID-19. We provide evidence of public’s misconceptions and topics of discussion, which can inform public health communications.

Weather tweets

Deutsch AR, Lustfield R, Jalali MS. Community-based System Dynamics Modeling of Sensitive Public Health Issues: Maximizing Diverse Representation of Individuals with Personal Experiences. 2021;Abstract

Community-based system dynamics (CBSD) models enhance our understanding of stigmatized public health issues and related health disparities. The accuracy and usefulness of these models depend upon the individuals who take part in group modeling sessions. Marginalized individuals that are personally impacted by these health issues are critical in the function and development of the models. However, the extent of inclusion varies between studies since such individuals are often hard to recruit. There is substantial diversity in how individuals experience a stigmatized public health issue and with the underrepresentation of individuals with personal experience, research may conclude in biased model development. The purpose of this study was to explore a method that would increase representation for individuals with personal experience of stigmatized issues in model development. We used a case study from a CBSD project on the association between alcohol misuse (AM) and intimate partner violence (IPV) within a Northern Plains American Indian community. Group model building sessions were held at three community organizations: a faith-based re-entry program, a substance use rehabilitation program for pregnant women and mothers, and a domestic violence shelter. Session participants (clients of these organizations) were quick to understand the systems method and were engaged in the modeling process. There were few similarities between the three CBSD models. Each model contributed unique system components, and a consolidated model provided a rich picture of the complex AM-IPV system, as well as the ways in which health disparities are maintained. Coupled with an emphasis on transparency and trust building between researchers and modelers, our approach illuminated the diversity of ways in which individuals with personal experience can perceive AM-IPV systems. Using similar strategies for model building can complement existing efforts to build representative models for stigmatized public health issues within communities.

Economic Evaluation in Opioids Modeling: Systematic Review
Beaulieu E, DiGennaro C, Stringfellow E, Connolly A, Hamilton A, Hyder A, Cerdá M, Keyes KM, Jalali MS. Economic Evaluation in Opioids Modeling: Systematic Review. Value in Health 2021;24(2):158-173.Abstract

The rapid increase in opioid overdose and opioid use disorder (OUD) over the past 20 years is a complex problem associated with significant economic costs for healthcare systems and society. Simulation models have been developed to capture and identify ways to manage this complexity and to evaluate the potential costs of different strategies to reduce overdoses and OUD. A review of simulation-based economic evaluations is warranted to fully characterize this set of literature.

A systematic review of simulation-based economic evaluation (SBEE) studies in opioid research was initiated by searches in PubMed, EMBASE, and EbscoHOST. Extraction of a predefined set of items and a quality assessment were performed for each study.

The screening process resulted in 23 SBEE studies ranging by year of publication from 1999 to 2019. Methodological quality of the cost analyses was moderately high. The most frequently evaluated strategies were methadone and buprenorphine maintenance treatments; the only harm reduction strategy explored was naloxone distribution. These strategies were consistently found to be cost-effective, especially naloxone distribution and methadone maintenance. Prevention strategies were limited to abuse-deterrent opioid formulations. Less than half (39%) of analyses adopted a societal perspective in their estimation of costs and effects from an opioid-related intervention. Prevention strategies and studies’ accounting for patient and physician preference, changing costs, or result stratification were largely ignored in these SBEEs.

The review shows consistently favorable cost analysis findings for naloxone distribution strategies and opioid agonist treatments and identifies major gaps for future research.

economic_evaluation_in_opioid_modeling.pdf supplement_economic_evaluation_in_opioid_modeling.pdf
Trends in Thyroid Surgery and Guideline-Concordant Care in the United States, 2007-2018
Toumi A, DiGennaro C, Vahdat V, Jalali MS, Gazelle SG, Chhatwal J, Kelz RR, Lubitz CC. Trends in Thyroid Surgery and Guideline-Concordant Care in the United States, 2007-2018. Thyroid 2020;Abstract
Background: The American Thyroid Association (ATA) published the 2015 Management Guidelines for Patients with Thyroid Nodules and Differentiated Thyroid Cancer recommending a shift to less aggressive diagnostic, surgical, and post-operative treatment strategies. At the same time and perhaps related to the new guidelines, there has been a shift to outpatient thyroid surgery. The aim of the current study was to assess physician adherence to these recommendations by identifying and quantifying temporal trends in the rates and indications for thyroid procedures in the inpatient and outpatient settings. Methods: Using the IBM® MarketScan® Commercial database, we identified employer-insured patients in the United States who underwent outpatient and inpatient thyroid surgery from 2007 to 2018. Thyroid surgery was classified as total thyroidectomy (TT), thyroid lobectomy (TL) or a completion thyroidectomy. The surgical indication diagnosis was also determined and classified as either benign or malignant thyroid disease. We compared outpatient and inpatient trends in surgery between benign and malignant thyroid disease before and after the release of the 2015 ATA guidelines. Results: A total of 220,088 patients who underwent thyroid surgery were included in the analysis. Approximately 80% of thyroid lobectomies (TL) were performed in the outpatient setting vs. 70% of total thyroidectomies (TT). Longitudinal analysis showed a statistically significant changepoint for TT proportion occurring in November 2015. The proportion of TT as compared to TL decreased from 80% in September 2015 to 39% by December 2018. For thyroid cancer, there is an increasing trend in performing TL over TT, increasing from 17% in 2015 to 28% by the end of 2018. Conclusions: There was a significant changepoint occurring in November 2015 in the operative and management trends for benign and malignant thyroid disease.
The opioid crisis: need for systems science research
Jalali MS, Botticelli M, Hwang R, Koh HK, McHugh RK. The opioid crisis: need for systems science research. Health Research Policy and Systems 2020;18(88)Abstract
The opioid epidemic in the United States has had a devastating impact on millions of people as well as on their families and communities. The increased prevalence of opioid misuse, use disorder and overdose in recent years has highlighted the need for improved public health approaches for reducing the tremendous harms of this illness. In this paper, we explain and call for the need for more systems science approaches, which can uncover the complexities of the opioid crisis, and help evaluate, analyse and forecast the effectiveness of ongoing and new policy interventions. Similar to how a stream of systems science research helped policy development in infectious diseases and obesity, more systems science research is needed in opioids.
Telemedicine, Privacy, and Information Security in the Age of COVID-19
Jalali MS, Landman A, Gordon WJ. Telemedicine, Privacy, and Information Security in the Age of COVID-19. Journal of the American Medical Informatics Association 2020; telemedicine_privacy_and_information_security_preprint.pdf