Decouttere C, Banzimana S, Davidsen Pål, Riet CV, Vandermeulen C, Mason E, Jalali MS, Vandaele N. Behavioural Dynamics of Vaccine Hesitancy: The Case of sub-Saharan African Communities. Submitted;
Tatar M, Jalali MS, Tak HJ, Chen L-W, Araz OM, Wilson FA. Impact of Florida’s Prescription Drug Monitoring Program on Fatal Vehicle Crashes: A Difference-in-Differences Approach. Submitted;Abstract

Objective: To examine the impact of Florida's implementation of a mandatory PDMP on drug-related motor vehicle crashes occurring on public roads.

Methods: We employed a difference-in-differences approach to estimate the difference in prescription drug-related fatal crashes in Florida associated with its 2011 PDMP implementation relative to those in Georgia, which did not utilize PDMPs during the same period (2009-2013).

Results: In Florida, there was a significant decline in drug-related vehicle crashes during the 22 months post-PDMP. PDMP implementation was associated with approximately two (-2.21; 95%-CI: [-4.04,-0.37]; P<0.05) fewer prescribed opioid-related fatal crashes every month, indicating 25% reduction in the number of monthly crashes. We also conducted sensitivity analyses to investigate the impact of PDMP implementation on CNS depressants and stimulants as well as cocaine and marijuana-related fatal crashes but found no robust significant reductions.

Conclusions: The implementation of PDMPs in Florida provided important benefits for traffic safety, reducing the rates of prescription opioid-related vehicle crashes.

Policy implications: Mandatory PDMP implementation policies could decrease drug-related fatal vehicle crashes.

Opioid Overdose Admissions within a Large Healthcare System in Massachusetts during COVID-19. Submitted;
Cerdá M, Jalali MS, Hamilton A, Digennaro C, Hyder A, Santaella-Tenorio J, Kaur N, Wang C, Keyes KM. The promise of simulation models to track and address the opioid crisis: a systematic review. Submitted;Abstract

The opioid overdose crisis is driven by an intersecting set of social, structural, and economic forces. Simulation models offer a tool to help us understand and address this complex, dynamic, nonlinear, social phenomenon. We conducted a systematic review of the literature on simulation models of opioid use and overdose up to September 2019. We extracted modeling types, target populations, interventions, and findings. Further, we created a database of model parameters used for model calibration, and evaluated study transparency and reproducibility. Of the 1,381 articles screened, we identified 72 eligible articles. The most frequent types of models were Markov (26%), compartmental (25%), system dynamics (19%), and Agent-Based models (18%). Almost half (46%) evaluated intervention cost-effectiveness, while 29% of studies focused on treatment and harm reduction services for people with opioid use disorder (OUD). More than half (57%) calibrated their models to empirical data, and 31% discussed validation approaches used in their modeling process. From the 51 studies that provided data on model parameters, we mapped out the data sources for parameters on opioid use, OUD, OUD treatment, cessation/relapse, emergency medical services, and mortality. This database offers a tool that future modelers can use to identify model inputs, and to evaluate comparability of their models to prior work.  Future applications of simulation models to this field should actively tackle key methodological challenges, including the potential for bias in the choice of parameter inputs, investment in model calibration and validation, and transparency in the assumptions and mechanics of simulation models to facilitate reproducibility.

DiGennaro C, Garcia G-GP, Stringfellow EJ, Wakeman S, Jalali MS. Opioid Overdose Deaths during COVID-19: A Year-Over-Year Comparison in Massachusetts. Submitted;Abstract

Objectives: To assess the relationship between the COVID-19 pandemic and the ongoing opioid overdose crisis in Massachusetts.

Methods: We conducted year-over-year comparisons using Massachusetts vital statistics data to analyze overall trends in opioid overdose deaths and investigated the relationship between county-level COVID-19 case fatality rates and changes in opioid overdose deaths.

Results: During the period between March 24 (stay-at-home order) and June 11, 2020 (last reliable data), the proportion of overdoses involving heroin decreased by 49% from 2019. Overdoses involving cocaine, alcohol, and amphetamines have increased by 33%, 51%, and 111%, respectively. Fentanyl (92%) and prescription opioid (21%) presence remained steady. We find no significant increase in overall overdose deaths from 2019 and no significant correlation between COVID-19 case fatality rate and changes in overdose death.

Conclusions: Increased deaths involving stimulants reflect concerning national trends that predate COVID-19. Reported rises in opioid overdose deaths during COVID-19 are not reflected in Massachusetts. COVID-19 serves as a model for overcoming difficulties in data availability, but opioid overdose surveillance data are months-to-years behind. Given lags in mortality data, surveillance efforts must incorporate up-to-date data from first responders.

Xu R, Rahmandad H, Gupta M, DiGennaro C, Ghaffarzadegan N, Jalali MS. Weather Conditions and COVID-19 Transmission: Estimates and Projections. Submitted;Abstract

Background: Understanding and projecting the spread of COVID-19 requires reliable estimates of how weather components are associated with the transmission of the virus. Prior research on this topic has been inconclusive. Identifying key challenges to reliable estimation of weather impact on transmission we study this question using one of the largest assembled databases of COVID-19 infections and weather.
Methods: We assemble a dataset that includes virus transmission and weather data across 3,739 locations from December 12, 2019 to April 22, 2020. Using simulation, we identify key challenges to reliable estimation of weather impacts on transmission, design a statistical method to overcome these challenges, and validate it in a blinded simulation study. Using this method and controlling for location-specific response trends we estimate how different weather variables are associated with the reproduction number for COVID-19. We then use the estimates to project the relative weather-related risk of COVID-19 transmission across the world and in large cities.
Results: We show that the delay between exposure and detection of infection complicates the estimation of weather impact on COVID-19 transmission, potentially explaining significant variability in results to-date. Correcting for that distributed delay and offering conservative estimates, we find a negative relationship between temperatures above 25 degrees Celsius and estimated reproduction number (R ̂), with each degree Celsius associated with a 3.1% (95% CI, 1.5% to 4.8%) reduction in R ̂. Higher levels of relative humidity strengthen the negative effect of temperature above 25 degrees. Moreover, one millibar of additional pressure increases R ̂ by approximately 0.8 percent (95% CI, 0.6% to 1%) at the median pressure (1016 millibars) in our sample. We also find significant positive effects for wind speed, precipitation, and diurnal temperature on R ̂. Sensitivity analysis and simulations show that results are robust to multiple assumptions. Despite conservative estimates, weather effects are associated with a 43% change in R ̂ between the 5th and 95th percentile of weather conditions in our sample.
Conclusions: These results provide evidence for the relationship between several weather variables and the spread of COVID-19. However, the (conservatively) estimated relationships are not strong enough to seasonally control the epidemic in most locations.


Online simulator:

Deutsch AR, Lustfield R, Jalali MS. Community-based System Dynamics Modeling of Sensitive Public Health Issues: Maximizing Diverse Representation of Individuals with Personal Experiences. Submitted;Abstract

Community-based system dynamics (CBSD) models enhance our understanding of stigmatized public health issues and related health disparities. The accuracy and usefulness of these models depend upon the individuals who take part in group modeling sessions. Marginalized individuals that are personally impacted by these health issues are critical in the function and development of the models. However, the extent of inclusion varies between studies since such individuals are often hard to recruit. There is substantial diversity in how individuals experience a stigmatized public health issue and with the underrepresentation of individuals with personal experience, research may conclude in biased model development. The purpose of this study was to explore a method that would increase representation for individuals with personal experience of stigmatized issues in model development. We used a case study from a CBSD project on the association between alcohol misuse (AM) and intimate partner violence (IPV) within a Northern Plains American Indian community. Group model building sessions were held at three community organizations: a faith-based re-entry program, a substance use rehabilitation program for pregnant women and mothers, and a domestic violence shelter. Session participants (clients of these organizations) were quick to understand the systems method and were engaged in the modeling process. There were few similarities between the three CBSD models. Each model contributed unique system components, and a consolidated model provided a rich picture of the complex AM-IPV system, as well as the ways in which health disparities are maintained. Coupled with an emphasis on transparency and trust building between researchers and modelers, our approach illuminated the diversity of ways in which individuals with personal experience can perceive AM-IPV systems. Using similar strategies for model building can complement existing efforts to build representative models for stigmatized public health issues within communities.

CBSD for sensitive issues.pdf
Readiness of Healthcare Providers for e-Hospitals: Cross-sectional Analysis in China. Submitted; eHospitals_healthcare providers.pdf
Jalali MS, DiGennaro C, Guitar A, Lew K, Rahmandad H. Evolution of Simulation Modeling in Health Policy over Half a Century. Submitted;Abstract

Background: Simulation models are increasingly used to inform health policy. We provide an overview of applications of simulation models in health policy, analyze the use of best reporting practices, and assess the reproducibility of existing studies.

Method: Studies that used simulation modeling as the core method to address any health policy questions were included. Health policy domain distribution and changes in quality over time were well-characterized using MeSH terms and model characteristics, respectively. Reproducibility was assessed using predefined, categorical criteria.

Findings: 1,613 studies were analyzed. We found an exponential growth in the number of studies over the past half century, with the highest growth in dynamic modeling approaches. The largest subset of studies is focused on disease policy models (70%), within which pathological conditions, viral diseases, neoplasms, and cardiovascular diseases account for one-third of the articles. Nearly half of the studies do not report the details of their models. A subset of 100 articles (50 highly cited and 50 random) were selected to analyze in-depth criteria for reporting quality and reproducibility. Significant gaps between best modeling practices could be found in both the random and highly cited samples; only seven of 26 in-depth evaluation criteria were satisfied by more than 80% of samples. We found no evidence that the highly cited samples adhered better to the modeling best practices.

Interpretation: Our results suggest crucial areas for increased applications of simulation modeling, and opportunities to enhance the rigor and documentation in the conduct and reporting of simulation modeling in health policy.

Evaluation criteria

Trends in Thyroid Surgery and Guideline-Concordant Care in the United States, 2007-2018
Toumi A, DiGennaro C, Vahdat V, Jalali MS, Gazelle SG, Chhatwal J, Kelz RR, Lubitz CC. Trends in Thyroid Surgery and Guideline-Concordant Care in the United States, 2007-2018. Thyroid 2020;Abstract
Background: The American Thyroid Association (ATA) published the 2015 Management Guidelines for Patients with Thyroid Nodules and Differentiated Thyroid Cancer recommending a shift to less aggressive diagnostic, surgical, and post-operative treatment strategies. At the same time and perhaps related to the new guidelines, there has been a shift to outpatient thyroid surgery. The aim of the current study was to assess physician adherence to these recommendations by identifying and quantifying temporal trends in the rates and indications for thyroid procedures in the inpatient and outpatient settings. Methods: Using the IBM® MarketScan® Commercial database, we identified employer-insured patients in the United States who underwent outpatient and inpatient thyroid surgery from 2007 to 2018. Thyroid surgery was classified as total thyroidectomy (TT), thyroid lobectomy (TL) or a completion thyroidectomy. The surgical indication diagnosis was also determined and classified as either benign or malignant thyroid disease. We compared outpatient and inpatient trends in surgery between benign and malignant thyroid disease before and after the release of the 2015 ATA guidelines. Results: A total of 220,088 patients who underwent thyroid surgery were included in the analysis. Approximately 80% of thyroid lobectomies (TL) were performed in the outpatient setting vs. 70% of total thyroidectomies (TT). Longitudinal analysis showed a statistically significant changepoint for TT proportion occurring in November 2015. The proportion of TT as compared to TL decreased from 80% in September 2015 to 39% by December 2018. For thyroid cancer, there is an increasing trend in performing TL over TT, increasing from 17% in 2015 to 28% by the end of 2018. Conclusions: There was a significant changepoint occurring in November 2015 in the operative and management trends for benign and malignant thyroid disease.
The opioid crisis: need for systems science research
Jalali MS, Botticelli M, Hwang R, Koh HK, McHugh RK. The opioid crisis: need for systems science research. Health Research Policy and Systems 2020;18(88)Abstract
The opioid epidemic in the United States has had a devastating impact on millions of people as well as on their families and communities. The increased prevalence of opioid misuse, use disorder and overdose in recent years has highlighted the need for improved public health approaches for reducing the tremendous harms of this illness. In this paper, we explain and call for the need for more systems science approaches, which can uncover the complexities of the opioid crisis, and help evaluate, analyse and forecast the effectiveness of ongoing and new policy interventions. Similar to how a stream of systems science research helped policy development in infectious diseases and obesity, more systems science research is needed in opioids.
Data Needs in Opioid Systems Modeling: Challenges and Future Directions
Jalali MS, Ewing E, Bannister CB, Glos L, Eggers S, Lim TY, Stringfellow E, Stafford C, Pacula RL, Jalal H, Kazemi-Tabriz R. Data Needs in Opioid Systems Modeling: Challenges and Future Directions. American Journal of Preventive Medicine 2020;Abstract

Background: The opioid crisis is a pervasive public health threat in the U.S. Simulation modeling approaches that integrate a systems perspective are used to understand the complexity of this crisis and to analyze what policy interventions can best address it. However, limitations in currently available data sources can hamper the quantification of these models.

Methods: To understand and discuss data needs and challenges for opioid systems modeling, a meeting of federal partners, modeling teams, and data experts was held at the U.S. Food and Drug Administration in April 2019. This paper synthesizes the meeting discussions and interprets them in the context of ongoing simulation modeling work.

Results: The current landscape of national-level quantitative data sources of potential use in opioid systems modeling is identified, and significant issues within data sources are discussed. Major recommendations on how to improve data sources are to: maintain close collaboration among modeling teams; enhance data collection to better fit modeling needs; focus on bridging the most crucial information gaps; engage in direct and regular interaction between modelers and data experts; and gain a clearer definition of policymakers’ research questions and policy goals.

Conclusions: This article provides an important step in identifying and discussing data challenges in opioid research in general and opioid systems modeling in particular. It also identifies opportunities for systems modelers and government agencies to improve opioid systems models.

slides_data_challenges_in_opioid_systems_modeling.pdf data_needs_opioids_modeling.pdf
Whether the Weather Will Help Us Weather the COVID-19 Pandemic: Using Machine Learning to Measure Twitter Users’ Perceptions
Gupta M, Bansal A, Jain B, Rochelle J, Oak A, Jalali MS. Whether the Weather Will Help Us Weather the COVID-19 Pandemic: Using Machine Learning to Measure Twitter Users’ Perceptions. International Journal of Medical Informatics 2020;Abstract

Objective: The potential ability for weather to affect SARS-CoV-2 transmission has been an area of controversial discussion during the COVID-19 pandemic. Individuals’ perceptions of the impact of weather can inform their adherence to public health guidelines; however, there is no measure of their perceptions. We quantified Twitter users’ perceptions of the effect of weather and analyzed how they evolved with respect to real-world events and time.

Materials and Methods: We collected 166,005 tweets posted between January 23 and June 22, 2020 and employed machine learning/natural language processing techniques to filter for relevant tweets, classify them by the type of effect they claimed, and identify topics of discussion.

Results: We identified 28,555 relevant tweets and estimate that 40.4% indicate uncertainty about weather’s impact, 33.5% indicate no effect, and 26.1% indicate some effect. We tracked changes in these proportions over time. Topic modeling revealed major latent areas of discussion.

Discussion: There is no consensus among the public for weather’s potential impact. Earlier months were characterized by tweets that were uncertain of weather’s effect or claimed no effect; later, the portion of tweets claiming some effect of weather increased. Tweets claiming no effect of weather comprised the largest class by June. Major topics of discussion included comparisons to influenza’s seasonality, President Trump’s comments on weather’s effect, and social distancing.

Conclusion: There is a major gap between scientific evidence and public opinion of weather’s impacts on COVID-19. We provide evidence of public’s misconceptions and topics of discussion, which can inform public health communications.

Weather tweets

Telemedicine, Privacy, and Information Security in the Age of COVID-19
Jalali MS, Landman A, Gordon WJ. Telemedicine, Privacy, and Information Security in the Age of COVID-19. Journal of the American Medical Informatics Association 2020; telemedicine_privacy_and_information_security_preprint.pdf
Transparency Assessment of COVID-19 Models
Jalali MS, DiGennaro C, Sridhar D. Transparency Assessment of COVID-19 Models. The Lancet Global Health 2020;Abstract
As the COVID-19 pandemic has caused major societal unrest, modelers have worked to project future trends of COVID-19 and predict upcoming challenges and impacts of policy action. These models, alone or in aggregate, are influential for decision-makers at every level. Therefore, the method and documentation of COVID-19 models must be highly transparent to ensure that projections and consequential policies put forth have sound epistemological grounds. We evaluated 29 COVID-19 models receiving high attention levels within the scientific community and/or informing government responses. We evaluated these models against 27 transparency criteria. We found high levels of transparency in model documentation aspects such as reporting uncertainty analysis; however, about half of the models do not share code and a quarter do not report equations. These discrepancies underscore the need for transparency and reproducibility to be at the forefront of researchers’ priorities, especially during a global health crisis when stakes are critically high.
transparency_assessment_covid19.pdf transparency_assessment_covid19_appendix.pdf
How Telemedicine Integrated into China’s Anti-COVID-19 Strategies: Case from a National Referral Center
Li P, Liu X, Mason E, Hu G, Zhou Y, Li W, Jalali MS. How Telemedicine Integrated into China’s Anti-COVID-19 Strategies: Case from a National Referral Center. BMJ Health & Care Informatics 2020;(27):e100164.Abstract

Introduction We present the integration of telemedicine into the healthcare system of West China Hospital of Sichuan University (WCH), one of the largest hospitals in the world with 4300 inpatient beds, as a means for maximising the efficiency of healthcare delivery during the COVID-19 pandemic.

Methods Implemented on 22 January 2020, the telemedicine technology allowed WCH providers to conduct teleconsultations, telerounds, teleradiology and tele-intensive care unit, which in culmination provided screening, triage and treatment for COVID-19 and other illnesses. To encourage its adoption, the government and the hospital publicised the platform on social media and waived fees.

Discussion From 1 February to 1 April 2020, 10557 online COVID-19 consultations were conducted for 6662 individuals; meanwhile, 32676 patients without COVID completed virtual follow-ups. We discuss that high-quality, secure, affordable and user-friendly telemedical platforms should be integrated into global healthcare systems to help decrease the transmission of the virus and protect healthcare providers from infection.


Trends in Hospitalization Expenditures for Acute Exacerbations of COPD in Beijing from 2009 to 2017
Liang L, Shang Y, Xie W, Shi J, Tong Z, Jalali MS. Trends in Hospitalization Expenditures for Acute Exacerbations of COPD in Beijing from 2009 to 2017. International Journal of Chronic Obstructive Pulmonary Disease 2020;(15):1165-1175.Abstract
Background: Chronic obstructive pulmonary disease (COPD) is the cause of substantial economic and social burden. We investigated trends in hospitalizations for acute exacerbation of COPD in Beijing, China, from 2009 to 2017.
Patients and Methods: Investigations were conducted using data from the discharge records of inpatients that were given a primary diagnosis of acute exacerbation of COPD. The dataset was a retrospective review of information collected from electronic medical records and included 315,116 admissions (159,368 patients). Descriptive analyses and multivariate regressions were used to investigate trends in per admission and per capita expenditures, as well as other potential contributing factors.
Results: The mean per admission expenditures increased from 19,760 CNY ($2893, based on USD/CNY=6.8310) in 2009 to 20,118 CNY ($2980) in 2017 (a growth rate of 0.11%). However, the per capita expenditures increased from 23,716 CNY ($3472) in 2009 to 31,000 CNY ($4538) in 2017 (a growth rate of 1.7%). In terms of per admission expenditures, drug costs accounted for 52.9% of the total expenditures in 2009 and dropped to 39.4% in 2017 (P trend < 0.001). The mean length of stay (LOS) decreased from 16.0 days to 13.5 days (P trend < 0· 001). Age, gender, COPD type, LOS, and hospital level were all associated with per admission and per capita expenditures.
Interpretation: Relatively stable per admission expenditures along with the decline in drug costs and LOS reflect the effectiveness of cost containment on some indicators in China’s health care reform. However, the increase in hospitalization expenditures per capita calls for better policies for controlling hospitalizations, especially multiple admissions.
Patients’ Perceptions of Barriers and Facilitators to the Adoption of e-Hospitals: Cross-sectional Study in Western China
Li P, Luo Y, Yu X, Wen J, Mason E, Li W, Jalali MS. Patients’ Perceptions of Barriers and Facilitators to the Adoption of e-Hospitals: Cross-sectional Study in Western China. Journal of Medical Internet Research 2020;22(6):e17221.Abstract

Background: As an innovative approach to providing web-based health care services from physical hospitals to patients at a distance, e-hospitals (ie, extended care hospitals through the internet) have been extensively developed in China. This closed health care delivery chain was developed by combining e-hospitals with physical hospitals; treatment begins with web-based consultation and registration, and then, patients are diagnosed and treated in a physical hospital. This approach is promising in its ability to improve accessibility, efficiency, and quality of health care. However, there is limited research on end users’ acceptance of e-hospitals and the effectiveness of strategies aimed to prompt the adoption of e-hospitals in China.

Objective: This study aimed to provide insights regarding the adoption of e-hospitals by investigating patients’ willingness to use e-hospitals and analyzing the barriers and facilitators to the adoption of this technology.

Methods: We used a pretested self-administered questionnaire and performed a cross-sectional analysis in 1032 patients across three hierarchical hospitals in West China from June to August 2019. Patients’ sociodemographic characteristics, medical history, current disease status, proficiency with electronic devices, previous experience with web-based health services, willingness to use e-hospitals, and perceived facilitators and barriers were surveyed. Multiple significance tests were employed to examine disparities across four age groups, as well as those between patients who were willing to use e-hospitals and those who were not. Multivariate logistic regression was also performed to identify the potential predictors of willingness to use e-hospitals.

Results: Overall, it was found that 65.6% (677/1032) of participants were willing to use e-hospitals. The significant predictors of willingness to use e-hospitals were employment status (P=.02), living with children (P<.001), education level (P=.046), information technology skills (P<.001), and prior experience with web-based health care services (P<.001), whereas age, income, medical insurance, and familiarity with e-hospitals were not predictors. Additionally, the prominent facilitators of e-hospitals were convenience (641/677, 94.7%) and accessibility to skilled medical experts (489/677, 72.2%). The most frequently perceived barrier varied among age groups; seniors most often reported their inability to operate technological devices as a barrier (144/166, 86.7%), whereas young participants most often reported that they avoided e-hospital services because they were accustomed to face-to-face consultation (39/52, 75%).

Conclusions: We identified the variables, facilitators, and barriers that play essential roles in the adoption of e-hospitals. Based on our findings, we suggest that efforts to increase the adoption of e-hospitals should focus on making target populations accustomed to web-based health care services while maximizing ease of use and providing assistance for technological inquiries.

Economic Evaluation in Opioids Modeling: Systematic Review
Beaulieu E, DiGennaro C, Stringfellow E, Connolly A, Hamilton A, Hyder A, Cerdá M, Keyes KM, Jalali MS. Economic Evaluation in Opioids Modeling: Systematic Review. Value in Health 2020;Abstract

The rapid increase in opioid overdose and opioid use disorder (OUD) over the past 20 years is a complex problem associated with significant economic costs for healthcare systems and society. Simulation models have been developed to capture and identify ways to manage this complexity and to evaluate the potential costs of different strategies to reduce overdoses and OUD. A review of simulation-based economic evaluations is warranted to fully characterize this set of literature.

A systematic review of simulation-based economic evaluation (SBEE) studies in opioid research was initiated by searches in PubMed, EMBASE, and EbscoHOST. Extraction of a predefined set of items and a quality assessment were performed for each study.

The screening process resulted in 23 SBEE studies ranging by year of publication from 1999 to 2019. Methodological quality of the cost analyses was moderately high. The most frequently evaluated strategies were methadone and buprenorphine maintenance treatments; the only harm reduction strategy explored was naloxone distribution. These strategies were consistently found to be cost-effective, especially naloxone distribution and methadone maintenance. Prevention strategies were limited to abuse-deterrent opioid formulations. Less than half (39%) of analyses adopted a societal perspective in their estimation of costs and effects from an opioid-related intervention. Prevention strategies and studies’ accounting for patient and physician preference, changing costs, or result stratification were largely ignored in these SBEEs.

The review shows consistently favorable cost analysis findings for naloxone distribution strategies and opioid agonist treatments and identifies major gaps for future research.

economic_evaluation_in_opioid_modeling.pdf supplement_economic_evaluation_in_opioid_modeling.pdf
Why Employees (Still) Click on Phishing Links: Investigation in Hospitals
Jalali MS, Bruckes M, Westmattelmann D, Schewe G. Why Employees (Still) Click on Phishing Links: Investigation in Hospitals. Journal of Medical Internet Research 2020;22(1):e16775.Abstract

Background: Hospitals have been one of the major targets for phishing attacks. Despite efforts to improve information security compliance, hospitals still significantly suffer from such attacks, impacting the quality of care and the safety of patients.

Objective: This study aimed to investigate why hospital employees decide to click on phishing emails by analyzing actual clicking data.

Methods: We first gauged the factors that influence clicking behavior using the theory of planned behavior (TPB) and integrating trust theories. We then conducted a survey in hospitals and used structural equation modeling to investigate the components of compliance intention. We matched employees’ survey results with their actual clicking data from phishing campaigns.

Results: Our analysis (N=397) reveals that TPB factors (attitude, subjective norms, and perceived behavioral control), as well as collective felt trust and trust in information security technology, are positively related to compliance intention. However, compliance intention is not significantly related to compliance behavior. Only the level of employees’ workload is positively associated with the likelihood of employees clicking on a phishing link.

Conclusions: This is one of the few studies in information security and decision making that observed compliance behavior by analyzing clicking data rather than using self-reported data. We show that, in the context of phishing emails, intention and compliance might not be as strongly linked as previously assumed; hence, hospitals must remain vigilant with vulnerabilities that cannot be easily managed. Importantly, given the significant association between workload and noncompliance behavior (ie, clicking on phishing links), hospitals should better manage employees’ workload to increase information security. Our findings can help health care organizations augment employees’ compliance with their cybersecurity policies and reduce the likelihood of clicking on phishing links.

phishing investigation.pdf