BACKGROUND: Linguistic and cultural discordance between clinicians and patients contributes to suboptimal care of Latinx patients with diabetes. Bilingual pedagogies may help learners with pre-existing Spanish skills to improve health communication with linguistic minorities.
APPROACH: We designed a diabetes workshop for health professions students that applied the educational principles of bilingual pedagogies, focus on learners with prior Spanish skills, and intersectionality of language and culture. The session introduced diabetes concepts in Spanish and English, applied Latinx cultural beliefs to clinical conversations about diabetes and explored the impact of clinician language and cultural skills on diabetes-related communication.
EVALUATION: Voluntary surveys evaluated learner knowledge, confidence and attitudes before and after the intervention. Of 60 attendees, 57 participants (95%) completed surveys. Comparison of pre and post responses yielded statistically significant increases in respondents' cultural knowledge and confidence in explaining diabetes concepts in Spanish (all p < 0.001). When controlling for Spanish level, most items still showed a significant improvement. Latinx and heritage learners were more likely to express lower confidence in explaining diabetes post-workshop (p < 0.05), suggesting an increased awareness of limitations.
IMPLICATIONS: Bilingual educational interventions that emphasise culturally appropriate clinical skills may be an effective way to prepare health professions students who are heritage Spanish or second-language learners to better communicate with Latinx patients about diabetes. We provide actionable recommendations for clinical educators interested in incorporating communication skills training for linguistically diverse patient care.
Background and study aims Studies have linked cecal intubation rate with adenoma detection rate; however, the direct association between technical performance during colonoscopy intubation and withdrawal has never been explored. Thus, it remains unclear whether gentle and efficient intubation predicts superior mucosal inspection. The aim of this study was to investigate the correlation between performance during intubation and withdrawal in a simulation-based setup. Methods Twenty-four physicians with various experience in colonoscopy performed twice on the Endoscopy Training System (ETS). Intubation skills were evaluated by assessing tasks on the ETS related to intubation (scope manipulation and loop management) and use of a computerized assessment tool called the 3D-Colonoscopy Progression Score (3D-CoPS). Diagnostic accuracy was defined by the number of polyps found during the ETS task of mucosal inspection. Pearson's correlations were calculated to explore associations between intubation skill and diagnostic accuracy. Results The correlation analysis between 3D-CoPS and number of polyps found during mucosal inspection revealed a weak and insignificant correlation (0.157, P = 0.3). Likewise, an insignificant correlation was seen between ETS intubation and number of polyps found (0.149, P = 0.32). Conclusions We found no evidence to support that technical performance during intubation is correlated with mucosal inspection performance in a simulation-based setting.
Introduction: Hispanic/Latinx patients experience health disparities in endocrine disease, such as higher rates of mortality due to diabetes mellitus, worse outcomes in the surgical treatment of thyroid cancer, and lack of knowledge about bone health and osteoporotic fracture prevention. Educational strategies to teach cultural and linguistic medical Spanish knowledge to medical students have the potential to improve Hispanic/Latinx endocrine health.
Methods: We created an 8-hour medical Spanish endocrine module targeting language and cultural skills acquisition. Specifically, students practiced obtaining a past medical history, obtaining a medications history, providing and explaining a diagnosis, explaining discharge instructions, and discussing sociocultural aspects of endocrine health. We divided the module into four 2-hour sessions: (1) preclass self-study, (2) class period, (3) role-play/interviewing practice session, and (4) case/cultural activity assignment. Participants completed a pre- and postassessment to measure student confidence level and knowledge.
Results: We implemented the module four times at one medical school, with 47 fourth-year medical students with intermediate or higher general Spanish skills. Confidence increased for all learners in the endocrine-focused interview and exam in Spanish. Heritage Spanish learners, who were exposed to Spanish at home as children, reported higher postmodule confidence in eliciting the past medical history of endocrine problems.
Discussion: The medical Spanish endocrine module improved the communication skills of medical students with intermediate through advanced Spanish proficiency. Future study should evaluate learner clinical performance metrics and applications to other groups of learners, such as resident physicians or health professions involved in the care of patients with endocrine disease.
CONTEXT: Competency-based assessment of learners may benefit from a more holistic, inclusive, approach for determining readiness for unsupervised practice. However, despite movements toward greater patient partnership in healthcare generally, inclusion of patients in postgraduate medical learners' assessment is largely absent.
METHODS: We conducted a scoping review to map the nature, extent and range of literature examining the inclusion (or exclusion) of patients within the assessment of postgraduate medical learners. Guided by Arskey and O'Malley's framework and informed by Levac et al. and Thomas et al., we searched 2 databases (MEDLINE® and Embase®) from inception until February 2021 using subheadings related to assessment, patients and postgraduate learners. Data analysis examined characteristics regarding the nature and factor influencing patient involvement in assessment.
RESULTS: We identified 41 papers spanning four decades. Some literature suggests patients are willing to be engaged in assessment, however, choose not to engage when for example, language barriers may exist. When stratified by specialty or clinical setting, the influence of factors such as gender, race, ethnicity, or medical condition, seem to remain consistent. Patients may participate in assessment as a standalone group or part of a multisource feedback process. Patients generally provided high ratings but commented on the observed professional behaviors and communication skills in comparison to physicians who focused on medical expertise.
CONCLUSION: Factors that influence patient involvement in assessment are multifactorial including patients' willingness themselves, language and reading-comprehension challenges and available resources for training programs to facilitate the integration of patient assessments. These barriers however are not insurmountable. While understudied, research examining patient involvement in assessment is increasing; however, our review suggests that the extent which the unique insights will be taken up in postgraduate medical education may be dependent on assessment systems readiness, and in particular, physician readiness to partner with patients in this way.
Introduction Patient-clinician communication is a key factor in patient satisfaction with care. Clinicians take medical language courses to improve communication with linguistically diverse populations, yet little is known about how patients perceive clinicians' skills. Methods We designed a prospective, comparative survey study of patient perception of clinician communication using a convenience sampling of health professionals enrolled in an interprofessional medical Spanish course. We analyzed the patient-reported quality of communication skills from 214 clinical encounters and self-evaluations of 18 clinicians with Spanish- and English-speaking patients. Results Communication scores were lower for Spanish vs. English encounters as reported by both patients and clinicians (p<0.001). Clinician-reported scores were lower than patient-reported scores in Spanish encounters (9.05±0.23 vs. 8.05±0.23; p<0.001), whereas there was no difference in English encounters (11.17±0.15 vs. 11.35±0.19; p=0.914). The effect of language remained significant (p<0.001) when controlling for medical setting and complexity. Conclusion Spanish-speaking patients report lower-quality communication from clinicians learning Spanish than do English-speaking patients. Incorporating and further evaluating patient perceptions of clinician Spanish communication skills may improve language-appropriate healthcare and clinician education.
BACKGROUND: The coronavirus pandemic continues to shake the embedded structures of traditional in-person education across all learning levels and across the globe. In healthcare simulation, the pandemic tested the innovative and technological capabilities of simulation programs, educators, operations staff, and administration. This study aimed to answer the question: What is the state of distance simulation practice in 2021?
METHODS: This was an IRB-approved, 34-item open survey for any profession involved in healthcare simulation disseminated widely and internationally in seven languages from January 14, 2021, to March 3, 2021. Development followed a multistep process of expert design, testing, piloting, translation, and recruitment. The survey asked questions to understand: Who was using distance simulation? What driving factors motivated programs to initiate distance sim? For what purposes was distance sim being used? What specific types or modalities of distance simulation were occurring? How was it being used (i.e., modalities, blending of technology and resources and location)? How did the early part of the pandemic differ from the latter half of 2020 and early 2021? What information would best support future distance simulation education? Data were cleaned, compiled, and analyzed for dichotomized responses, reporting frequencies, proportions, as well as a comparison of response proportions.
RESULTS: From 32 countries, 618 respondents were included in the analysis. The findings included insights into the prevalence of distance simulation before, during, and after the pandemic; drivers for using distance simulation; methods and modalities of distance simulation; and staff training. The majority of respondents (70%) reported that their simulation center was conducting distance simulation. Significantly more respondents indicated long-term plans for maintaining a hybrid format (82%), relative to going back to in-person simulation (11%, p < 0.001).
CONCLUSION: This study gives a perspective into the rapid adaptation of the healthcare simulation community towards distance teaching and learning in reaction to a radical and quick change in education conditions and environment caused by COVID-19, as well as future directions to pursue understanding and support of distance simulation.
BACKGROUND: Nearly all U.S. medical students engage in a 4-8 week period of intense preparation for their first-level licensure exams, termed a "dedicated preparation period" (DPP). It is widely assumed that student well-being is harmed during DPPs, but evidence is limited. This study characterized students' physical, intellectual, emotional, and social well-being during DPPs.
METHODS: This was a cross-sectional survey sent electronically to all second-year students at four U.S. medical schools after each school's respective DPP for USMLE Step 1 or COMLEX Level 1 in 2019. Survey items assessed DPP characteristics, cost of resources, and perceived financial strain as predictors for 18 outcomes measured by items with Likert-type response options. Open-ended responses on DPPs' influence underwent thematic analysis.
RESULTS: A total of 314/750 (42%) students completed surveys. DPPs lasted a median of 7 weeks (IQR 6-8 weeks), and students spent 70 h/week (IQR 56-80 h/week) studying. A total of 62 (20%) reported experiencing a significant life event that impacted their ability to study during their DPPs. Most reported 2 outcomes improved: medical knowledge base (95%) and confidence in ability to care for patients (56%). Most reported 9 outcomes worsened, including overall quality of life (72%), feeling burned out (77%), and personal anxiety (81%). A total of 25% reported paying for preparation materials strained their finances. Greater perceived financial strain was associated with worsening 11 outcomes, with reported amount spent associated with worsening 2 outcomes. Themes from student descriptions of how DPPs for first-level exams influenced them included (1) opportunity for synthesis of medical knowledge, (2) exercise of endurance and self-discipline required for professional practice, (3) dissonance among exam preparation resource content, formal curriculum, and professional values, (4) isolation, deprivation, and anguish from competing for the highest possible score, and (5) effects on well-being after DPPs.
CONCLUSIONS: DPPs are currently experienced by many students as a period of personal and social deprivation, which may be worsened by perceived financial stress more than the amount of money they spend on preparation materials. DPPs should be considered as a target for reform as medical educators attempt to prevent student suffering and enhance their well-being.
BACKGROUND: Low first-time pass rates of the Fundamentals of Endoscopic Surgery (FES) exam stimulated development of virtual reality (VR) simulation curricula for test preparation. This study evaluates the transfer of VR endoscopy training to live porcine endoscopy performance and compares the relative effectiveness of a proficiency-based vs repetition-based VR training curriculum.
METHODS: Novice endoscopists completed pretesting including the FES manual skills examination and Global Assessment of GI Endoscopic Skills (GAGES) assessment of porcine upper and lower endoscopy. Participants were randomly assigned one of two curricula: proficiency-based or repetition-based. Following curriculum completion, participants post-tested via repeat FES examination and GAGES porcine endoscopy assessments. The two cohorts pre-to-post-test differences were compared using ANCOVA.
RESULTS: Twenty-two residents completed the curricula. There were no differences in demographics or clinical endoscopy experience between the groups. The repetition group spent significantly more time on the simulator (repetition: 242.2 min, SD 48.6) compared to the proficiency group (proficiency: 170.0 min, SD 66.3; p = 0.013). There was a significant improvement in porcine endoscopy (pre: 10.6, SD 2.8, post: 16.6, SD 3.4; p < 0.001) and colonoscopy (pre: 10.4, SD 2.7, post: 16.4, SD 4.2; p < 0.001) GAGES scores as well as FES manual skills performance (pre: 270.9, SD 105.5, post: 477.4, SD 68.9; p < 0.001) for the total cohort. There was no difference in post-test GAGES performance or FES manual skills exam performance between the two groups. Both the proficiency and repetition group had a 100% pass rate on the FES skills exam following VR curriculum completion.
CONCLUSION: A VR endoscopy curriculum translates to improved performance in upper and lower endoscopy in a live animal model. VR curricula type did not affect FES manual skills examination or live colonoscopy outcomes; however, a proficiency curriculum is less time-consuming and can provide a structured approach to prepare for both the FES exam and clinical endoscopy.
OBJECTIVE: To assess whether an integrated Advanced Modular Manikin (AMM) provides improved participant experience compared to use of peripheral simulators alone, during a standardized trauma team scenario.
BACKGROUND: Simulation-based team training has been shown to improve team performance. To address limitations of existing manikin simulators the AMM platform was created, that enables interconnectedness, interoperability and integration of multiple simulators ("peripherals") into an adaptable, comprehensive training system.
METHODS: A randomized single-blinded, cross-over study with two conditions was employed to assess learner experience differences when using the integrated AMM platform versus peripheral simulators. First responders, anesthesiologists, and surgeons rated their experience and workload with the conditions in a 3-scene standardized trauma scenario. Participant ratings were compared and focus groups conducted to obtain insight into participant experience.
RESULTS: Fourteen teams (n=42) participated. Team experience ratings were higher for the integrated AMM condition compared with peripherals (Cohen's d = .25, p = .016). Participant experience varied by background with surgeons and first responders rating their experience significantly higher compared with anesthesiologists (p < .001). Higher workload ratings were observed with the integrated AMM condition (Cohen's d = .35, p = .014) driven primarily by anesthesiologist ratings. Focus groups revealed that participants preferred the integrated AMM condition based upon its increased realism, physiologic responsiveness, and feedback provided on their interventions.
CONCLUSION: This first comprehensive evaluation suggests that integration with the AMM platform provides benefits over individual peripheral simulators and has the potential to expand simulation-based learning opportunities and enhance learner experience, especially for surgeons.
Background: The transition to residency marks a significant shift in the financial circumstances of medical trainees. Despite existing resources, residents still cite uncertainty in this domain. A personal finance curriculum is needed to close this educational gap and improve the financial well-being of trainees.
Methods: The curriculum was developed using Kern's framework. Two needs assessments informed the consensus development of goals and objectives, educational strategies, and assessments. Course material was hosted online for asynchronous review and complemented by two 1-hour webinars. The curriculum was piloted at one institution. Participants completed (1) knowledge assessments before and after the intervention, (2) a survey of reactions to the curriculum, and (3) an assessment of financial behavioral changes after the intervention.
Results: Thirty-seven residents (37/49, 76%) enrolled in the curriculum. Among participants, 20 (20/37, 54%) completed the curriculum. Most participants agreed or strongly agreed that the content was relevant (20/20, 100%) and clearly presented (19/20, 95%) and that they would recommend the curriculum to other residents (20/20, 100%). Performance on the knowledge assessment improved 21% after the intervention (mean ± SD = pretest 57% ± 17%, posttest = 78% ± 12%; p < 0.001). Most residents (17/20, 85%) also reported behavioral changes including setting new financial goals (12/20, 60%), taking new action toward financial planning (11/20, 55%), and changing financial habits (6/20, 30%). There were no direct financial costs incurred in the implementation of this pilot.
Conclusions: This is a successful pilot of a virtual personal finance curriculum with positive outcomes data. Addressing this problem at scale will require buy-in from educators around the country to deliver this information to residents that may not otherwise seek it out. Future study should assess curricular outcomes in other settings and the durability of acquired knowledge and behavioral changes over time.
OBJECTIVE: To gather validity evidence supporting the use and interpretation of scores from the American College of Surgeons Entering Resident Readiness Assessment (ACS ERRA) Program.
SUMMARY AND BACKGROUND DATA: ACS ERRA is an online formative assessment program developed to assess entering surgery residents' ability to make critical clinical decisions, and includes 12 clinical areas and 20 topics identified by a national panel of surgeon educators and residency program directors.
METHODS: Data from three national testing administrations of ACS ERRA (2018-20) were used to gather validity evidence regarding content, response process, internal structure (reliability), relations to other variables, and consequences.
RESULTS: Over the three administrations, 1,975 surgery residents participated from 125 distinct residency programs. Overall scores (Mean = 64% [SD = 7%]) remained consistent across the three years (p = .670). There were no significant differences among resident characteristics (gender, age, IMG status). The mean case discrimination index was 0.54 [SD = .15]. Kappa inter-rater reliability for scoring was 0.87; the overall test score reliability (G-coefficient) was 0.86 (Φ-coefficient = 0.83). Residents who completed residency readiness programs had higher ACS ERRA scores (66% vs. 63%, Cohen's d = 0.23, p < .001). On average, 15% of decisions made (21/140 per test) involved potentially harmful actions. Variability in scores from graduating medical schools (7%) carried over twice as much weight than from matched residency programs (3%).
CONCLUSIONS: ACS ERRA scores provide valuable information to entering surgery residents and surgery program directors to aid in development of individual and group learning plans.
PURPOSE: Systematic differences among raters' approaches to student assessment may result in leniency or stringency of assessment scores. This study examines the generalizability of medical student workplace-based competency assessments including the impact of rater-adjusted scores for leniency and stringency.
METHODS: Data were collected from summative clerkship assessments completed for 204 students during 2017-2018 the clerkship at a single institution. Generalizability theory was used to explore variance attributed to different facets (rater, learner, item, and competency domain) through three unbalanced random-effects models by clerkship including applying assessor stringency-leniency adjustments.
RESULTS: In the original assessments, only 4-8% of the variance was attributed to the student with the remainder being rater variance and error. Aggregating items to create a composite score increased variability attributable to the student (5-13% of variance). Applying a stringency-leniency ('hawk-dove') correction substantially increased the variance attributed to the student (14.8-17.8%) and reliability. Controlling for assessor leniency/stringency reduced measurement error, decreasing the number of assessments required for generalizability from 16-50 to 11-14.
CONCLUSIONS: Similar to prior research, most of the variance in competency assessment scores was attributable to raters, with only a small proportion attributed to the student. Making stringency-leniency corrections using rater-adjusted scores improved the psychometric characteristics of assessment scores.
BACKGROUND AND AIMS: The success of preventing colorectal cancer relies on the expertise of the colonoscopists. Studies suggest that the retraction technique is a powerful indicator of expertise in distinguishing endoscopists with various adenoma detection rates (ADRs). We aimed to develop a retraction technique score and explore the correlation between endoscopists' retraction technique and their ADR.
METHODS: In a prospective, multicenter study, 8 colonoscopist nurses and physicians with various ADRs were included. Data from patients admitted for a colonoscopy, as part of the Danish Nationwide screening program, were gathered directly from the Olympus ScopeGuide system providing XYZ-coordinates from the coils along the length of the colonoscope. Motor skill measures were developed based on Tip Retraction, Retraction Efficiency, and Retraction Distance. The principal component analysis was used to study the association between the 3 measures and the historical ADR to create a combined score, the Colonoscopy Retraction Score (CoRS).
RESULTS: A total of 333 recordings were analyzed. We demonstrated a significant and strong correlation between CoRS and ADR (0.90, p < 0.01). Contrary, withdrawal time did not correlate significantly with ADR (0.33, p=0.42). In procedures without polypectomies or biopsies, a significant and strong correlation was found between CoRS and ADR (0.88, p<0.01) and between withdrawal time and ADR (0.75, p=0.03).
CONCLUSION: This study presents a novel, real-time computerized and unbiased assessment tool for colonoscopy withdrawal. CoRS strongly correlated with ADR with and without therapeutical interventions during withdrawal and could be used to ensure quality instead of minimal withdrawal time.
CONTEXT: The Japan Residency Matching Program (JRMP) launched in 2003 and is now a significant event for graduating medical students and postgraduate residency hospitals. The environment surrounding JRMP changed due to Japanese health policy, resulting in an increase in the number of unsuccessfully-matched students in the JRMP. Beyond policy issues, we suspected there were also common characteristics among the students who do not get a match with residency hospitals.
METHODS: In total 237 out of 321 students at The University of Tokyo Faculty of Medicine graduates from 2018 to 2020 participated in the study. The students answered to the questionnaire and gave written consent for using their personal information including the JRMP placement, scores of the pre-clinical clerkship (CC) Objective Structured Clinical Examinations (OSCE), the Computer-Based Test (CBT), the National Board Examination (NBE), and domestic scores for this study. The collected data were statistically analyzed.
RESULTS: The JRMP placements were correlated with some of the pre-CC OSCE factors/stations and/or total scores/global scores. Above all, the result of neurological examination station had most significant correlation between the JRMP placements. On the other hand, the CBT result had no correlation with the JRMP results. The CBT results had significant correlation between the NBE results.
CONCLUSIONS: Our data suggest that the pre-clinical clerkship OSCE score and the CBT score, both undertaken before the clinical clerkship, predict important outcomes including the JRMP and the NBE. These results also suggest that the educational resources should be intensively put on those who did not make good scores in the pre-clinical clerkship OSCE and the CBT to avoid the failure in the JRMP and the NBE.
PURPOSE: To identify the content of an educational handover letter from undergraduate to graduate education in General Surgery.
METHOD: Expert consensus was attained on the content of an educational handover letter. A three stage Delphi technique was employed with eight experts in each of four stakeholder groups: program directors in general surgery, medical student surgical acting internship or prep course directors, authors of medical student performance evaluations and current categorical General Surgery residents. Data were collected from April through July 2019. A mixed-method analysis was performed to quantitatively assess items selected for inclusion and qualitatively provide guidance for the implantation of such a letter.
RESULTS: All 32 experts participated in at least one round. Of the 285 initially identified individual items 22 were ultimately selected for inclusion in the letter. All but one expert agreed that the list represents what the content of an educational handover letter in General Surgery should be. Qualitative analysis was performed on 395 comments and identified four themes to guide the implementation of the letter: "minimize redundancy, optimize impact, use appropriate assessments and mitigate risk."
CONCLUSIONS: A framework and proposed template are provided for an educational handover letter from undergraduate to graduate medical education in General Surgery based on the quantitative and qualitative analysis of expert consensus of major stakeholders. This letter holds promise to enhance the transition from undergraduate to graduate medical education by allowing programs to capitalize on strengths and efficiently address knowledge gaps in new trainees.
Some communities recover more quickly after a disaster than others. Some differentials in recovery are explained by variation in the level of disaster-related community damage and differences in pre-disaster community characteristics, e.g., the quality of housing stock. But distinct communities that are similar on the above characteristics may experience different recovery trajectories, and, if so, these different trajectories must be due to more subtle differences among them. Our principal objective is to assess short-term and long-term post-disaster mental health for Vietnamese and African Americans living in two adjacent communities in eastern New Orleans that were similarly flooded by Hurricane Katrina. We employ data from two population-based cohort studies that include a sample of African American adults (the Gulf Coast Child and Family Health [GCAFH study]) and a sample of Vietnamese American adults (Katrina Impacts on Vietnamese Americans [KATIVA NOLA study]) living in adjacent neighborhoods in eastern New Orleans who were assessed near the second and thirteenth anniversaries of the disaster. Using the 12-Item Short Form Survey (SF-12) as the basis of our outcome measure, we find in multivariate analysis a significant advantage in post-disaster mental health for Vietnamese Americans over their African American counterparts at the two-year mark, but that this advantage had disappeared by the thirteenth anniversary of the Katrina disaster.
BACKGROUND: Opioid misuse, overprescribing, dependency, and overdose remains a significant concern in the United States. A quality improvement study was conducted at the University of Illinois Hospital & Health Sciences System to determine the effect of standardizing the default orders for hydrocodone-acetaminophen products implemented on June 22, 2016.
METHODS: Prior to the intervention, default orders had variable dose tablet numbers (1 or 2) and dosing frequencies (every 4 or 6 hours), and no default dispense quantity. Defaults were modified to 1 tablet every 6 hours as needed for pain and dispense quantities of 3 and 5 days' supply were added. Number of tablets per order, dosing frequency, and days' supply prescribed between January 1, 2016, and June 21, 2016, were compared to those placed between June 22, 2016, and December 31, 2016. Opioid doses were converted into morphine milligram equivalents (MME). Analyses were performed to determine the effect of the intervention on daily opioid dose and number of days' supply prescribed.
RESULTS: 22,052 orders were included in this study. Following the intervention, the number of tablets prescribed was reduced by an average of 19,832 tablets per month. Every 6 hours dosing (as opposed to every 4 hours) increased by 21.52 percentage points. Prescriptions with ≥ 50 MME/day dropped by 5.8 percentage points, and > 3 days' supply decreased by 2.54 percentage points. Linear regression demonstrated an increase in opioid prescriptions with daily < 50 MME (odds ratio [OR] = 1.72, p < 0.001) and ≤ 3 days' supply (OR = 1.27, p < 0.001).
CONCLUSION: Default electronic health record settings strongly influence prescribing patterns.
PURPOSE: Assessment of the Core Entrustable Professional Activities for Entering Residency (Core EPAs) requires direct observation of learners in the workplace to support entrustment decisions. The purpose of this study was to examine the internal structure validity evidence of the Ottawa Surgical Competency Operating Room Evaluation (O-SCORE) scale when used to assess medical student performance in the Core EPAs across clinical clerkships.
METHOD: During the 2018-2019 academic year, the Virginia Commonwealth University School of Medicine implemented a mobile-friendly, student-initiated workplace-based assessment (WBA) system to provide formative feedback for the Core EPAs across all clinical clerkships. Students were required to request a specified number of Core EPA assessments in each clerkship. A modified O-SCORE scale (1= "I had to do" to 4 = "I needed to be in room just in case") was used to rate learner performance. Generalizability theory was applied to assess the generalizability (or reliability) of the assessments. Decision studies were then conducted to determine the number of assessments needed to achieve a reasonable reliability.
RESULTS: A total of 10,680 WBAs were completed on 220 medical students. The majority of ratings were completed on EPA 1 (history and physical) (n = 3,129; 29%) and EPA 6 (oral presentation) (n = 2,830; 26%). Mean scores were similar (3.5-3.6 out of 4) across EPAs. Variance due to the student ranged from 3.5% to 8%, with the majority of the variation due to the rater (29.6%-50.3%) and other unexplained factors. A range of 25 to 63 assessments were required to achieve reasonable reliability (Phi > 0.70).
CONCLUSIONS: The O-SCORE demonstrated modest reliability when used across clerkships. These findings highlight specific challenges for implementing WBAs for the Core EPAs including the process for requesting WBAs, rater training, and application of the O-SCORE scale in medical student assessment.
OBJECTIVE: To determine if simulation training is required to pass the Fundamentals of Endoscopic Surgery (FES) skills test and assess the relationship between simulation training, clinical training, and FES skills test performance.
SUMMARY AND BACKGROUND DATA: The ABS began requiring completion of the Flexible Endoscopy Curriculum (FEC) for all applicants beginning in 2018. The role of simulation-based training in FES skills test performance after this requirement has not been evaluated.
METHODS: De-identified data from the initial FES skills tests after the FEC requirement was reviewed, and 731 unique participants with reported simulation experience demographics were identified. Self reported data included gender, upper (UE) and lower (LE) endoscopy experience, and simulator training hours (SE). Final FES skills exam scores and pass/fail designations for each participant were reported by the FES program staff.
RESULTS: There was a statistically discernible difference in mean FES total scores between those reporting no SE and more experienced groups (p = 0.002), and between less and more experienced UE and LE groups (p < 0.001). There was no statistically discernible difference in FES skills exam pass rates between SE groups (p = 0.2), but there was a strong relationship between clinical experience (UE & LE) and pass rate (p < 0.001). Finally, on logistic regression analysis, LE was a discernible predictor of passing (OR = 1.4, 95% CI 1.1-1.8, p = 0.02), while UE (OR = 1, 95% CI 0.8-1.3, p = 0.9) and SE (OR = 1, 95% CI 0.9-1.3, p = 0.7) were not.
CONCLUSIONS: There is no threat to the validity of the FES skills test from a need for simulation training to pass the FES skills test. Similarly, the amount of simulation practice is not predictive of passing, but can improve performance on certain FES tasks.