Job and practice analysis is a commonly used method for determining examination content specifications. However, difficulties arise when many domains are present, as mainstream approaches do not fully adhere to the essence of the weighing process, namely a "comparison-evaluation-decision" framework for assigning percentage values to the…

The purpose of this quantitative, non-experimental, correlational study was to determine if there was a correlation between first-time pass rates on the USMLE Step 1 and the use of active learning strategies in medical school classrooms. For the utility of this study, active learning strategies were defined as any activity in which medical…

With the increased restrictions on physical distancing due to the COVID-19 pandemic, remote proctoring has emerged as an alternative to traditional onsite proctoring to ensure the continuity of essential assessments, such as computer-based medical licensing exams. Recent literature has highlighted the significant impact of different proctoring…

The purpose of this quantitative study was to explore the impact the COVID-19 pandemic had on medical education by comparing USMLE Step 1 first attempt pass rates of examinees from US/Canadian schools in Doctor of Medicine (MD) programs from 2020-2022 to prior years' pass first attempt pass rates (2017-2019). Additionally, the effectiveness of the…

To address the computational constraints of parameter estimation in the polytomous Cognitive Diagnosis Model (pCDM) in large-scale high data volume situations, this study proposes two two-stage polytomous attribute estimation methods: P_max and P_linear. The effects of the two-stage methods were studied via a Monte Carlo simulation study, and the…

Despite extensive studies on large language models and their capability to respond to questions from various licensed exams, there has been limited focus on employing chatbots for specific subjects within the medical curriculum, specifically medical neuroscience. This research compared the performances of Claude 3.5 Sonnet (Anthropic), GPT-3.5 and…

This paper reports the findings of a Canada based multi-institutional study designed to investigate the relationships between admissions criteria, in-program assessments, and performance on licensing exams. The study's objective is to provide valuable insights for improving educational practices across different institutions. Data were gathered…

Purpose: Our US medical school uses National Board of Medical Examiners (NBME) tests as progress tests during the pre-clerkship curriculum to assess students. In this study, we examined students' growth patterns using progress tests in the first year of medical school to identify students at risk for failing United States Medical Licensing…

The COVID-19 pandemic severely disrupted assessment models that were commonplace in the testing industry for decades. As a response to this disturbance, remote proctoring has emerged as a promising and potentially sound alternative to offer examinations, while adhering to public health authority guidelines. However, validity evidence in support of…

Anatomy is essential for understanding healthy and disease states as well as for the successful completion of clinical clerkships and board examinations. This project provided structured workshops aimed to review anatomical concepts for clerkships and Step 1 and provided a means for medical students to assess their anatomical knowledge. We…

Retrieval practice improves long-term retention. Use of interactive retrieval practice in large group, in-person and online live classes, in combination with outside resources, is unreported for medical physiology classes. The primary study purpose was to compare student cohorts' performance with or without retrieval practice in renal physiology…

China's Clinical Medicine Level Test (CMLT) was designed to evaluate the key factors that affect the learning outcome of medical students prior to entering clinical practice. In this study, we systemically analyzed the performance of a cohort of medical students at Jinan University School of Medicine participating in the recent CMLT. The…

West Virginia's three state-funded medical schools enroll more medical students per capita than any other state in the country. Due to its large number of medical student slots, the state typically can offer all qualified West Virginians the opportunity to complete their medical education in the state. In the academic year of 2023, 25 percent…

Dynamic Measurement Modeling (DMM) is a psychometric paradigm that uses longitudinal data to estimate students' capacity to learn over the course of an educational program (i.e., growth scores). Here, we provide justification for this approach in health professions education and demonstrate its proof of concept with three time-points of USMLE Step…

Test administrators are appropriately concerned about the potential for time constraints to impact the validity of score interpretations; psychometric efforts to evaluate the impact of speededness date back more than half a century. The widespread move to computerized test delivery has led to the development of new approaches to evaluating how…