The extant literature has largely ignored a potentially significant source of variance in multiple mini-interview (MMI) scores by "hiding" the variance attributable to the sample of attributes used on an evaluation form. This potential source of hidden variance can be defined as rating items, which typically comprise an MMI evaluation…

This study investigates the effects of sample size and test length on item-parameter estimation in test development utilizing three unidimensional dichotomous models of item response theory (IRT). For this purpose, a real language test comprised of 50 items was administered to 6,288 students. Data from this test was used to obtain data sets of…

With integrated curricula and multidisciplinary assessments becoming more prevalent in medical education, there is a continued need for educational research to explore the advantages, consequences, and challenges of integration practices. This retrospective analysis investigated the number of items needed to reliably assess anatomical knowledge in…

Utilizing a longitudinal item response model, this study investigated the effect of item parameter drift (IPD) on item parameters and person scores via a Monte Carlo study. Item parameter recovery was investigated for various IPD patterns in terms of bias and root mean-square error (RMSE), and percentage of time the 95% confidence interval covered…

The comparison of item-level computerized adaptive testing (CAT) and multistage adaptive testing (MST) has been researched extensively (e.g., Kim & Plake, 1993; Luecht et al., 1996; Patsula, 1999; Jodoin, 2003; Hambleton & Xing, 2006; Keng, 2008; Zheng, 2012). Various CAT and MST designs have been investigated and compared under the same…

In Italy, from the 2003 reports to the present, the National Institute for the Educational Evaluation of Instruction and Training (INVALSI) has conducted research on Programme for International Student Assessment (PISA) results in order to understand Italian students' low achievement in mathematics. In the present paper, data from a representative…

The current simulation study examined the effects of Item Parameter Drift (IPD) occurring in a short scale on parameter estimates in multilevel models where scores from a scale were employed as a time-varying predictor to account for outcome scores. Five factors, including three decisions about IPD, were considered for simulation conditions. It…

This study aimed to develop a three-dimensional geometric thinking test to determine the geometric thinking of early graders in the paper-pencil environment. First, we determined the components of three-dimensional geometric thinking and prepared questions for each component. Then, we conducted the pilot studies of the test at three stages in six…

This study focused on the effect of examinees' ability levels on the relationship between Reflective-Impulsive (RI) cognitive style and item response time in computerized adaptive testing (CAT). The total of 56 students majoring in Educational Technology from Shandong Normal University participated in this study, and their RI cognitive styles were…

Valid indigenous language assessments are needed to further our understanding of how children learn language around the world. We assessed the psychometric properties and performance of two caregiver-report measures of Wolof language skill (language milestones achieved and vocabulary knowledge) for 500 children (ages 0;4 to 2;6) living in rural…

Analyzing students' growth remains an important topic in educational research. Most recently, Diagnostic Classification Models (DCMs) have been used to track skill acquisition in a longitudinal fashion, with the purpose to provide an estimate of students' learning trajectories in terms of the change of fine-grained skills overtime. Response time…

Cognates are known to facilitate second language acquisition and use, as learners tend to assign to a new L2 word the meaning of a similar L1 word. Consequently, for L2 tests that rely largely on lexical items, performance may prove inflated for examinees whose L1 shares many cognates with the language being tested. This article examines the…

This article draws on three sources of evidence that together indicate hermeneutical weaknesses in exam courses on Christianity in English Religious Education (RE). It scrutinises a single exam paper and an associated text book from a recent authorised course. It conceptually explores features of a new style of long Religious Studies (RS) exam…

This article examines the test-taking strategies of high- and low-scoring Chinese-speaking participants when they answer English multiple-choice reading comprehension questions. Thirty-two participants took a TOEIC reading test, provided think-aloud protocols, and joined a post-task interview. The data come primarily from qualitative analysis and…

The aim of this simulation study, determine the relationship between true latent scores and estimated latent scores by including various control variables and different statistical models. The study also aimed to compare the statistical models and determine the effects of different distribution types, response formats and sample sizes on latent…