This article reviews the evidence on the extent to which experts' perceptions of item difficulties, captured using comparative judgement, can predict empirical item difficulties. This evidence is drawn from existing published studies on this topic and also from statistical analysis of data held by Cambridge Assessment. Having reviewed the…

In the context of examinations, the phrase "maintaining standards" usually refers to any activity designed to ensure that it is no easier (or harder) to achieve a given grade in one year than in another. Specifically, it tends to mean activities associated with setting examination grade boundaries. Benton et al (2020) describes a method…

For well over a century, scientists have investigated individual differences in performance. The majority of studies have focused on either differences in practice, or differences in cognitive resources. However, the predictive ability of either practice or cognitive resources varies considerably across tasks. We are the first to examine task…

This study tests the theoretical predictions regarding effects of increasing task complexity (Robinson, 2001a, 2001b, 2007, 2010; Robinson & Gilabert, 2007) for second language (L2) pronunciation. Specifically, we examine whether more complex tasks (a) lead to greater incidence of pronunciation-focused language-related episodes (LREs) and (b)…

Missing data is a common problem in datasets that are obtained by administration of educational and psychological tests. It is widely known that existence of missing observations in data can lead to serious problems such as biased parameter estimates and inflation of standard errors. Most of the missing data imputation methods are focused on…

This study investigated the effect of complex structure on dimensionality assessment in compensatory multidimensional item response models using DETECT- and NOHARM-based methods. The performance was evaluated via the accuracy of identifying the correct number of dimensions and the ability to accurately recover item groupings using a simple…

The purpose of this study was to test the effects of octave and timbre on advanced college musicians' (N = 63) ability to tune their instruments. We asked: "Are there differences in tuning accuracy due to octave (B-flat 2, B-flat 4) and stimulus timbre (oboe, clarinet, electronic tuner, tuba)?" and "To what extent do participants'…

The ability to learn in a self-regulated way is important for adolescents' academic achievements. Monitoring one's own learning is a prerequisite skill for successful self-regulated learning. However, accurate monitoring has been found to be difficult for adolescents, especially for learning problem-solving tasks such as can be found in math and…

In the context of a generic harmonic oscillator, we investigated students' accuracy in determining the period, frequency, and angular frequency from mathematical and graphical representations. In a series of studies including interviews, free response tests, and multiple-choice tests developed in an iterative process, we assessed students in both…

An adequate level of linguistic complexity in learning materials is believed to be of crucial importance for learning. The implication for school textbooks is that reading complexity should differ systematically between grade levels and between higher and lower tracks in line with what can be called the systematic complexification assumption.…

This inquiry is an investigation of item response theory (IRT) proficiency estimators' accuracy under multistage testing (MST). We chose a two-stage MST design that includes four modules (one at Stage 1, three at Stage 2) and three difficulty paths (low, middle, high). We assembled various two-stage MST panels (i.e., forms) by manipulating two…

As reading and reading assessment become increasingly implemented on electronic devices, the question arises whether reading on screen is comparable with reading on paper. To examine potential differences, we studied reading processes on different proficiency and complexity levels. Specifically, we used data from the standardization sample of the…

This study had two aims. The first was to explain the process of using the Rasch measurement model to validate tests in an easy-to-understand way for those unfamiliar with the Rasch measurement model. The second was to validate two final exams with several shared items. The exams were given to two groups of students with slightly differing English…

The aim of the present study is to compare the writing products of English as a Foreign Language (EFL) university students using accuracy and grammatical complexity as measures. It also aims at examining the development of the morphological, syntactic, lexical, spelling, and punctuation errors committed by these learners when writing in English.…

A major debate in the mental-rotation literature concerns the question of whether objects are represented holistically during rotation. Effects of object complexity on rotational speed are considered strong evidence against such holistic representations. In Experiment 1, such an effect of object complexity was markedly present. A closer look on…