Ramsay and Wiberg used a new version of item response theory that represents test performance over nonnegative closed intervals such as [0, 100] or [0, n] and demonstrated that optimal scoring of binary test data yielded substantial improvements in point-wise root-mean-squared error and bias over number right or sum scoring. We extend these…

With the advent of big data and advances in technology, psychological assessments have become increasingly sophisticated and complex. Nevertheless, traditional psychometric issues concerning the validity, reliability, and measurement bias of such assessments remain fundamental in determining whether score inferences of human attributes are…

The Fundamental Concepts for Organic Reaction Mechanisms Inventory (FC-ORMI) is a concept inventory with most items in a two-tier design in which an answer tier is followed by a reasoning tier. Statistical results provided strong evidence for the validity and reliability of the data obtained using the FC-ORMI. In this study, differential item…

The waning interest of students in science became a global concern. The purpose of this research was to translate, adapt, and validate the My Attitude toward Science [MATS] questionnaire instrument, which was used to measure students' attitudes toward science in the Indonesian context. We also investigated the items that contributed to gender and…

Argumentation is fundamental to science education, both as a prominent feature of scientific reasoning and as an effective mode of learning--a perspective reflected in contemporary frameworks and standards. The successful implementation of argumentation in school science, however, requires a paradigm shift in science assessment from the…

This study evaluated differential item functioning (DIF) in achievement motivation items before and after using anchoring vignettes as a statistical tool to account for group differences in response styles across gender and ethnicity. We applied the nonparametric scoring of the vignettes to motivation items from the 2015 Programme for…

Conducting a fair test is important for educational research. Unfair assessments can lead to gender disparities in academic achievement, ultimately resulting in disparities in opportunities, wages, and career choice. Differential Item Function [DIF] analysis is presented to provide evidence of whether the test is truly fair, where it does not harm…

The COVID-19 pandemic had a profound impact on education, forcing many teachers and students who were not used to online education to adapt to an unanticipated reality by improvising new teaching and learning methods. Within the realm of virtual education, the evaluation methods underwent a transformation, with some assessments shifting towards…

Human judgement is widely used in workplace-based assessment despite criticism that it does not meet standards of objectivity. There is an ongoing push within the literature to better embrace subjective human judgement in assessment not as a 'problem' to be corrected psychometrically but as legitimate perceptions of performance. Taking a step back…

The use and interpretation of educational and psychological test scores are paramount to individual outcomes and opportunities. Methods for detecting differential item functioning (DIF) are imperative for item analysis when developing and revising assessments, particularly as it pertains to fairness across populations, languages, and cultures. We…

Background: Research about the psychometric properties of the Behavioural Appraisal Scales (BAS) in people with profound intellectual and multiple disabilities (PIMD) is limited. This study evaluates invariance in factor structure, item bias and convergent validity of the BAS. Methods: Data on the BAS from two studies (n = 25; n = 52) were…

How do teachers develop their understanding of the foundation principles of assessment, stay up to date with the latest classroom approaches and have the confidence to evaluate and question the effectiveness of new methods? This professional resource for teachers supports them to understand the what, why and how of assessment. It provides key…

This article extends multivariate generalizability theory (MGT) to tests with different random-effects designs for each level of a fixed facet. There are numerous situations in which the design of a test and the resulting data structure are not definable by a single design. One example is mixed-format tests that are composed of multiple-choice and…

This study used item response theory (IRT) methods to investigate group differences in responses to the 12-item Student Risk Screening Scale--Internalizing and Externalizing (SRSS-IE12) in a sample of 3,837 U.S. elementary school students. Using factor analysis and graded response models from IRT methods, we examined the factor structure and the…

One of the primary goals of international large-scale assessments in education is the comparison of country means in student achievement. This article introduces a framework for discussing differential item functioning (DIF) for such mean comparisons. We compare three different linking methods: concurrent scaling based on full invariance,…