In this article, the authors applied profile analysis to Maths exam data to demonstrate how different exam forms, differing in difficulty and length, can be reported and easily interpreted. The results were presented for different groups of participants and for different institutions in different Maths domains by evaluating the balance. Some…

Purpose: This study examines the response patterns of 278 Spanish-English dual language learners (DLLs) on a standardized test of receptive English vocabulary. Method: Investigators analyzed responses to 131 items on the Peabody Picture Vocabulary Test--Fourth Edition (Dunn & Dunn, 2007) focusing on differential accuracy on items influenced by…

The purpose of this study is to determine whether English test items of Undergraduate Placement Exam (UPE) in 2016 contain differential item functioning (DIF) and differential bundle functioning (DBF) in terms of gender and school type and examine the possible sources of bias of DIF items. Mantel Haenszel (MH), Simultaneous Item Bias Test…

This paper presents a framework for choosing between 4-and 6-point response options for use with online surveys. Using data that have both 4- and 6-point Likert-type items, we compare correlations, fit of factor analytic models, and several different reliability estimates as a way of identifying if there is empirical support for choosing a…

We illustrate the application of mixture IRT [item response theory] models to evaluate the possibility of respondent confusion due to the negative wording of certain items on a social-emotional learning (SEL) assessment. Using actual student self-report ratings on four social-emotional learning scales collected from students in grades 3-12 from…

Subscores can be of diagnostic value for tests that cover multiple underlying traits. Some items require knowledge or ability that spans more than a single trait. It is thus natural for such items to be included on more than a single subscore. Subscores only have value if they are reliable enough to justify conclusions drawn from them and if they…

Cognitive Diagnostic Assessment (CDA) is an alternative assessment which can give a clear picture of pupils' learning process and cognitive structures to education stakeholders so that appropriate instructional strategies can be designed to tailored pupils' needs. Coincide with this function, the Ordered Multiple-Choice (OMC) items were…

In line with the widespread proliferation of digital technology in everyday life, many countries are now beginning to use computer-based exams (CBEs) in their post-primary education systems. To ensure that these CBEs are delivered in a manner that preserves their fairness, validity, utility and credibility, several factors pertaining to their…

While research and practice centred around students and academics working together to co-create in the higher level sector has increased, co-creation in assessment remains relatively rare in a higher education context. It is acknowledged in the literature that deeper comprehension of content can be realised when students author their own questions…

The calibration of test materials generally involves the interaction between empirical analysis and expert judgement. This paper explores the extent to which scale familiarity might affect expert judgement as a component of test validation in the calibration process. It forms part of a larger study that investigates the alignment of the…

Vocabulary size measures serve important functions, not only with respect to placing learners at appropriate levels on language courses but also with a view to examining the progress of learners. One of the widely reported formats suitable for these purposes is the Yes/No vocabulary test. The primary aim of this study was to introduce and provide…

Artificial intelligence (AI) technologies have the potential to reduce the workload for the second language (L2) teachers and test developers. We propose two AI distractor-generating methods for creating Chinese vocabulary items: semantic similarity and visual similarity. Semantic similarity refers to antonyms and synonyms, while visual similarity…

Technology-enhanced items (TEIs) are innovative, computer-delivered test items that allow test takers to better interact with the test environment compared to traditional multiple-choice items (MCIs). The interactive nature of TEIs offer improved construct coverage compared with MCIs but little research exists regarding students' performance on…

Language researchers and teachers have long been interested in the timing of learning, and the distributed practice effect, whereby greater inter-session intervals result in longer retention, is well-known (Kim & Webb, 2022). Many L2 studies have focused on the intentional learning of lexis (Edmonds, Gerbier, Palasis, & Whyte, 2021),…

The Head Start Family and Child Experiences Survey (FACES) and the American Indian and Alaska Native Head Start Family and Child Experiences Survey (AIAN FACES) are separate studies done successively over time. One goal for these studies is to provide a national picture of children's readiness for school. In this research brief, the authors use…