Construct validity and building validity arguments are some of the main challenges facing the language assessment community. The notion of construct validity and validity arguments arose from research in psychological assessment and developed into the gold standard of validation/validity research in language assessment. At a theoretical level,…

The COVID-19 pandemic has changed the university admissions and proficiency testing landscape. One change has been the meteoric rise in use of the fully automated Duolingo English Test (DET) for university entrance purposes, offering test-takers a cheaper, shorter, accessible alternative. This rapid response study is the first to investigate the…

Argument-based validation requires test developers and researchers to specify what is entailed in test interpretation and use. Doing so has been shown to yield advantages (Chapelle, Enright, & Jamieson, 2010), but it also requires an analysis of how the concerns of language testers can be conceptualized in the terms used to construct a…

With the present study I investigated the sources of score variance and dependability in a local oral English proficiency test for potential international teaching assistants (ITAs) across four first language (L1) groups, and suggested alternative test designs. Using generalizability theory, I examined the relative importance of L1s (i.e., Indian,…

This study explores whether and to what extent the background information supplied by 10,155 immigrants who took an official language test in Norwegian affected their chances of passing one, two, or all three parts of the test. The background information included in the analysis was prior education, region (location of their home country),…

Measuring language dominance, broadly defined as the relative strength of each of a bilingual's two languages, remains a crucial methodological issue in bilingualism research. While various methods have been proposed, the Bilingual Language Profile (BLP) has been one of the most widely used tools for measuring language dominance. While previous…

Language assessment literacy (LAL) has recently gained substantial attention among language educators and other stakeholders. However, existing models focus almost exclusively on teachers, test developers, and administrators, and lack students' perspectives in their conceptualizations. To address this gap, with this exploratory study we aimed to…

The evaluation of sign language proficiency needs to be based on measures with well-established psychometric proprieties. To date, no valid and reliable test is available to assess Polish Sign Language ("Polski Jezyk Migowy," PJM) skills in deaf children. Hence, our aim with this study was to adapt the British Sign Language Receptive…

Originally designed to measure reading and passage comprehension in L1 readers, cloze tests continue to be used for L2 assessment purposes. However, there remain disputes about whether or not cloze items can measure beyond local comprehension information, as well as whether or not they are purely a test of reading alone, or if performance can be…

Sign languages present particular challenges to language assessors in relation to variation in signs, weakly defined citation forms, and a general lack of standard-setting work even in long-established measures of productive sign proficiency. The present article addresses and explores these issues via a mixed-methods study of a human-rated…

Researchers frequently evaluate rater judgments in performance assessments for evidence of differential rater functioning (DRF), which occurs when rater severity is systematically related to construct-irrelevant student characteristics after controlling for student achievement levels. However, researchers have observed that methods for detecting…

This article describes the implementation of a special accommodations policy for a suite of localised English language and numeracy tests, the Workplace Literacy and Numeracy (WPLN) Assessments. The WPLN are computer-delivered assessments, part of the WPLN training and assessment programme, which exists to provide access to workforce skills…

For institutions where English is the primary language of instruction, English assessments for admissions such as the Test of English as a Foreign Language (TOEFL) and International English Language Testing System (IELTS) give admissions decision-makers a sense of a student's skills in academic English. Despite this explicit purpose, these exams…

The overarching aim of the study is to explore the extent to which test takers' performances on monologic speaking tasks provide information about their interactional competence. This is an important concern from a test use perspective, as stakeholders tend to consider test scores as providing comprehensive information about all aspects of L2…

In this study, we present the development of individualized feedback for a large-scale listening assessment by combining standard setting and cognitive diagnostic assessment (CDA) approaches. We used the performance data from 3,358 students' item-level responses to a field test of a national EFL test primarily intended for tertiary-level EFL…