Recently, psychometricians and researchers have voiced their concern over the exploration of language test items in light of Messick's validation framework. Validity has been central to test development and use; however, it has not received due attention in language tests having grave consequences for test takers. The present study sought to…

The way in which validity has been conceptualized has changed throughout the years. The focus in validation studies shifted from evaluating distinct components of validity to developing a comprehensive argument for the use and interpretations of test scores. The argument-based approach to validity incorporates the distinct types of the…

Cognitive diagnostic assessment is known for its capacity to provide detailed feedback to test-takers, which can inform instruction and guide learning. Besides this core strength of the assessment approach, the measurement procedure also works as a construct validation method in itself. Previous studies do not call attention to this aspect of…

The Pearson Test of English Academic (PTE-A), a widely used high-stakes language proficiency test for university admissions and migration purposes, underwent a notable change from a three-hour to a two-hour version in November 2021. The implementation of the new version has prompted inquiries into the washback effects on various stakeholders.…

This article presents the process of constructing and validating a test of metalinguistic awareness (MLA) for young school children (age 8-10). The test was developed between 2021 and 2023 as part of the MetaLearn research project, financed by The Research Council of Norway. The research team defines MLA as using metalinguistic knowledge at a…

With recent technological advances, researchers have begun to explore the potential use of spoken dialog systems (SDSs) for L2 oral communication assessment. While several studies support the feasibility of building these systems for various types of oral tasks, research on the construct validity of SDS-delivered tasks is still limited. Thus, this…

Purpose and background: The purpose of this paper is to critically review the traditional and contemporary validation frameworks--the content, criterion, and construct validations; the evidence-gathering; the socio-cognitive model; the test usefulness; and an argument-based approach--as well as empirical studies using an argument-based approach to…

Construct validity and building validity arguments are some of the main challenges facing the language assessment community. The notion of construct validity and validity arguments arose from research in psychological assessment and developed into the gold standard of validation/validity research in language assessment. At a theoretical level,…

The TOEFL Junior® tests are designed to evaluate young language students' English reading, listening, speaking, and writing skills in an English-medium secondary instructional context. This paper articulates a validity argument constructed to support the use and interpretation of the TOEFL Junior test scores for the purpose of placement, progress…

This study investigates the validity of syntactic priming as a measure of implicit language aptitude. Syntactic priming refers to the tendency to reproduce a linguistic structure due to a previous exposure to the structure. The validity of the construct was verified by collating evidence for divergent validity--whether it is dissociable from…

Language testing and assessment have grown in popularity and gained significance in the last few decades, and there is a rising need for assessment literate stakeholders in the field of language education. As teachers play a major role in assessing students, there is a need to make sure they have the right level of assessment knowledge and skills…

Recently a novel instrument -- the Core Academic Language Instrument (CALS-I) -- aimed at testing a constellation of school-relevant English language skills was developed and validated for use in the United States (Uccelli, Barr, Dobbs, Galloway, Meneses and Sánchez, 2015. Core Academic Language Skills: An expanded operational construct and a…

The Common European Framework of Reference for Languages Companion Volume (CEFR CV) emphasizes macro-functions of language (i.e., reception, production, interaction, and mediation). However, there seems to be little consensus on whether the macro-functions are commensurable with CEFR-based proficiency tests. This paper focuses on the Examination…

Existing methods used to validate self-report questionnaires in foreign language teaching effectiveness have relied on Classical Test Theory (CTT). However, the use of CTT approaches limits the reliability and validity of self-report instruments. The Rasch Model, which is based on the principles of objective measurement, addresses some of the…

This paper will deal, firstly, with the South African context, that cries out for attention to responsible language assessment. The renewed interest in language testing in South Africa is well illustrated in assessments of language ability for educational purposes generally, and more specifically in the assessment of academic literacy. Secondly,…