A critical component of cognitive diagnostic models (CDMs) is a Q-matrix that stipulates associations between items of a test and their required attributes. The present study aims to develop and empirically validate a Q-matrix for the listening comprehension section of the International English Language Testing System (IELTS). To this end, a…

Cognitive diagnostic models (CDMs) have received much interest within the field of language testing over the last decade due to their great potential to provide diagnostic feedback to all stakeholders and ultimately improve language teaching and learning. A large number of studies have demonstrated the application of CDMs on advanced large-scale…

In the realm of language proficiency assessments, the domain description inference and the extrapolation inference are key components of a validity argument. Biber et al.'s description of the lexicogrammatical features of the spoken and written registers in the T2K-SWAL corpus has served as support for the TOEFL iBT test's domain description and…

Ensuring that test-score use brings about socially positive consequences for test-takers is an important aspect of test validation. While many studies use an inductive approach to evaluate test consequences, few studies have implemented Appraisal analysis. To that end, this case study investigated the test consequences of an English reading…

Background: Validity is a notable concept in language testing which has concerned many researchers and scholars in the field of language testing due to its importance in decision making process. Tests' results always introduce consequences to test takers' lives which emphasizes the need to ensure their validity. Detecting and delineating the…

Background: This study aims to empirically answer the question of whether the role of sub-reading skills changes depending on the test format (e.g., multiple-choice vs. open-ended reading questions). The test format effect also addresses the issue of test validity--whether the reading test properly elicits construct-relevant reading skills or…

Investigating the processes underlying test performance is a major source of data supporting the explanation inference in the validity argument (Chappelle, 2021). One way of modeling the cognitive processes underlying test performance is by constructing a Q-matrix, which is essentially about summarizing the attributes explaining test-takers'…

Criterion referenced tests of second language speaking performance are administered in different institutions using different procedures. The present study reports raters' practices of second language speaking tests, in particular the correspondence between test-takers' grades when assessed individually and in groups. Data derived from…

Investigations of the validity of a number of high-stakes language assessments are conducted using an argument-based approach, which requires evidence for inferences that are critical to score interpretation (Chapelle, Enright, & Jamieson, 2008b; Kane, 2013). The current study investigates the extrapolation inference for a high-stakes test of…

Those who work in second- and foreign-language testing often find Koretz's concern for validity inferences under high-stakes (VIHS) conditions both welcome and familiar. While the focus of the article is more narrowly on the potential for two instructional responses to test-based accountability, "reallocation" and "coaching,"…

Performance assessments have become the norm for evaluating language learners' writing abilities in international examinations of English proficiency. Two aspects of these assessments are usually systematically varied: test takers respond to different prompts, and their responses are read by different raters. This raises the possibility of undue…

The Ontario Secondary School Literacy Test (OSSLT) is a cross-curricular literacy test issued to all secondary school students in the province of Ontario. The test consists of a reading and a writing component, both of which must be successfully completed for secondary school graduation in Ontario. This study elicited 16 first language and second…

Within the context of increasing numbers of second language (L2) learners in Canadian schools and expanding standards-driven testing frameworks, a passing score on the Ontario Secondary School Literacy Test (OSSLT) is a recently imposed secondary school graduation requirement in Ontario. There is evidence, however, that tests designed on the basis…

Examines one aspect of the many interfaces between second language (L2) learning and L2 testing. The aspect is the oral interaction--the dialogue--that occurs within small groups. Discusses from within a sociocultural theory of mind, that in a group, performance is jointly constructed and distributed across the participants. (Author/VWL)

Discusses the importance of conversational implicatures in cross-cultural communication and argues that the use of opened-ended questions to study such implicatures is inherently flawed. It is asserted that multiple-choice tests are better investigational devices, and an ongoing investigation of the cross-cultural interpretation of implicature…