In sociolinguistic contexts where standardized languages coexist with regional dialects, the study of differential item functioning is a valuable tool for examining certain linguistic uses or varieties as threats to score validity. From an ecological perspective, this paper describes three stages in the study of differential item functioning…

This quasi-experimental study examines the impact of a diagnosis-based pedagogical intervention on the development of micro English as a foreign language (EFL) reading abilities over time. Specifically, 105 first-year graduate students in a Chinese university completed three diagnostic reading assessments at the beginning, middle, and end of a…

The Korean College Scholastic Ability Test (CSAT) aims to assess Korean high school students' scholastic ability required for college readiness. As a high-stakes test, the examination serves as a pivotal hurdle for university admission and exerts a strong washback effect on the educational system in Korea. The present study set out to investigate…

The interaction between language proficiency and discipline-related background knowledge is an important feature of LSAP (Languages for Specific Academic Purposes) tests and is viewed as a necessity in these tests unlike general purpose language tests which try to diminish the potential effect of discipline-related knowledge. Therefore, variables…

Thinking skills play a critical role in determining language performance. Recent advancement in cognitive diagnostic modelling (CDM) provides a powerful tool for obtaining fine-grained information regarding these thinking skills during reading. Studies are scant, however, exploring the relations between thinking skills and language performance,…

This study investigates meaning-making processes in language and literacy assessments. Using a social semiotic perspective, it examines how adult second language learners with emerging literacy self-articulate their understanding of multimodal elements and components utilized in low-stakes assessments, and the strategies they use to make meaning…

Technology-enhanced items (TEIs) are innovative, computer-delivered test items that allow test takers to better interact with the test environment compared to traditional multiple-choice items (MCIs). The interactive nature of TEIs offer improved construct coverage compared with MCIs but little research exists regarding students' performance on…

Recently, many universities in Vietnam have adopted the TOEIC (Test of English for International Communication) Listening and Reading test as a language exit test for non-English major students. This study aimed at examining the perceived impact of the TOEIC Listening and Reading test on English teaching and the mechanism by which this test impact…

It is not unusual for tests in less-commonly taught languages (LCTLs) to be developed by an experienced item writer with no proficiency in the language being tested, in collaboration with a language informant who is a speaker of the target language, but lacks language assessment expertise. How this approach to item writing works in practice, and…

The study reported here investigates the validity of judgments made when aligning tests to the "Common European Framework of Reference" (CEFR). Listening tests operationalizing pre-defined difficulty-determining characteristics were to be aligned to CEFR levels. We employed a modified version of the item-descriptor-matching-method. Ten…

The present study examined the factor structures across features of 446 examinees' responses to a writing task that integrates reading and listening modalities as well as reading and listening comprehension items of the TOEFL iBT[R] (Internet-based test). Both human and automated scores obtained for the integrated essays were utilized. Based on a…

Integrated reading/writing tasks are becoming more common in large-scale language tests. Much of the research on these tasks has focused on writing through reading; assessing reading through writing is a less explored area. In this article we describe a reading-into-writing task that is intended to measure both reading comprehension and language…

This article examines how the use of automated scoring procedures for short-answer reading tasks can affect the constructs being assessed. In particular, it highlights ways in which the development of scoring algorithms intended to apply the criteria used by human raters can lead test developers to reexamine and even refine the constructs they…

Within the context of Taiwan, where growing numbers of universities seek teaching excellence and better English learning outcomes, passing a designated English test has been imposed on thousands of university students as one of their degree requirements. This educational policy has become feasible only after the debut of the indigenous General…

In the last 15 years or so, language testing practitioners have increasingly favored assessing vocabulary in context. The discrete-point vocabulary measure used in the old version of the Test of English as a Foreign Language (TOEFL) has long been criticized for encouraging test candidates to memorize wordlists out of context although test items…