This article compares two methods for detecting local item dependence (LID): residual correlation examination and Rasch testlet modeling (RTM), in a commonly used 3:6 matching format and an extended matching test (EMT) format. The two formats are hypothesized to facilitate different levels of item dependency due to differences in the number of…

This quasi-experimental study examines the impact of a diagnosis-based pedagogical intervention on the development of micro English as a foreign language (EFL) reading abilities over time. Specifically, 105 first-year graduate students in a Chinese university completed three diagnostic reading assessments at the beginning, middle, and end of a…

The construct of "learning potential" has been proposed to capture differences between learner independent performance and performance during Dynamic Assessment (DA). This paper introduces a new LPS formula implemented in a DA study involving Pakistani learners of L2 Chinese. Learners were randomly assigned to a control or experimental…

This mixed methods study compared IELTS examiners' scores when assessing spoken performances under live and two 'non-live' testing conditions using audio and video recordings. Six IELTS examiners assessed 36 test-takers' performances under the live, audio, and video rating conditions. Scores in the three rating modes were calibrated using the…

Thinking skills play a critical role in determining language performance. Recent advancement in cognitive diagnostic modelling (CDM) provides a powerful tool for obtaining fine-grained information regarding these thinking skills during reading. Studies are scant, however, exploring the relations between thinking skills and language performance,…

The TOEFL iBT test was introduced in 2005 to better reflect the language demands of real-life academic tasks than did previous versions of the test. The task-based design of the test was intended to support the interpretation of its scores as a trustworthy measure of international students' ability to use English in an academic environment. Until…

The present study examined the performance of the bi-factor multidimensional item response theory (MIRT) model and higher-order (HO) cognitive diagnostic models (CDM) in providing diagnostic information and general ability estimation simultaneously in a listening test. The data used were 1,611 examinees' item-level responses to an in-house EFL…

L2 test developers often use scripted spoken texts in their L2 listening tests, because it is efficient and practical to create scripted spoken texts that meet predetermined test specifications. But because scripted spoken texts differ in a number of fundamental ways from unscripted spoken language, there are potential threats to validity when…

This study examines the relationships among various major factors that may affect the difficulty level of language tests in an attempt to enhance the robustness of item difficulty estimation, which constitutes a crucial factor ensuring the equivalency of high-stakes tests. The observed difficulties of the reading and listening sections of two EFL…

Technology-enhanced items (TEIs) are innovative, computer-delivered test items that allow test takers to better interact with the test environment compared to traditional multiple-choice items (MCIs). The interactive nature of TEIs offer improved construct coverage compared with MCIs but little research exists regarding students' performance on…

Effective listening comprehension skills are an important prerequisite for the academic success of primary school students. However, the assessment of listening skills in the instructional language appears to have received only scant attention in the literature. Therefore, the goal of the present study was twofold. Firstly, a comprehensive…

The purpose of diagnostic language assessment is to identify learner strengths and weaknesses so that subsequent learning activity can be planned according to learner needs, a purpose which aligns with pedagogical recommendations to individualize instruction. Indeed, the extent to which diagnostic assessment can inform the selection of…

Suprasegmental features have received growing attention in the field of oral assessment. In this article we describe a set of computer algorithms that automatically scores the oral proficiency of non-native speakers using unconstrained English speech. The algorithms employ machine learning and 11 suprasegmental measures divided into four groups…

As a branch of artificial intelligence, automated speech recognition (ASR) technology is increasingly used to detect speech, process it to text, and derive the meaning of natural language for various learning and assessment purposes. ASR inaccuracy may pose serious threats to valid score interpretations and fair score use for all when it is…

The discussion takes up the common theme across the seven contributions of this special issue, namely the CEFR's suitability as a basis for setting university entrance standards. The special issue allows insights into this theme from a multitude of contexts, languages and perspectives, including data from stakeholders, students, tests and their…