Despite the growing interest in AI for language assessment, there remains a significant research gap regarding its usefulness for assessing less proficient language skills, particularly those of learners of English as a second or foreign language (S/FL). AI models often prioritize proficient writing, neglecting the intricacies of learner language.…

The construct of "learning potential" has been proposed to capture differences between learner independent performance and performance during Dynamic Assessment (DA). This paper introduces a new LPS formula implemented in a DA study involving Pakistani learners of L2 Chinese. Learners were randomly assigned to a control or experimental…

Drawing on the framework of invariant measurement from Rasch measurement theory, the purpose of this study is to psychometrically evaluate the 20 language and teaching skill domains of the International Teaching Assistant (ITA) Test using the many-facet Rasch model and to empirically explore performance differences between females and males in…

The Common European Framework of Reference for Languages Companion Volume (CEFR CV) emphasizes macro-functions of language (i.e., reception, production, interaction, and mediation). However, there seems to be little consensus on whether the macro-functions are commensurable with CEFR-based proficiency tests. This paper focuses on the Examination…

The factors that influence rater scoring have been a subject of great interest to researchers in second language assessment. However, the research on the impact of test-takers' speech profiles (e.g., a jagged or a flat profile reflecting analytic subscores) on raters' scoring behaviors remains to be seen. To investigate the role of speech profiles…

L2 test developers often use scripted spoken texts in their L2 listening tests, because it is efficient and practical to create scripted spoken texts that meet predetermined test specifications. But because scripted spoken texts differ in a number of fundamental ways from unscripted spoken language, there are potential threats to validity when…

Recent research conducted as part of the English Profile identified grammatical criterial features that are characteristic of each Common European Framework of Reference (CEFR) proficiency level. The extent to which these criterial features attest in English learners of various first language backgrounds call for an empirical examination. In this…

This research conducted diagnostic assessment of problems in first-year undergraduates' English academic papers and tracked potential sources of the problems to the writing process and strategy use. Data collected include 339 term papers and interviews with 17 students. The samples were manually error tagged and marked against a detailed…

This interview with James E. Purpura via Zoom comes in uncertain times, with CVOID-19 turning into a global pandemic. The author delves into different issues related to James E. Purpura's early career and his Language Assessment Quarterly (LAQ) editorial experience. Then moves on and talk about his research and teaching. The final part will…

Adopting Linacre's guidelines for evaluating rating scale effectiveness, we examined whether and how a six-point rating scale functioned differently across raters, speech acts, and second language (L2) proficiency levels. We developed a 12-item Computerized Oral Discourse Completion Task (CODCT) for assessing the production of requests, refusals,…

Grammar assessment has long been a pivotal and regular component of almost all high-stakes English tests in China. In writing grammar items, however, test developers tend to depend largely on their individual knowledge of grammar, or even intuition, without much reference to the typicality or de facto uses of the language. To address this problem…

Global efforts for standards-based linguistic assessment increasingly hold that examinees should be tested in the language with which they are most familiar. Yet, language use still occurs differently from its characterization in exams, even as exams are increasingly developed in historically minoritized languages. Drawing from two years of…

In vocabulary research there has been a shift from focusing on single words to considering multiword sequences, such as collocations. Despite the general consensus among language researchers that collocation is essential to effective language use in real-world communication, particularly oral communication, language-testing researchers have made…

Examining spoken features across proficiency levels allows researchers to explore the nature of speaking proficiency as it develops. This line of research has thus far primarily focused on adult second language (L2) learners. Using cross-sectional data based on a large-scale language assessment intended for young L2 learners, in this study, we…

This study used think-aloud protocols to compare essay-rating processes across holistic and analytic rating scales in the context of China's College English Test Band 6 (CET-6). A group of 9 experienced CET-6 raters scored the same batch of 10 CET-6 essays produced in an operational CET-6 administration twice, using both the CET-6 holistic…