This study evaluated the efficacy of two proposed methods in an operational standard-setting study conducted for a high-stakes language proficiency test of the U.S. government. The goal was to seek low-cost modifications to the existing Yes/No Angoff method to increase the validity and reliability of the recommended cut scores using a convergent…

This study presents a comparative analysis of classical reliability measures, including Cronbach's alpha, test-retest, and parallel forms reliability, alongside modern psychometric methods such as the Rasch model and Mokken scaling, to evaluate the reliability of C-tests in language proficiency assessment. Utilizing data from 150 participants…

The linguistic characteristics of text productions depend on various factors, including individual language proficiency as well as the tasks used to elicit the production. To date, little attention has been paid to whether some writing tasks are more suitable than others to represent and differentiate students' proficiency levels. This issue is…

Language complexity reveals the ability to use a wide and varied range of sophisticated structures and vocabulary. Although different languages compose complexity differently, complexity measures such as the T-unit have typically been based on clause subordination, which may underrepresent complexity and threaten the validity of studies. This…

This study aims to investigate ChatGPT's ability to comprehend input from nonnative speakers, specifically those learning English as a second language, with Japanese speakers serving as the model population. The experiment examines how ChatGPT evaluates the difficulty levels of the Test of English for International Communication (TOEIC), which is…

The purpose of this study was to compare the functioning of five restrictive CDMs, including DINA, DINO, A-CDM, LLM, and RRUM, against the G-DINA model to identify the best-fitting CDM which can better explain the interaction underlying the attributes of the reading comprehension section of an Iranian high-stakes language proficiency test. To this…

Challenges in reading Chinese as a foreign language involve the large proportion of two-character compound words which have complex intra-word morphological structures and scriptal distance between learner's native language (L1) and Chinese as a second or foreign language. This study extended a previous investigation on the processing of Chinese…

Language proficiency policies act as critical gatekeepers for multilingual international students (MIS) who intend to pursue an education at English-medium universities. The current study revisits the question of the predictive validity of a standardized English proficiency exam--International English Language Testing System (IELTS)--in comparison…

Accomplishing a communication task in the real world requires the ability not only to do the task per se but also to manage aspects of the context in which it occurs. For this reason, simulations of target language use contexts have been incorporated into the design of communicative language tests as a way of enhancing the authenticity of…

In order to assess the accuracy and validity of proficiency diagnostic tests in Second Language (L2), specifically regarding the linguistic (orthographic, semantic, syntactic, lexical) and cognitive (verbal reasoning, lexical decision) components for the immigrant population in Portugal, a study of cut-off points of 6 tests was conducted. This…

Task design has been viewed to be essential in the context of language assessment. This study investigated whether increasing task complexity affects learners' writing performance. It employs three writing tasks with different levels of complexity based on Robinson's Componential Framework. A cohort of 278 participants was selected using a simple…

A C-Test is a gap-filling test for measuring language competence in the first and second language. C-Tests are usually analyzed with polytomous Rasch models by considering each passage as a super-item or testlet. This strategy helps overcome the local dependence inherent in C-Test gaps. However, there is little research on the best polytomous…

In response to the calls for making key stakeholders' perspectives relevant in the test validation process, the study discussed in this report sought test-taker feedback as part of collecting validity evidence and supporting the ongoing field testing efforts of the new "TOEFL ITP"® Speaking section. Specifically, I aimed to investigate…

The aim of this experiment was to investigate how glossing influences second language (L2) reading comprehension in relation to text difficulty and the two local and global meaning representations. Fifty-eight undergraduate students were asked to read three easy, moderate, and difficult texts and, following each passage, answer twenty…

Elicited Imitation Tests (EITs) are commonly used in second language acquisition (SLA)/bilingualism research contexts to assess the general oral proficiency of study participants. While previous studies have provided valuable EIT construct-related validity evidence, some key gaps remain. This study uses an integrative data analysis to further…