This study evaluated the efficacy of two proposed methods in an operational standard-setting study conducted for a high-stakes language proficiency test of the U.S. government. The goal was to seek low-cost modifications to the existing Yes/No Angoff method to increase the validity and reliability of the recommended cut scores using a convergent…

This study presents a comparative analysis of classical reliability measures, including Cronbach's alpha, test-retest, and parallel forms reliability, alongside modern psychometric methods such as the Rasch model and Mokken scaling, to evaluate the reliability of C-tests in language proficiency assessment. Utilizing data from 150 participants…

The linguistic characteristics of text productions depend on various factors, including individual language proficiency as well as the tasks used to elicit the production. To date, little attention has been paid to whether some writing tasks are more suitable than others to represent and differentiate students' proficiency levels. This issue is…

Chulalongkorn University Language Institute (CULI) test was developed as a local standardised test of English for professional and international communication. To ensure that the CULI test fulfils its intended purposes, this study employed Kane's argument-based validation and Rasch measurement approaches to construct the validity argument for the…

The goal of this paper is to tease apart two approaches to the source of children's consistent scope assignment in negative sentences containing logical connectives: the Semantic Subset Principle and the Semantic Subset Maxim. Previous developmental work has observed that four- to six-year-old children across languages have difficulty with…

This study aims to investigate ChatGPT's ability to comprehend input from nonnative speakers, specifically those learning English as a second language, with Japanese speakers serving as the model population. The experiment examines how ChatGPT evaluates the difficulty levels of the Test of English for International Communication (TOEIC), which is…

This study addressed a gap in existing research on Multiple-Choice (MC) cloze tests by focusing on the learners' perspective, specifically examining the difficulties faced by vocational high school students (VHSs). A nationwide sample of 293 VHSs participated, providing both quantitative and qualitative data through a self-developed questionnaire.…

The current paper intends to exploit the Many Facet Rasch Model to investigate and compare the impact of situations (items) and raters on test takers' performance on the Written Discourse Completion Test (WDCT) and Discourse Self-Assessment Tests (DSAT). In this study, the participants were 110 English as a Foreign Language (EFL) students at…

The purpose of this study was to compare the functioning of five restrictive CDMs, including DINA, DINO, A-CDM, LLM, and RRUM, against the G-DINA model to identify the best-fitting CDM which can better explain the interaction underlying the attributes of the reading comprehension section of an Iranian high-stakes language proficiency test. To this…

Challenges in reading Chinese as a foreign language involve the large proportion of two-character compound words which have complex intra-word morphological structures and scriptal distance between learner's native language (L1) and Chinese as a second or foreign language. This study extended a previous investigation on the processing of Chinese…

Language proficiency policies act as critical gatekeepers for multilingual international students (MIS) who intend to pursue an education at English-medium universities. The current study revisits the question of the predictive validity of a standardized English proficiency exam--International English Language Testing System (IELTS)--in comparison…

This study examined the relationship of linguistic complexity, captured using a set of lexical richness, syntactic complexity, and discoursal complexity indices, to second language (L2) learners' perception of text difficulty, captured using L2 raters' comparative judgment on text comprehensibility and reading speed. Testing materials were 180…

Task design has been viewed to be essential in the context of language assessment. This study investigated whether increasing task complexity affects learners' writing performance. It employs three writing tasks with different levels of complexity based on Robinson's Componential Framework. A cohort of 278 participants was selected using a simple…

Scheduling of practice to create a spacing effect has been observed to lead to superior retention of L2 vocabulary and grammar. Two of the various methods of scheduling L2 task practice discussed widely in recent times are interleaving and blocking. In a desirable difficulty framework, both of these methods have been recommended for task…

This study employed a multi-methods design to investigate the impact of preparation on Chinese test-takers' perceptions of the integrated TOEFL iBT speaking and writing design. Combining results from over 1700 surveys and 10 interviews, it was found that these Chinese test-takers, who are the most vulnerable group in the multimillion testing…