This study evaluated the efficacy of two proposed methods in an operational standard-setting study conducted for a high-stakes language proficiency test of the U.S. government. The goal was to seek low-cost modifications to the existing Yes/No Angoff method to increase the validity and reliability of the recommended cut scores using a convergent…

This study introduces a novel differential item functioning (DIF) method based on propensity score matching that tackles two challenges in analyzing performance assessment data, that is, continuous task scores and lack of a reliable internal variable as a proxy for ability or aptitude. The proposed DIF method consists of two main stages. First,…

In this study we evaluated the stability of item parameters and student scores, using the pre-equated (pre-pandemic) parameters from Spring 2019 and post-equated (post-pandemic) parameters from Spring 2021 in two calibration and equating designs related to item parameter treatment: re-estimating all anchor parameters (Design 1) and holding the…

The present research paper introduces a translation evaluation method called Calibrated Parsing Items Evaluation (CPIE hereafter). This evaluation method maximizes translators' performance through identifying the parsing items with an optimal p-docimology and d-index (item discrimination). This method checks all the possible parses (annotations)…

Language testing and assessment have grown in popularity and gained significance in the last few decades, and there is a rising need for assessment literate stakeholders in the field of language education. As teachers play a major role in assessing students, there is a need to make sure they have the right level of assessment knowledge and skills…

Cognitive diagnostic models (CDMs) have received much interest within the field of language testing over the last decade due to their great potential to provide diagnostic feedback to all stakeholders and ultimately improve language teaching and learning. A large number of studies have demonstrated the application of CDMs on advanced large-scale…

The main objective of the article is to compare several methods of evaluating multiple-choice options through classical item analysis. The methods subjected to examination include the tabulation of choice distribution, the interpretation of trace lines, the point-biserial correlation, the categorical analysis of trace lines, and the investigation…

This study investigated the elicited imitation task (EIT) as a tool for measuring linguistic proficiency in a second/foreign (L2) language, focusing on French. Nonnative French speakers (n = 94) and native French speakers (n = 6) completed an EIT that included 50 sentences varying in length and complexity. Three raters evaluated productions on…

Dynamic Assessment (DA) derives from Vygotsky's (1987) insight that the use of mediation, attuned to learner needs, enables learners to perform beyond their current level of functioning, thereby providing insights into emerging capabilities. The instructional quality of mediation means that this process simultaneously creates possibilities for…

Dynamic assessment as a complementary approach to traditional static assessment emphasizes the learning process and accounts for the amount and nature of examiner investment. The present qualitative study analyzed interactions for 270 reading test items which were recorded and tape scripted. The reading ability of 9 EFL participants at three…

Assessment is central in education and the teaching-learning process. This study attempts to explore the perspectives and views about quality assessment among teachers of English as a Foreign Language (EFL), and to find ways of promoting quality assessment. Quantitative methodology was used to collect data. To answer the study questions, a…

This study investigated the effect of the interaction between test takers' background knowledge and language proficiency on their performance on the "TOEFL iBT"® reading section. Test takers with the target content background knowledge (the focal groups) and those without (the reference groups) were identified for each of the 5 selected…

With recent statistical advances in cognitive diagnostic assessment (CDA), the CDA approach has been increasingly applied to non-diagnostic tests partly to meet accountability demands for student achievement. The study aimed to evaluate critically the validity of the CDA application to an existing non-diagnostic L2 reading comprehension test and…

This research project addresses the validity of assessments used to measure the performance of English language learners (ELLs), such as those mandated by the No Child Left Behind Act of 2001 (NCLB, 2002). The goals of the research are to help educators understand and improve ELL performance by investigating the validity of their current…

Issues in measuring language proficiency of monolingual and bilingual persons are discussed. Psychometric procedures in developing both English- and Spanish-language proficiency instruments are illustrated through development of the New York City Language Assessment Battery, an assessment of proficiency in both languages used to screen eligibility…