This study proposes a classification method for multiple text reading test formats in English language proficiency tests. A preliminary study involving 11 proficiency tests revealed two tests that fit the scope of the main study. Results show that multiple text reading test formats use complementary texts rather than conflicting texts. As for…

Assessments with a large amount of small, similar, or often repetitive tasks are being used in educational, neurocognitive, and psychological contexts. For example, respondents are asked to recognize numbers or letters from a large pool of those and the number of correct answers is a count variable. In 1960, George Rasch developed the Rasch…

Sometimes a treatment, such as receiving a high school diploma, is assigned to students if their scores on two inputs (e.g., math and English test scores) are above established cutoffs. This forms a multidimensional regression discontinuity design (RDD) to analyze the effect of the educational treatment where there are two running variables…

Various stakeholders, such as policymakers and educators, require diagnostic feedback and actionable test results. One particular context in which fine-grained test results are of utmost importance is English language proficiency (ELP) assessments as they are used for critical decisions for students. Diagnostic classification models (DCMs) afford…

The emergence of digital-first assessments is prompting reconsideration of, and innovation in, aspects of psychometrics, test validation, and test use. Using the Duolingo English Test (DET) as an example, this three-paper series seeks to address issues concerning the estimation of classification consistency and the reporting of results for such…

In 2006-2007, California revised its English language proficiency (ELP) assessment, the California English Language Development Test (CELDT), which resulted in more stringent criteria and higher cut scores for meeting proficiency. Using regression discontinuity designs and a difference-in-differences approach, this study examines the effects of…

The present quasi-experimental study investigated which component of processing instruction (PI) is responsible for its beneficial effects and whether EFL learners with different learning styles similarly benefit from PI components. In doing so, a sample of 67 first-year students took part in a study with a non-equivalent control group…

Exploratory cognitive diagnosis models (CDMs) estimate the Q matrix, which is a binary matrix that indicates the attributes needed for affirmative responses to each item. Estimation of Q is an important next step for improving classifications and broadening application of CDMs. Prior research primarily focused on an exploratory version of the…

In an upcoming coverage-comprehension study, we plan to assess learners' meaning-recall knowledge of words as they occur in the study's reading passage. As several meaning-recall test formats exist, the purpose of this small-scale study (N = 10) was to determine which of three formats was most similar to a criterion interview regarding mean score…

The current study compared the model fit indices, skill mastery probabilities, and classification accuracy of six Diagnostic Classification Models (DCMs): a general model (G-DINA) against five specific models (LLM, RRUM, ACDM, DINA, and DINO). To do so, the response data to the grammar and vocabulary sections of a General English Achievement Test,…

Time metaphorical expressions are common in all languages and in general as well as specialized contexts. This study explores the similarities and differences between English and Arabic time metaphorical expressions containing , and the difficulties that student-translators have in translating them; the translation strategies they use and the…

Some listeners exhibit higher sensitivity to subphonemic acoustic differences (i.e., higher speech gradiency). Here, we asked whether higher gradiency in a listener's first language (L1) facilitates foreign language learning and explored the possible sources of individual differences in L1 gradiency. To address these questions, we tested 164…

Research on the use of standardized test scores in higher education reveals significant variations in attitudes and perceptions of language proficiency tests among test score users. Most test score users have limited knowledge about test score interpretations in terms of what English as additional language (EAL) students typically know and can do…

In this study, we present the development of individualized feedback for a large-scale listening assessment by combining standard setting and cognitive diagnostic assessment (CDA) approaches. We used the performance data from 3,358 students' item-level responses to a field test of a national EFL test primarily intended for tertiary-level EFL…

The English Resultative Construction (ERC) is a satellite-framed structure with no identical equivalent in Spanish. In a series of studies, we analyzed and compared recognition (acceptability judgment task) and comprehension (sentence comprehension task) of three ERC subtypes with the English Depictive Construction (EDC) (which has a Spanish…