The emergence of digital-first assessments is prompting reconsideration of, and innovation in, aspects of psychometrics, test validation, and test use. Using the Duolingo English Test (DET) as an example, this three-paper series seeks to address issues concerning the estimation of classification consistency and the reporting of results for such…

Automated assessment using Natural Language Processing (NLP) has the potential to make English speaking assessments more reliable, authentic, and accessible. Yet without careful examination, NLP may exacerbate social prejudices based on gender or native language (L1). Current NLP-based assessments are prone to such biases, yet research and…

Pragmatic competence has been increasingly recognized as a component of academic language proficiency assessment (Youn, 2018). However, the practicality of administering L2 pragmatic competence assessments involving interlocutors remains problematic (e.g., high labor cost) (Ikeda, 2017). The pandemic makes it even more difficult to administer…

The digital transformation in educational assessment has led to the proliferation of large-scale data, offering unprecedented opportunities to enhance language learning, and testing through machine learning (ML) techniques. Drawing on the extensive data generated by online English language assessments, this dissertation investigates the efficacy…

Millions of public-school students in the United States are identified as English language learners (ELLs), whose academic success is tied to their second language (L2) English education. Previous research in adult populations indicates that L2 proficiency is related to the contextual variable of the prevalence of one's first language (L1) among…

With the wide introduction of English-medium instruction (EMI) to higher education institutions throughout East Asian countries, many East Asian universities are using English proficiency tests that were not originally designed for this context to make admissions and placement decisions. To support the use of these tests in this new EMI context,…

Students are expected to complete high-stakes standardized tests on computers, but many are disadvantaged due to their lack of technical skills and experience. While students complete a required state practice test, the brief overview of the technology used during testing is often not enough to ensure that students can successfully navigate the…

With over one million emergent bilinguals enrolled in Texas public schools, a call to action and a review of high-stakes online assessment policy for this student population is essential to achieve equity in education. The purpose of this phenomenological study aimed to explore teachers' perceptions of high-stakes online TELPAS testing and digital…

While recent research has shown that multimodal feedback and use of formulaic sequences (FS) are effective in improving student performance on writing tasks, there is a dearth of studies on the impact of these aspects on computer-based academic speaking assessments. This dissertation seeks to fill this gap by examining the impact of both…

Latinx dual language learners (DLLs) are an enormous and growing student body that faces educational adversities and inequities. One of the most salient findings to emerge from decades of educational research is that DLLs who close the gap with their monolingual peers and have the best long-term academic performance are those who develop strong…

The purpose of this sequential mixed method study was to determine if there were differences between central office administrators', school administrators', and bilingual and English as a second language (ESL) teachers' overall and levels of action, self-regulation, and means efficacies, when controlling for years of experience with bilingual and…

Speaking performance tests typically employ raters to produce scores; accordingly, variability in raters' scoring decisions has important consequences for test reliability and validity. One such source of variability is the rater's level of expertise in scoring. Therefore, it is important to understand how raters' performance is influenced by…

A challenge for todays instructional leaders is identifying the most effective method of educating English Language Learners (ELLs) who participate in the English as a Second Language (ESL) program. The primary purpose of the study was to compare the effectiveness of three programs used for ESL students in twelve elementary schools in a Urban…

By centralizing the issue of test fairness in language proficiency assessments, this study responds to a call by researchers for developing greater social responsibility in the language testing agenda. As inquiries into language attitude and psychology indicate, there is an underlying uncertainty pertaining to the validity of test use and score…

This dissertation provides an automated scoring method of speech fluency for second language learners of English (L2 learners) based that uses speech recognition technology. Non-standard pronunciation, frequent disfluencies, faulty grammar, and inappropriate lexical choices are crucial characteristics of L2 learners' speech. Due to the ease of…