This study employed a multi-methods design to investigate the impact of preparation on Chinese test-takers' perceptions of the integrated TOEFL iBT speaking and writing design. Combining results from over 1700 surveys and 10 interviews, it was found that these Chinese test-takers, who are the most vulnerable group in the multimillion testing…

The National Assessment of Educational Progress (NAEP), often referred to as The Nation's Report Card, offers a window into the state of U.S. K-12 education system. Since 2017, NAEP has transitioned to digital assessments, opening new research opportunities that were previously impossible. Process data tracks students' interactions with the…

Vocabulary assessment is an important part of measuring language proficiency in both monolingual and bilingual children. The LITMUS Cross-Linguistic Lexical Tasks (CLT) provides a framework for assessing the vocabulary of monolingual and bilingual children using a standardized procedure and comparable stimuli across languages. All language…

A conceptualization of multiple-choice exams in terms of signal detection theory (SDT) leads to simple measures of item difficulty and item discrimination that are closely related to, but also distinct from, those used in classical item analysis (CIA). The theory defines a "true split," depending on whether or not examinees know an item,…

The purpose of this study was to determine if there were significant statistical differences between scores on constructed response and computer-scorable questions on an accelerated middle school math placement test in a large urban school district in Ohio, and to ensure that all students have an opportunity to take the test. Five questions on a…

In the wake of the COVID-19 boom in remote administration of language tests, it appears likely that remote administration will be a permanent fixture in the language testing landscape. Accordingly, language test providers, stakeholders, and researchers must grapple with the implications of remote proctoring on valid, fair, and just uses of tests.…

Disengaged responding poses a severe threat to the validity of educational large-scale assessments, because item responses from unmotivated test-takers do not reflect their actual ability. Existing identification approaches rely primarily on item response times, which bears the risk of misclassifying fast engaged or slow disengaged responses.…

In this Brief Report, we describe an evaluation of and revisions to a rubric adapted from the Jacobs et al.'s (1981) ESL COMPOSITION PROFILE, with four rubric categories and 20-point rating scales, in the context of an intensive English program writing placement test. Analysis of 4 years of rating data (2016-2021, including 434 essays) using…

The study investigated hurdles to the quality of student learning assessment by examining issues related to assessment procedures and practices, learners and learning, learning resources and test constructs, and test admin and feedback. Quantitative and qualitative data were collected from two Ethiopian universities using two types of…

Test input has often been taken as a given in test design practice. Nearly all guides for test designers provide extensive coverage of how to design test items but pay little attention to test input. This paper presents the case that test input plays a crucial role in designing tests of soft skills that have rarely been assessed in existing tests.…

During the COVID pandemic, universities around the globe had to move not only their content delivery online, but also their assessments. Due to COVID causing significant upheaval in Higher Education (HE), this enforced experiment also afforded an opportunity to reflect on traditional, invigilated, closed book exams (ICBE) resulting in research and…

We compared coefficient alpha with five alternatives (omega total, omega RT, omega h, GLB, and coefficient H) in two simulation studies. Results showed for unidimensional scales, (a) all indices except omega h performed similarly well for most conditions; (b) alpha is still good; (c) GLB and coefficient H overestimated reliability with small…

This article was stimulated by a review of instruments assessing creative and innovative social environments seventeen years ago. This stands alone as the only published, comprehensive, comparative review of multiple instruments aimed at this conceptual space. Although this review provided an important contribution to the literature, there are a…

This study aims to develop a valid and reliable achievement test to measure the achievement of eighth-grade students in the "Seasons and Climate" unit of the science course. The research was carried out using a survey model. The study group consists of 298 eighth-grade students studying in various cities in Turkey in the 2020-2021…

My dissertation examines three current challenges of international large-scale assessments (ILSAs) associated with the transition from linear testing to an adaptive testing design. ILSAs are important for making comparisons among populations and informing countries about the quality of their educational systems. ILSA's results inform policymakers…