This study explores the effectiveness of using ChatGPT, an Artificial Intelligence (AI) language model, as an Automated Essay Scoring (AES) tool for grading English as a Foreign Language (EFL) learners' essays. The corpus consists of 50 essays representing various types including analysis, compare and contrast, descriptive, narrative, and opinion…

This empirical research study investigates the relationship between the utilization of Universal Design (UD) elements and math performance among eighth graders. We analyzed 2017 National Assessment of Educational Progress process data using Poisson Generalized Linear Mixed-Effects Models to examine how the frequency of UD element usage varies…

This technical report provides detailed information regarding the technical, statistical, and measurement attributes of the New York State Testing Program (NYSTP) for the Grades 3-8 English Language Arts (ELA) and Mathematics 2018 Operational Tests. This report includes information about test content and test development, item (i.e., individual…

As online education becomes a more popular and permanent option for obtaining an education after high school, it also raises questions as to the academic rigor of such classes and the academic integrity of the students taking the classes. The purpose of the current study is to explore the integrity issue and to investigate student performance on…

All Florida schools teach the Florida Standards in English Language Arts (ELA) and Mathematics. Student performance on the Florida Standards Assessments (FSA) provides important information to parents/guardians, teachers, policy makers, and the general public regarding how well students are learning the Florida Standards. This document has been…

In typical differential item functioning (DIF) assessments, an item's DIF status is not influenced by its status in previous test administrations. An item that has shown DIF at multiple administrations may be treated the same way as an item that has shown DIF in only the most recent administration. Therefore, much useful information about the…

Under the generalizability-theory (G-theory) framework, the estimation precision of variance components (VCs) is of significant importance in that they serve as the foundation of estimating reliability. Zhang and Lin advanced the discussion of nonadditivity in data from a theoretical perspective and showed the adverse effects of nonadditivity on…

Virtual performance assessments (VPAs), such as game- and simulation-based assessments, provide promising ways for assessing complex and integrated skills. However, the high cost, long development cycle, and complex scoring process significantly hinder the adoption of VPAs, particularly in large-scale assessments with tight deadlines and limited…

This paper explores how to apply the principles of accommodations and universal design (UD) in research methods involved in quantitative research (e.g. questionnaires). In particular, we focus on how to make research more accessible for individuals with Learning Disabilities (LD), while also providing suggestions for potential participants of…

This study examined the potential use of keystroke dynamics to create keyprints (typing fingerprints) to authenticate individuals in online assessment situations. The implications of this study are best understood in terms of the keystroke behavioral biometric. While previous studies considered the degree to which keystroke typing patterns are…

Computerized adaptive testing (CAT) and multistage testing (MST) have become two of the most popular modes in large-scale computer-based sequential testing. Though most designs of CAT and MST exhibit strength and weakness in recent large-scale implementations, there is no simple answer to the question of which design is better because different…

Daniel Koretz is the Henry Lee Shattuck Professor of Education at the Harvard Graduate School of Education. His research focuses on educational assessment and policy, particularly the effects of high-stakes testing on educational practice and the validity of score gains. He is the author of "Measuring Up: What Educational Testing Really Tells…

In recent years, there has been increasing focus on the issue of multiple hypotheses testing in education evaluation studies. In these studies, researchers are typically interested in testing the effectiveness of an intervention on multiple outcomes, for multiple subgroups, at multiple points in time or across multiple treatment groups. When…

The study investigated the amount of extended time (ET) used by postsecondary students with learning disabilities, attention-deficit/hyperactivity disorder (ADHD), and psychological disorders granted a 50% ET accommodation (i.e., time and one-half). Extended time used was evaluated by disability type, including comorbid presentations, and year in…

In learning, errors are ubiquitous and inevitable. As these errors may signal otherwise latent cognitive processes, tutors--and students alike--can greatly benefit from the information they provide. In this paper, we introduce and evaluate the Systematic Error Tracing (SET) model that identifies the possible causes of systematically observed…