This study evaluated the forecasting accuracy of trend estimation methods applied to time-series data from computer adaptive tests (CATs). Data were collected roughly once a month over the course of a school year. We evaluated the forecasting accuracy of two regression-based growth estimation methods (ordinary least squares and Theil-Sen). The…

As more states move to universal computer-based assessments, an emergent issue concerns the effect that device type might have on student results. Although, several research studies have explored device effects, most of these studies focused on the differences between tablets and desktops/laptops. In the current study, we distinguish between…

In 2017, the mathematics assessments that are part of the National Assessment of Educational Progress (NAEP) program underwent a transformation shifting the administration from paper-and-pencil formats to digitally-based assessments (DBA). This shift introduced new interactive item types that bring rich process data and tremendous opportunities to…

Little evidence exists regarding students' actual use of testing time in naturalistic settings, particularly in the context of state accountability assessments. This study investigates students' test completion time and performance in the context of a statewide, English Language Arts and Mathematics computer-based assessment administered in grades…

Assessment of language skills for upper elementary and middle schoolers is important due to the strong link between language and reading comprehension. Yet, currently few practical, reliable, valid, and instructionally informative assessments of language exist. This study provides validation evidence for Monster, P.I., which is a gamified,…

This study evaluated the construct validity of six scoring traits of an automated writing evaluation (AWE) system called "MI Write." Persuasive essays (N = 100) written by students in grades 7 and 8 were randomized at the sentence-level using a script written with Python's NLTK module. Each persuasive essay was randomized 30 times (n =…

The New York State Education Department (NYSED) has a partnership with Questar Assessment Inc. (Questar) for the development of the 2020 Grades 3-8 Mathematics Tests. Teachers from across the State work with NYSED in a variety of activities to ensure the validity and reliability of the New York State Testing Program (NYSTP). The 2020 Grades 6-8…

This paper presents results of a score interpretation study for a computer adaptive mathematics assessment. The study purpose was to test the efficacy of item developers' alignment of items to Range Achievement-Level Descriptors (RALDs; Egan et al.) against the empirical achievement-level alignment of items to investigate the use of RALDs as the…

Scores from computer-based tests are increasingly used to inform a variety of school- and student-level decisions. An underlying assumption is that the associated scores represent effortful responding by each student with respect to the tasks presented. An innovative method for examining evidence for this assumption involves an examination of item…

The New York State Education Department (NYSED) has a partnership with Questar Assessment Inc. (Questar) for the development of the 2019 Grades 3-8 English Language Arts Tests. Teachers from across the State work with NYSED in a variety of activities to ensure the validity and reliability of the New York State Testing Program (NYSTP). The 2019…

The New York State Education Department (NYSED) has a partnership with Questar Assessment Inc. (Questar) for the online delivery of the 2022 Elementary-Level (Grade 5) and Intermediate-Level (Grade 8) Science Computer-Based Field Tests. Teachers from across the State work with NYSED in a variety of activities to ensure the validity and reliability…

This research study investigates the impact of the rapid transition from paper-based to online administration of the State of Texas Assessments of Academic Readiness (STAAR) exams on student performance in a South Texas school district. The transition, mandated by House Bill 3261, presented Texas public schools with a tight timeline for adapting…

Given the growing trend toward using technology to assess student learning, this investigation examined test mode comparability of student achievement scores obtained from paper-pencil and computerized assessments of statewide End-of-Course and End-of-Grade examinations in the subject areas of high school biology and eighth-grade English Language…

The Big Data for Education Spoke of the NSF Northeast Big Data Innovation Hub and ETS co-sponsored an educational data mining competition in which contestants were asked to predict efficient time use on the NAEP 8th grade mathematics computer-based assessment, based on the log file of a student's actions on a prior portion of the assessment. In…

The New York State Education Department (NYSED) has a partnership with Questar Assessment Inc. (Questar) for the development of the 2020 Grades 3-8 English Language Arts Tests. Teachers from across the State work with NYSED in a variety of activities to ensure the validity and reliability of the New York State Testing Program (NYSTP). The 2020…