Written answers to open-ended questions can have a higher long-term effect on learning than multiple-choice questions. However, it is critical that teachers immediately review the answers, and ask to redo those that are incoherent. This can be a difficult task and can be time-consuming for teachers. A possible solution is to automate the detection…

Star Math (SM) is a popular computer adaptive test (CAT) schools use to screen students for academic risk. Despite its popularity, few independent investigations of its diagnostic accuracy have been conducted. We evaluated the diagnostic accuracy of SM based upon vendor provided cut-scores (25th and 40th percentiles nationally) in predicting…

This study explored the validity of growth on two computer adaptive tests, Star Reading and Star Math, in explaining performance on an end-of-year achievement test for a sample of students in Grades 3 through 6. Results from quantile regression analyses indicate that growth on Star Reading explained a statistically significant amount of variance…

In 2017, the mathematics assessments that are part of the National Assessment of Educational Progress (NAEP) program underwent a transformation shifting the administration from paper-and-pencil formats to digitally-based assessments (DBA). This shift introduced new interactive item types that bring rich process data and tremendous opportunities to…

The New York State Education Department (NYSED) has a partnership with Questar Assessment Inc. (Questar) for the development of the 2020 Grades 3-8 Mathematics Tests. Teachers from across the State work with NYSED in a variety of activities to ensure the validity and reliability of the New York State Testing Program (NYSTP). The 2020 Grades 3-5…

This paper presents results of a score interpretation study for a computer adaptive mathematics assessment. The study purpose was to test the efficacy of item developers' alignment of items to Range Achievement-Level Descriptors (RALDs; Egan et al.) against the empirical achievement-level alignment of items to investigate the use of RALDs as the…

The current study examined the criterion validity of a computer adaptive universal screener with an end-of-year state mathematics assessment using extant data provided by a local education agency. Participants included 1,195 third through eighth graders. Correlational analyses were used to report predictive and concurrent validity coefficients for…

The purpose of this evaluation was to examine the impact of i-Ready Personalized Instruction on math achievement, as measured by SBA scores. We compared students in schools using i-Ready Personalized Instruction school-wide (Treatment students) and students who only received i-Ready Diagnostic assessments and who were in schools only partially…

We examined semiotic exchangeability in pop-up glossary translations and illustrations used as supports for second language learners (SLLs) in computer-administered mathematics tests. In a sample of 516 mathematics items, Grades 3-8 and 11, from a large-scale assessment program in the US, test developers identified terms that could be translated…

Over the last two decades, the educational use of digital devices, including digital assessments, has become a regular feature of teaching in primary education in the Netherlands. However, researchers have not reached a consensus about the so-called "mode effect," which refers to the possible impact of using computer-based tests (CBT)…

Background: TIMSS 2019 is the first assessment in the TIMSS transition to a computer-based assessment system, called eTIMSS. The TIMSS 2019 Item Equivalence Study was conducted in advance of the field test in 2017 to examine the potential for mode effects on the psychometric behavior of the TIMSS mathematics and science trend items induced by the…

Within multitiered systems of support, assessment practices that limit the amount of time students miss instruction should be prioritized. At the same time, decisions about student response to intervention need to be based upon technically adequate data. We evaluated the impact of data collection frequency and trend estimation method on the…

The impact of the COVID-19 pandemic continues to overwhelm the functioning and outcomes of educational systems throughout the nation. The public education system is under particular scrutiny given that students, families, and educators are under considerable stress to maintain academic progress. Since the beginning of the crisis, school-systems…

Computer adaptive tests (CATs) hold promise to monitor student progress within multitiered systems of support. However, the relationship between how long and how often data are collected and the technical adequacy of growth estimates from CATs has not been explored. Given CAT administration times, it is important to identify optimal data…

The National Assessment of Educational Progress (NAEP) is the largest nationally representative and continuing assessment of what the nation's students know and can do in various subjects such as mathematics, reading, science, and writing, as well as civics, geography, technology and engineering literacy, and U.S. history. The results of NAEP are…