The ability to solve complex problems successfully represents a key competence for students' educational success and beyond. While strategy application and metastrategic knowledge constitute two underlying components that drive successful complex problem solving (CPS), little is known about how these two facets develop individually and jointly in…

Prior research on the Measures of Academic Progress (MAP), a computer-adaptive test distributed by the Northwest Evaluation Association, has primarily focused on the Reading MAP for screening/benchmarking in elementary grades. The purpose of this study was to explore the functional form of growth and the extent to which student variability in…

Assessment of language skills for upper elementary and middle schoolers is important due to the strong link between language and reading comprehension. Yet, currently few practical, reliable, valid, and instructionally informative assessments of language exist. This study provides validation evidence for Monster, P.I., which is a gamified,…

The New York State Education Department (NYSED) has a partnership with Questar Assessment Inc. (Questar) for the development of the 2020 Grades 3-8 Mathematics Tests. Teachers from across the State work with NYSED in a variety of activities to ensure the validity and reliability of the New York State Testing Program (NYSTP). The 2020 Grades 6-8…

This study evaluated the construct validity of six scoring traits of an automated writing evaluation (AWE) system called "MI Write." Persuasive essays (N = 100) written by students in grades 7 and 8 were randomized at the sentence-level using a script written with Python's NLTK module. Each persuasive essay was randomized 30 times (n =…

This paper presents results of a score interpretation study for a computer adaptive mathematics assessment. The study purpose was to test the efficacy of item developers' alignment of items to Range Achievement-Level Descriptors (RALDs; Egan et al.) against the empirical achievement-level alignment of items to investigate the use of RALDs as the…

Scores from computer-based tests are increasingly used to inform a variety of school- and student-level decisions. An underlying assumption is that the associated scores represent effortful responding by each student with respect to the tasks presented. An innovative method for examining evidence for this assumption involves an examination of item…

The New York State Education Department (NYSED) has a partnership with Questar Assessment Inc. (Questar) for the development of the 2019 Grades 3-8 English Language Arts Tests. Teachers from across the State work with NYSED in a variety of activities to ensure the validity and reliability of the New York State Testing Program (NYSTP). The 2019…

This research study investigates the impact of the rapid transition from paper-based to online administration of the State of Texas Assessments of Academic Readiness (STAAR) exams on student performance in a South Texas school district. The transition, mandated by House Bill 3261, presented Texas public schools with a tight timeline for adapting…

The New York State Education Department (NYSED) has a partnership with Questar Assessment Inc. (Questar) for the development of the 2020 Grades 3-8 English Language Arts Tests. Teachers from across the State work with NYSED in a variety of activities to ensure the validity and reliability of the New York State Testing Program (NYSTP). The 2020…

The current study examined the criterion validity of a computer adaptive universal screener with an end-of-year state mathematics assessment using extant data provided by a local education agency. Participants included 1,195 third through eighth graders. Correlational analyses were used to report predictive and concurrent validity coefficients for…

When student achievement is assessed, we seek to elicit a student's maximum performance -- a goal requiring the assumption that the student is fully engaged. Otherwise, to the extent that disengagement occurs, test performance is likely to suffer. Effectively managing test-taking disengagement requires an understanding of the testing conditions…

We examined semiotic exchangeability in pop-up glossary translations and illustrations used as supports for second language learners (SLLs) in computer-administered mathematics tests. In a sample of 516 mathematics items, Grades 3-8 and 11, from a large-scale assessment program in the US, test developers identified terms that could be translated…

Introduction: Students usually answer comprehension questions from texts as part of their academic activities. Elaborated Feedback (EF) has been found to be relatively effective to improve comprehension and learning from texts. However, there is little research on how computer-based feedback influences the question-answering process. This study…

The purpose of this evaluation was to examine the impact of i-Ready Personalized Instruction that met Curriculum Associates' recommended usage levels on ELA achievement, as measured by the Massachusetts Comprehensive Assessment System (MCAS) ELA assessment. This study compared the ELA achievement growth in the 2020-21 school year of students who…