The Cross-Classified Mixed Effects Model (CCMEM) has been demonstrated to be a flexible framework for evaluating reliability by measurement specialists. Reliability can be estimated based on the variance components of the test scores. Built upon their accomplishment, this study extends the CCMEM to be used for evaluating validity evidence.…

Background & Context: We describe the rationale, design, and initial validation of computational thinking (CT) assessments to pair with curricular lessons that integrate fractions and CT. Objective: We used cognitive models of CT (learning trajectories; LTs) to design assessments and obtained evidence to support a validity argument. Method: We…

This study introduces the signal weighted teacher value-added model (SW VAM), a value-added model that weights student-level observations based on each student's capacity to signal their assigned teacher's quality. Specifically, the model leverages the repeated appearance of a given student to estimate student reliability and sensitivity…

Computerized game-based assessment (GBA) system for screening reading difficulties may provide substantial time and cost benefits over traditional paper-and-pencil assessment while providing means also to individually adapt learning content in educational games. To study the reliability and validity of a GBA system to identify struggling readers…

Machines have a long-demonstrated ability to find statistical relationships between qualities of texts and surface-level linguistic indicators of writing. More recently, unlocked by artificial intelligence, the potential of using machines to identify content-related writing trait criteria has been uncovered. This development is significant,…

This study aims to report the development an instrument that is standardized (reviewed by validity, reliability, and difficulty index) to detect science misconception in an elementary school teacher. This study used a 4-D model; defining, designing, developing, and disseminating. First, it was prepared with 47 opened-ended questions, and then it…

High-quality measurement tools are critical to school improvement efforts. Education researchers frequently employ surveys in order to assess a host of variables associated with school improvement. This article asserts that Rasch modeling techniques enhance the quality of a measurement tool because they comprise elements of both qualitative and…

A drawing assessment to gauge changes in fourth grade students' understanding of the essential components of the longleaf pine ecosystem was developed to support an out-of-school environmental education program. Pre- and post-attendance drawings were scored with a rubric that was determined to have content validity and reliability among users. In…

This research aimed to develop an assessment tool for students' Positive Experiences about Science (PES). A preliminary version of PSE was developed through literature review, consisting of academic emotion, self-concept, learning motivation, career aspiration, and attitude in science. A pilot test was conducted with 198 students and a main test…

Purpose: The purpose of this study was to examine how well a kindergarten dynamic assessment of decoding predicts future reading difficulty at 2nd, 3rd, 4th, and 5th grade and to determine whether the dynamic assessment improves the predictive validity of traditional static kindergarten reading measures. Method: With a small variation in sample…

International large-scale assessments of achievement often have a large degree of differential item functioning (DIF) between countries, which can threaten score equivalence and reduce the validity of inferences based on comparisons of group performances. It is important to understand potential sources of DIF to improve the validity of future…

In this article I advocate for a new discussion in the field of international large-scale assessments; one that calls for a reexamination of international large-scale assessments (ILSAs) and their use. Expanding on the high-quality work in this special issue I focus on three inherent limitations to international large-scale assessments noted by…

Students with significant cognitive disabilities present an assessment dilemma that centers on access and validity in large-scale testing programs. Typically, access is improved by eliminating construct-irrelevant barriers, while validity is improved, in part, through test standardization. In this article, one state's alternate assessment data…

Objective: Determine validity of Day in the Life Questionnaire-Colorado (DILQ-CO) as a dietary assessment tool for classroom-administered use. Methods: Agreement between DILQ-CO responses and weighed plate waste measured in 125 fourth-grade students in 2 low-income schools. Validity assessed by comparing reported school lunch items and portion…

The purpose of the present study was to examine validity aspects of teachers' judgements of pupils' reading skills. Data come from Sweden's participation in the Progress in International Reading Literacy Study (PIRLS) 2001, for Grades 3 and 4. For pupils at the same achievement levels, as measured by PIRLS 2001 test, teachers' judgements of…