As a result of the COVID-19 pandemic, at-home testing has become a popular delivery mode in many testing programs. When programs offer at-home testing to expand their service, the score comparability between test takers testing remotely and those testing in a test center is critical. This article summarizes statistical procedures that could be…

Automated scoring systems are typically evaluated by comparing the performance of a single automated rater item-by-item to human raters. This presents a challenge when the performance of multiple raters needs to be compared across multiple items. Rankings could depend on specifics of the ranking procedure; observed differences could be due to…

The scoring scheme for the functional movement screen implicitly assumes that the factor structure is consistent, stable, and congruent across different populations. To determine if this is the case, we compared principal components analyses of three samples: a healthy, general population (n = 100), a group of varsity athletes (n = 101), and a…

Since researchers investigated automatic scoring systems in writing assessments, they have dealt with relationships between human and machine scoring, and then have suggested evaluation criteria for inter-rater agreement. The main purpose of my study is to investigate the magnitudes of and relationships among indices for inter-rater agreement used…

In this report, systematic applications of statistical and psychometric methods are used to develop and evaluate scoring rules in terms of test reliability. Data collected from a situational judgment test are used to facilitate the comparison. For a well-developed item with appropriate keys (i.e., the correct answers), agreement among various…

The percentage of students retaking college admissions tests is rising (Harmston & Crouse, 2016). Researchers and college admissions offices currently use a variety of methods for summarizing these multiple scores. Testing companies, interested in validity evidence like correlations with college first-year grade-point averages (FYGPA), often…

As an alternative to rubric scoring, comparative judgment generates essay scores by aggregating decisions about the relative quality of the essays. Comparative judgment eliminates certain scorer biases and potentially reduces training requirements, thereby allowing a large number of judges, including teachers, to participate in essay evaluation.…

In a recent study, the 30-question Force Concept Inventory (FCI) was theoretically split into two 14-question "half-length" tests (HFCIs) covering the same set of concepts and producing mean scores that can be equated to those of the original FCI. The HFCIs require less administration time and reduce test-retest issues when different…

Over 30 states have adopted the Early Childhood Environmental Rating Scale-Revised (ECERS-R) as a component of their program quality assessment systems, but the use of ECERS-R on such a large scale has raised important questions about implementation. One of the most pressing question centers upon decisions users must make between two scoring…

This study explored differences that might exist in comprehension when students read digital and print texts. Ninety undergraduates read both digital and print versions of newspaper articles and book excerpts on topics of childhood ailments. Prior to reading texts in counterbalanced order, topic knowledge was assessed and students were asked to…

The visual-motor skills of 90 youth with high-functioning autism spectrum disorders (HFASDs) and 51 typically developing (TD) youth were assessed using the Beery-Buktenica Developmental Test of Visual-Motor Integration, Sixth Edition (VMI-VI) and Koppitz Developmental Scoring System for the Bender-Gestalt Test-Second Edition (KOPPITZ-2).…

This experimental study investigates the effect on the examination performance of a cohort of first-year undergraduate learners undertaking a Unified Modelling Language (UML) course using an adaptive learning system against a control group of learners undertaking the same UML course through a traditional lecturing environment. The adaptive…

Comparisons of literary experts and novices indicate that experts engage in interpretive processes to "get the point" during their reading of literary texts but novices do not. In two studies the reading and interpretive processes of literary novices (undergraduates with no formal training in literature study) were elicited through…

Mathematical representation is an important skill in mathematics learning that enables students to interpret and solve problems with ease. However, building confidence in such a skill can be difficult for some students, especially for those who lack self-motivation skills. Therefore, this study examines the effects of the think-talk-write strategy…

A quasi-experimental study was conducted to examine the impact of Strategic and Interactive Writing Instruction on 3rd-5th grade deaf and hard of hearing students' writing and written language compared to a business-as-usual condition (treatment group N = 41, comparison group N = 22). A total of 18 hours of instruction was provided for each of two…