We consider estimating the effect of a treatment on a given outcome measured on subjects tested both before and after treatment assignment in observational studies. A vast literature compares the competing approaches of modelling the post-test score conditionally on the pre-test score versus modelling the difference, namely, the gain score. Our…

In many psychological and educational applications, it is imperative to obtain valid and reliable score estimates of multilevel processes. For example, in order to assess the quality and characteristics of high impact learning processes, one must compute accurate scores representative of student- and classroom-level constructs. Currently, there…

Young children are assessed to meet federal mandates and inform policy decisions, provide teachers with useful information to make instructional decisions and set reasonable learning goals, and facilitate communication with families. While young children are frequently assessed using whole-child assessments which often yield criterion-referenced…

Sometimes a treatment, such as receiving a high school diploma, is assigned to students if their scores on two inputs (e.g., math and English test scores) are above established cutoffs. This forms a multidimensional regression discontinuity design (RDD) to analyze the effect of the educational treatment where there are two running variables…

High stakes test-based accountability systems primarily rely on aggregates and derivatives of scores from tests that were originally developed to measure individual student mastery of content specifications. Current validity models do not explicitly address this use of aggregate scores to measure the performance of teachers, administrators, and…

This study explores three understudied facets--quadratic effects, change over time, and gender as a moderator--of the otherwise well-documented relationships between statistics anxiety and academic performance. Using pre- and post- course survey data among a sample of 111 undergraduate students in Social Statistics courses at a U.S. Midwestern…

Educators commonly use school climate survey scores to inform and evaluate interventions for equitably improving learning and reducing educational disparities. Unfortunately, validity evidence to support these (and other) score uses often falls short. In response, Whitehouse et al. proposed a collaborative, two-part validity testing framework for…

The year 2022 is the 20th anniversary of Joseph Schafer and John Graham's paper titled "Missing data: Our view of the state of the art," currently the most highly cited paper in the history of "Psychological Methods." Much has changed since 2002, as missing data methodologies have continually evolved and improved; the range of…

Multilevel Rasch models are increasingly used to estimate the relationships between test scores and student and school factors. Response data were generated to follow one-, two-, and three-parameter logistic (1PL, 2PL, 3PL) models, but the Rasch model was used to estimate the latent regression parameters. When the response functions followed 2PL…

Background: The unprecedented access to information in the 21st century entails a deep understanding of information and communication technology (ICT)-related factors in education and their impacts on learning and teaching. The role of attitudes towards ICT is a proven factor in student achievement. However, there is no consensus about the…

In randomized control trials (RCT), the recent focus has shifted to how an intervention yields positive results on its intended outcome. This aligns with the recent push of implementation science in healthcare (Bauer et al., 2015) but goes beyond this. RCTs have moved to evaluating the theoretical framing of the intervention as well as differing…

The purpose of this study was to explore potential effects of a 12-week therapeutic mentoring program targeting social, emotional, and behavioral concerns in 52 children and adolescents between 11 and 17 years of age. Self-reported scores on a norm-referenced behavioral questionnaire were tracked across the span of a mentoring program, and then…

Lord's Paradox occurs when a continuous covariate is statistically controlled for and the relationship between a continuous outcome and group status indicator changes in both magnitude and direction. This phenomenon poses a challenge to the notion of evidence-based policy, where data are supposed to be self-evident. We examined 50 effect size…

This study used a propensity score approach to estimate treatment effects in a multilevel setting. The propensity score approach involves the estimation of propensity scores for covariate balancing and the estimation of treatment effects. This study aimed at understanding how propensity scores estimated through a simple logistic regression compare…

Although much attention has been given to rater effects in rater-mediated assessment contexts, little research has examined the overall stability of leniency and severity effects over time. This study examined longitudinal scoring data collected during three consecutive administrations of a large-scale, multi-state summative assessment program.…