Background/Context: Methodological background: Meta-analysis typically depends on the assumption that true effects follow the normal distribution. While assuming normality of effect "estimates" is often supported by a central limit theorem, normality for the distribution of interventions' "true" effects is a computational…

Background/Context: In the early elementary grades, many students do not achieve literacy proficiency due to inadequate personalized literacy instruction (Taylor et al., 2010). Despite increasing evidence that differentiated instruction informed by assessments of students' reading abilities can improve learning more effectively than…

Analysis of symmetrically predicted endogenous subgroups (ASPES) is an approach to assessing heterogeneity in an ITT effect from a randomized experiment when an intermediate variable (one that is measured after random assignment and before outcomes) is hypothesized to be related to the ITT effect, but is only measured in one group. For example,…

Background: Over the past decade, research on the development and assessment of tools to improve the generalizability of experimental findings has grown extensively (Tipton & Olsen, 2018). However, many experimental studies in education are based on small samples, which may include 30-70 schools while inference populations to which…

Background: This paper develops a new method to estimate quasi-experimental evaluation models when it is necessary to control for measurement error in predictors and individual assignment to the treatment group is based on these same fallible variables. A major methodological finding of the study is that standard methods of estimating models that…

Advocates for the use of a multi-measure system of performance evaluation argue that multiple measures may better capture meaningful differences in employee effectiveness and help align effort with organizational objectives. This may be particularly important in organizations such as schools that produce multiple outcomes. In this paper I use a…

A growing body of research shows that findings from studies often fail to replicate (Open Science Collaboration, 2015; Simmons, Nelson, & Simonsohn, 2011). Deviations from study protocols, sampling error, reduced statistical power, and unethical reporting procedures have been cited as reasons for replication failure (Gilbert, King, Pettigrew,…

Background/Context: The COVID-19 pandemic challenged the way we conduct research. For some modes of data collection, such as interviews, there was a ready (if not perfect) analog: face-to-face became phone-based; paper and pen surveys moved online. Others, such as direct assessments of child development, proved more challenging. Despite the…

We develop a structural after measurement (SAM) method for structural equation models (SEMs) that accommodates missing data. The results show that the proposed SAM missing data estimator outperforms conventional full information (FI) estimators in terms of convergence, bias, and root-mean-square-error in small-to-moderate samples or large samples…

Youth in the U.S. face a mental health crisis--one that has steadily worsened over the past 10 years (Centers for Disease Control, 2023). Youth with disabilities may experience increased mental health risk compared to their peers without disabilities. Recent national surveys and systematic reviews reveal that mental health risk may be elevated for…

Background, Purpose, & Research Questions: High school exit exams are meant to standardize quality and ensure that students graduate with basic knowledge and skills. However, policies that expand exit exams are associated with increased dropout rates, particularly among non-white students (e.g. Papay, Murnane, and Willet, 2010; Reardon and…

Background: Efficacy trials in education often possess a motivating theory of change: how and why should the desired improvement in outcomes occur as a consequence of the intervention? In scenarios with repeated measurements, certain subgroups may be more or less likely to manifest a treatment effect; the theory of change (TOC) provides guidance…

This paper develops models to measure growth in student achievement with a focus on the possibility of differential growth in achievement for low and high-achieving students. We consider a gap-closing model that evaluates the degree to which students in a target group -- students in the bottom quartile of measured achievement -- perform better…

Background/Context: Funders, researchers and developers share an interest in applying rapid cycle evaluation techniques in education (McNall & Foster-Fishman, 2007; Resch, 2016). Both rapid-cycle and other evaluation processes require monitoring the quality of implementation (Moir, 2018). By quickly monitoring and adjusting implementation,…

In recent years, there has been increasing focus on the issue of multiple hypotheses testing in education evaluation studies. In these studies, researchers are typically interested in testing the effectiveness of an intervention on multiple outcomes, for multiple subgroups, at multiple points in time or across multiple treatment groups. When…