Purpose: Multisite mediation studies are a cornerstone in mapping out developmental processes because they probe the mechanisms of a treatment while creating key opportunities to learn from and about variation in those mechanisms across sites. Despite the prevalence of multisite studies, a significant gap in the literature is how to plan such…

An extensive number of studies evaluating educational interventions are conducted and disseminated through peer-reviewed articles and conference presentations annually, but there is limited uptake of research among key stakeholders--educators and administrators (henceforth, practitioners; Farley-Ripple et al, 2018; Hollands et al., 2019; Slavin,…

Introduction: The What Works Clearinghouse (WWC) reviews rigorous research on educational interventions with a goal of identifying "what works" and making that information accessible to educators and policymakers. In rating the quality of causal research, the WWC has historically prioritized internal validity over external validity. One…

Despite a history that can be tracked to 1867, the US Department of Education is a relative newcomer to applying evidence standards to education research. The What Works Clearinghouse (WWC) reviews intervention research to support evidence-based decision making. This is done by applying standards of research quality to studies and giving those…

I. Purpose of the Study: Detecting whether interventions work or not (through main effect analysis) can provide empirical evidence regarding the causal linkage between malleable factors (e.g., interventions) and learner outcomes. In complement, moderation analyses help delineate for whom and under what conditions intervention effects are most…

Background: The emergence of causal research in education was almost strictly quantitative twenty years ago, however, that landscape has changed considerably. The number of intervention studies fielded and completed annually has increased substantially, and the quality of the evaluations is much more robust, including paying much greater attention…

Background: Since the founding of the Institute of Education Sciences (IES) in 2002, the field of education research has seen impressive progress in its efforts to understand which interventions can effectively improve student outcomes. This success can be seen in the rise of high-quality causal studies -- including both randomized trials and…

Background/Context: Attrition has been described as "the Achilles Heel of the randomized experiment" (Shadish et al., 1998 p.3). Attrition looms as a threat because it can undermine group equivalence, eroding the methodological strength at the heart of a randomized evaluation. In particular, attrition could result in unobserved…

Background: The Supporting Effective Educator Development (SEED) Grant Program aims to increase the number of highly effective educators by supporting the implementation of evidence-based practices that prepare, develop, or enhance the skills of educators. The three central goals of this program are: (1) Recruit, prepare, and retain effective and…

Background/Context: When a researcher tests an educational program, product, or policy in a randomized controlled trial (RCT) and detects a significant effect on an outcome, the intervention is usually classified as something that "works." When the expected effects are not found, however, there is seldom an orderly and transparent…

Introduction: The What Works Clearinghouse (WWC) reviews rigorous research on educational interventions with a goal of identifying "what works" and making that information accessible to educators and policymakers. The WWC has historically prioritized internal validity over external validity in rating the quality of research. One critique…

Background/Context: Methodological background: Meta-analysis typically depends on the assumption that true effects follow the normal distribution. While assuming normality of effect "estimates" is often supported by a central limit theorem, normality for the distribution of interventions' "true" effects is a computational…

Background/Context: Some observational studies involve multiple layers of treatment selection, specifically in the context of the extended time accommodation (ETA) for English language learners (ELLs). In ETA settings, the first selection occurs due to the eligibility rule, where students whose ELL English proficiency is below a certain threshold…

Background: Research clearinghouses (CHs) seek to play an important role in identifying successful social programs and practices. The evidence-based policy movement has grown tremendously over the last few decades and has gotten institutionalized in the form of CHs in the US. CHs (1) specify standards for evaluating the quality of evidence from…

Literacy skills are instrumental to unlocking a child's full potential and have a well-established link to long-term life outcomes. Children who can't read at grade level by 3rd grade are four times more likely to drop out of school, have fewer career prospects, and earn lower incomes as adults. However, despite some progress on literacy rates,…