Differences in effect sizes between researcher developed (RD) and independently developed (ID) outcome measures are widely documented but poorly understood in education research. We conduct a meta-analysis using item-level outcome data to test potential mechanisms that explain differences in effects by RD or ID outcome type. Our analysis of 45…

Background: Evidence from education randomized controlled trials (RCTs) in low- and middle-income countries (LMICs) demonstrates how interventions can improve children's educational achievement [1, 2, 3, 4]. RCTs assess the impact of an intervention by comparing outcomes--aggregate test scores--between treatment and control groups. A review of…

Researchers use test outcomes to evaluate the effectiveness of education interventions across numerous randomized controlled trials (RCTs). Aggregate test data--for example, simple measures like the sum of correct responses--are compared across treatment and control groups to determine whether an intervention has had a positive impact on student…

Meta-analytic studies of second language (L2) learning typically employ a classic approach to meta-analysis. Although the classic approach can clarify findings, a multivariate, multilevel meta-analysis (3M) approach increases transparency by accounting for (a) dependencies in the evidence presented by primary studies, (b) methodological…

This paper examines the scalability of the results from the Tennessee Student-Teacher Achievement Ratio (STAR) Project, a prominent educational experiment. We explore how the misalignment between the experimental design and the econometric model affects researchers' ability to learn about the intervention's scalability. We document heterogeneity…

This study aims to introduce network meta-analysis (NMA) to provide educational researchers with an extended view of the reviewing educational research. Meta-analytic methods have been widely used in educational research reviews. However, weaknesses have emerged in the multi-group comparison analysis of educational studies where different…

Education leaders know that they should use research when choosing interventions for their schools, but they don't always know how to read the research that is available. David Steiner explains some of the reasons that reading research is a low priority for educators on the front lines and offers some guidance for determining whether research…

The purpose of this study is to present a set of empirically derived effect size distributions in order to provide field-based benchmarks for assessing the relative effects of interventions aimed at reducing challenging behavior or increasing engagement for young children with and without disabilities. We synthesized 192 single-case designs that…

Many experimental designs in educational and behavioral research involve at least one level of clustering. Clustering affects the precision of estimators and its impact on statistics in cross-sectional studies is well known. Clustering also occurs in longitudinal designs where students that are initially grouped may be regrouped in the following…

Given the growing interest in retrieval practice among educators, it is valuable to know when retrieval practice does and does not improve student learning--particularly for educators who have limited classroom time and resources. In this literature review, we developed a narrow operational definition for "classroom research" compared to…

Single-case design has a long history of use for assessing intervention effectiveness for children with disabilities. Although these designs have been widely employed for more than 50 years, recent years have been especially dynamic in terms of growth in the use of single-case design and application of standards designed to improve the validity…

This supplement concerns Appendix E of the "What Works Clearinghouse (WWC) Procedures Handbook, Version 4.1." The supplement extends the range of designs and analyses that can generate effect size and standard error estimates for the WWC. This supplement presents several new standard error formulas for cluster-level assignment studies,…

In randomized control trials (RCT), the recent focus has shifted to how an intervention yields positive results on its intended outcome. This aligns with the recent push of implementation science in healthcare (Bauer et al., 2015) but goes beyond this. RCTs have moved to evaluating the theoretical framing of the intervention as well as differing…

Over the past 15 years, we have seen an increase in the use of cluster randomized trials (CRTs) to test the efficacy of educational interventions. These studies are often designed with the goal of determining whether a program works, or answering the what works question. Recently, the goals of these studies expanded to include for whom and under…

Descriptive analyses of socially important or theoretically interesting phenomena and trends are a vital component of research in the behavioral, social, economic, and health sciences. Such analyses yield reliable results when using representative individual participant data (IPD) from studies with complex survey designs, including educational…