This article reviews the origins and use of the terms quasi-experiment and natural experiment. It demonstrates how the terms conflate whether variation in the independent variable of interest falls short of random with whether researchers find, rather than intervene to create, that variation. Using the lens of assignment--the process driving…

This paper introduces a simple, quotient effect size, termed (for 'quotient'), suitable for reporting on the effectiveness of educational interventions. The quotient effect size for a pre-test-post-test design is defined as the gain score (i.e. post-test minus pre-test) for the intervention group, divided by the gain score for the control group.…

By design, randomized experiments (XPs) rule out bias from confounded selection of participants into conditions. Quasi-experiments (QEs) are often considered second-best because they do not share this benefit. However, when results from XPs are used to generalize causal impacts, the benefit from unconfounded selection into conditions may be offset…

Over the last twenty years, education researchers have increasingly conducted randomised experiments with the goal of informing the decisions of educators and policymakers. Such experiments have generally employed broad, consequential, standardised outcome measures in the hope that this would allow decisionmakers to compare effectiveness of…

The What Works Clearinghouse (WWC) identifies existing research on educational interventions, assesses the quality of the research, and summarizes and disseminates the evidence from studies that meet WWC standards. The WWC aims to provide enough information so educators can use the research to make informed decisions in their settings. This…

Randomized control trials are considered the pinnacle for causal inference. In many cases, however, randomization of participants in social work research studies is not feasible or ethical. This paper introduces the co-twin control design study as an alternative quasi-experimental design to provide evidence of causal mechanisms when randomization…

Education decisionmakers need access to the best evidence about the effectiveness of education interventions, including practices, products, programs, and policies. It can be difficult, time consuming, and costly to access and draw conclusions from relevant studies about the effectiveness of interventions. The What Works Clearinghouse (WWC)…

The What Works Clearinghouse (WWC) systematic review process is the basis of many of its products, enabling the WWC to use consistent, objective, and transparent standards and procedures in its reviews, while also ensuring comprehensive coverage of the relevant literature. The WWC systematic review process consists of five steps: (1) Developing…

We present an aggregation scheme that increases power in randomized controlled trials and quasi-experiments when the intervention possesses a robust and well-articulated theory of change. Intervention studies using longitudinal data often include multiple observations on individuals, some of which may be more likely to manifest a treatment effect…

Context: Assessments of baseline equivalency of intervention and control groups, "balance," play a critical role in evaluating educational interventions. The highest What Works Clearinghouse (WWC) of the Institute of Educational Studies (IES) standard for educational studies, "Meets WWC Design Standards Without Reservations,"…

We consider situations in which public charter school lotteries are neither universally conducted nor consistently documented. Such lotteries produce "broken" Randomized Control Trials, but provide opportunities to assess the internal validity of quasi-experimental research designs. Here, we present the results of a statewide charter…

Experimental studies are often employed to test the effectiveness of teaching innovations such as new pedagogy, curriculum, or learning resources. This article offers guidance on good practice in developing research designs, and in drawing conclusions from published reports. Random control trials potentially support the use of statistical…

In the past few decades, we have seen a rapid proliferation in the use of quasi-experimental research designs in education research. This trend, stemming in part from the "credibility revolution" in the social sciences, particularly economics, is notable along with the increasing use of randomized controlled trials in the strive toward…

Background: Current policy discourses emphasise the importance of evidence in education, including evidencing the impact of teacher professional learning on student outcomes. Randomised controlled trial (RCT) designs are promoted to measure 'impact'. Recent debates about this reflect longer standing methodological disputes. Advocates of…

Randomized controlled trials (RCTs) and quasi-experimental designs like regression discontinuity (RD) designs, instrumental variable (IV) designs, and matching and propensity score (PS) designs are frequently used for inferring causal effects. It is well known that the features of these designs facilitate the identification of a causal estimand…