The Comparative Short Interrupted Time Series (C-SITS) design is a frequently employed quasi-experimental method, in which the pre- and post-intervention changes observed in the outcome levels of a treatment group is compared with those of a comparison group where the difference between the former and the latter is attributed to the treatment. The…

Randomized controlled trials (RCTs) are considered the "gold standard" for evaluating an intervention's effectiveness. Recently, the federal government has placed increased emphasis on the use of opportunistic experiments. A key criterion for conducting opportunistic experiments, however, is that there is relatively easy access to data…

This study contextualizes an effect-size-like index of educational treatment effects or any group mean differences in academic achievement by referencing time. The new effect size metric can enrich effect size interpretations while serving as a supplement (but not substitute) for conventional standardized effect size measures. Specifically, the…

The current paper investigated the following research questions regarding measures of fidelity: (1) Is there a significant relationship between two different measures of fidelity of implementation: a survey of instructional practices and coded videos of classroom lessons? Does the strength of this relationship differ between treatment and control…

Researchers planning a randomized field trial to evaluate the effectiveness of an educational intervention often face the following dilemma. They plan to recruit schools to participate in their study. The question is, "Should the researchers randomly assign individuals (either students or teachers, depending on the intervention) within schools to…

As a condition of its i3 grant, KIPP contracted with an independent evaluator (Mathematica) to address a key research question: does KIPP maintain its demonstrated effectiveness as it scales? While this question sounds simple enough in theory, it poses several methodological and practical challenges. This paper outlines some of those key…

Attrition occurs when study participants who were assigned to the treatment and control conditions do not provide outcome data and thus do not contribute to the estimation of the treatment effects. It is very common in experimental studies in education as illustrated, for instance, in a meta-analysis studying "the effects of attrition on baseline…

The main confirmatory research question guiding the design of the impact evaluation is: "What is the impact of Success for All (SFA) on elementary school students' reading achievement, compared to students in non-SFA schools?" An answer to this question will determine the assessment of whether SFA is successful at turning around low-performing…

The purpose of the Childhood Resiliency Effects from Schoolwide Treatment (CREST) Pilot was to implement a comprehensive school wide social and character development program aimed at decreasing violence among students and assisting students exposed to violence in Belize City. This one-year pilot program implemented portions of the Positive Action…

For decades, college access programs have aimed to improve college readiness and enrollment rates, particularly for underrepresented populations. Though several published reports have produced comprehensive inventories of college access programs (Gandara, 2001; Perna, 2002; Tierney, Bailey, Constantine, Finkelstein, & Hurd, 2009), no systematic…

In randomized control trials (RCTs) of educational interventions, there is a growing literature on impact estimation methods to adjust for missing student outcome data using such methods as multiple imputation, the construction of nonresponse weights, casewise deletion, and maximum likelihood methods (see, for example, Allison, 2002; Graham, 2009;…

While the idea of teacher performance-pay is increasingly making its way into policy, the evidence on the effectiveness of such programs is both limited and mixed. The central questions in the literature on teacher performance pay to date have been whether teacher performance pay based on test scores can improve student achievement, and whether…