This review describes a meta-analysis of findings from 50 controlled evaluations of intelligent computer tutoring systems. The median effect of intelligent tutoring in the 50 evaluations was to raise test scores 0.66 standard deviations over conventional levels, or from the 50th to the 75th percentile. However, the amount of improvement found in…

Randomized experiments are often seen as the "gold standard" for causal research. Despite the fact that experiments use random assignment to treatment conditions, units are seldom selected into the experiment using probability sampling. Very little research on experimental design has focused on how to make generalizations to well-defined…

To answer "what works?" questions about policy interventions based on an experimental design, Peck (2003) proposes to use baseline characteristics to symmetrically divide treatment and control group members into subgroups defined by endogenously determined postrandom assignment events. Symmetric prediction of these subgroups in both…

In this article, we test an alternative approach to creating a counterfactual basis for estimating individual and average treatment effects. Instead of using control/comparison groups or before-measures, the so-called Counterfactual as Self-Estimated by Program Participants (CSEPP) relies on program participants' self-estimations of their own…

This study explores the extent to which differences in research design explain variation in Head Start program impacts. We employ meta-analytic techniques to predict effect sizes for cognitive and achievement outcomes as a function of the type and rigor of research design, quality and type of outcome measure, activity level of control group, and…

In an effort to bring rigorous research into the education and social sciences field, the past decade had seen an increase in the advocacy of randomized controlled trials. The argument in favor of using randomized controlled trial is that one can evaluate the impact of an intervention in comparison to a control group with confidence during a…

Using exogenous characteristics to identify endogenous subgroups, the approach discussed in this method note creates symmetric subsets within treatment and control groups, allowing the analysis to take advantage of an experimental design. In order to maintain treatment--control symmetry, however, prior work has posited that it is necessary to use…

In this paper the authors extend recent work on mediation in a multilevel setting and on causal inference under interference among units to develop a template for the mediation analysis of group randomized educational interventions. The present work will contribute to the literature on interference, in particular on interference in the context of…

The concept of "implementation fidelity" is broadly used to capture the extent to which an intervention is executed as intended by the designers of the intervention (Century, Rudnick, & Freeman, 2010; Huntley, 2005, McNaught, Tarr, & Sears, 2010, Munter, 2010). Though implementation fidelity instruments are often used to assess…

Developmental summer bridge programs are a popular strategy for increasing college readiness among recent high school graduates. Aimed at providing an alternative to traditional developmental education, these programs provide accelerated and focused learning opportunities in order to help students acquire the knowledge and skills needed for…

The purpose of this study was to evaluate the efficacy of a classroom-teacher-delivered reading intervention for struggling readers called the Targeted Reading Intervention (TRI), designed particularly for kindergarten and first-grade teachers and their struggling students in rural, low-wealth communities. The TRI was delivered via an innovative…

Across the country, a growing number of recent high school graduates are participating in summer bridge programs. These programs provide accelerated and focused learning opportunities in order to help students acquire the knowledge and skills needed for college success. The state of Texas has given particular attention to summer programs as a way…

Background: Recruiting schools into a matched-pair randomized control trial (MP-RCT) to evaluate the efficacy of a school-level prevention program presents challenges for researchers. We considered which of 2 procedures would be most effective for recruiting schools into the study and assigning them to conditions. In 1 procedure (recruit and…

This report addresses the following four questions by reporting on the impacts of Head Start on children and families during the children's preschool, kindergarten, and 1st grade years: (1) What difference does Head Start make to key outcomes of development and learning (and in particular, the multiple domains of school readiness) for low-income…

The gold standard in making causal inference on program effects is a randomized trial. Most randomization designs in education randomize classrooms or schools rather than individual students. Such "clustered randomization" designs have one principal drawback: They tend to have limited statistical power or precision. This study aims to…