Over the past 15 years, we have seen an increase in the use of cluster randomized trials (CRTs) to test the efficacy of educational interventions. These studies are often designed with the goal of determining whether a program works, or answering the what works question. Recently, the goals of these studies expanded to include for whom and under…

Programs that improve teaching effectiveness represent a core strategy to improve student educational outcomes and close student achievement gaps. This article compiles empirical values of intraclass correlations for designing effective and efficient experimental studies evaluating the effects of these programs. The Early Childhood Longitudinal…

One way in which financial aid is thought to promote college success is by minimizing the time students spend working. Yet, little research has examined if this intended first-order effect occurs, and results are mixed. We leverage a randomized experiment and find that students from low-income families in Wisconsin offered additional grant aid…

Much of the currently available evidence on the causal effects of public prekindergarten programs on school readiness outcomes comes from studies that use a regression-discontinuity design (RDD) with the age cutoff to enter a program in a given year as the basis for assignment to treatment and control conditions. Because the RDD has high internal…

In 2003, Chicago launched "Double-Dose Algebra," requiring students with pretest scores below the national median to take two periods of math--algebra and supplemental coursework. In many schools, assignment to Double Dose changed the peer composition of the algebra classroom. Using school-specific instrumental variables within a…

This study explores the extent to which differences in research design explain variation in Head Start program impacts. We employ meta-analytic techniques to predict effect sizes for cognitive and achievement outcomes as a function of the type and rigor of research design, quality and type of outcome measure, activity level of control group, and…

Despite recent shifts in research emphasizing the value of carefully designed experiments, the number of studies of teacher professional development with rigorous designs has lagged behind its student outcome counterparts. We outline a framework for the design of group randomized trials (GRTs) with teachers' knowledge as the outcome and…

The authors use nationally representative survey data and a research design that relies on contemporaneous within-student and within-teacher comparisons across two academic subjects to estimate how class size affects certain non-cognitive skills in middle school. Their results indicate that smaller eighth-grade classes are associated with…

Drawing on our five years of experience developing WWC evidence standards and reviewing studies against those standards as well as current literature on the design of impact studies, we highlight in this paper some of the most critical issues and common pitfalls in designing and conducting impact studies in education, and provide practical…

This paper provides practical guidance for researchers who are designing and analyzing studies that randomize schools--which comprise three levels of clustering (students in classrooms in schools)--to measure intervention effects on student academic outcomes when information on the middle level (classrooms) is missing. This situation arises…

This article examines the power analyses for the first wave of group-randomized trials funded by the Institute of Education Sciences. Specifically, it assesses the precision and technical accuracy of the studies. The authors identified the appropriate experimental design and estimated the minimum detectable standardized effect size (MDES) for each…

The present meta-analysis examines the effect of grade retention on academic outcomes and investigates systemic sources of variability in effect sizes. Using multilevel modeling (MLM), the authors investigate characteristics of 207 effect sizes across 22 studies published between 1990 and 2007 at two levels: the study (between) and individual…

Actions that the South Carolina Department of Education suggests to agencies completing the development of a Request for Proposal (RFP) are presented. Four parts of the RFP include: (1) scope of work; (2) proposal bidding and contractual information; (3) proposal evaluation; and (4) preparation guide. (RL)

The current literature is replete with instances of imprecise and conflicting usage of "qualitative" and "quantitative" evaluation. This paper presents a framework for making distinctions among several related concepts to eliminate this potential source of confusion. In the process, the terms "qualitative" and…

A design for evaluation studies of educational programs should provide valid and defensible inferences. Goals of evaluation are the identity of major components of inferences and specific validity concerns. Design problems may be resolved by creatively utilizing features of specific evaluations in designing unique conditions that permit valid…