Over the past 15 years, we have seen an increase in the use of cluster randomized trials (CRTs) to test the efficacy of educational interventions. These studies are often designed with the goal of determining whether a program works, or answering the what works question. Recently, the goals of these studies expanded to include for whom and under…

Programs that improve teaching effectiveness represent a core strategy to improve student educational outcomes and close student achievement gaps. This article compiles empirical values of intraclass correlations for designing effective and efficient experimental studies evaluating the effects of these programs. The Early Childhood Longitudinal…

One way in which financial aid is thought to promote college success is by minimizing the time students spend working. Yet, little research has examined if this intended first-order effect occurs, and results are mixed. We leverage a randomized experiment and find that students from low-income families in Wisconsin offered additional grant aid…

In 2003, Chicago launched "Double-Dose Algebra," requiring students with pretest scores below the national median to take two periods of math--algebra and supplemental coursework. In many schools, assignment to Double Dose changed the peer composition of the algebra classroom. Using school-specific instrumental variables within a…

Despite recent shifts in research emphasizing the value of carefully designed experiments, the number of studies of teacher professional development with rigorous designs has lagged behind its student outcome counterparts. We outline a framework for the design of group randomized trials (GRTs) with teachers' knowledge as the outcome and…

The authors use nationally representative survey data and a research design that relies on contemporaneous within-student and within-teacher comparisons across two academic subjects to estimate how class size affects certain non-cognitive skills in middle school. Their results indicate that smaller eighth-grade classes are associated with…

This paper provides practical guidance for researchers who are designing and analyzing studies that randomize schools--which comprise three levels of clustering (students in classrooms in schools)--to measure intervention effects on student academic outcomes when information on the middle level (classrooms) is missing. This situation arises…

The present meta-analysis examines the effect of grade retention on academic outcomes and investigates systemic sources of variability in effect sizes. Using multilevel modeling (MLM), the authors investigate characteristics of 207 effect sizes across 22 studies published between 1990 and 2007 at two levels: the study (between) and individual…

In an effort to maintain internal validity while maximizing the generalizability of evaluation results, the theory-testing methodology that links program activities to outcomes (program theory) is proposed. To evaluate a program by explicating and testing the program theory is to perform evaluation as theory testing. (PN)

The design and conduct of a national evaluation study is discussed, demonstrating that a control group may not provide the no-cause baseline information expected. Resolution of this problem requires reexamination of paradigms, research practices, and policies, as well as the underlying real world constraints and views that generate them. (PN)

LISREL V computer program is applied to a weak quasi-experimental design involving the Head Start program, as a multiple analysis attempt to assure that differences between nonequivalent control groups do not confound interpretation of a posteriori differences. (PN)

Glass and Smith's (EJ 207 324) conclusion supporting small class size may be true, but it is not supported by their evidence. Their analysis does not recognize intervening factors and varying class sizes. (CP)

ifferent evaluation models were applied to data from a high school career education program to investigate problems in statistical conclusion validity and program effectiveness judgments. If potential threats to internal validity are analyzed and protection strategies are developed, more confidence in unplanned ex post facto design using a…

School district central office administrators were studied to determine the effects of state and federal regulatory policies for categorical funding on administrative costs and administrator behavior. Findings showed significant hidden program costs for local districts. Additionally, regulations diverted local managers from educational concerns to…

This study collected exploratory data on a group of organizational adjustment variables (procedural, personnel, process, structural, and strategic) among a group of schools and school districts. Results provide a preliminary basis for suggesting that the adaptive organizational behavior of schools and school districts may be influenced by…