Programs that improve teaching effectiveness represent a core strategy to improve student educational outcomes and close student achievement gaps. This article compiles empirical values of intraclass correlations for designing effective and efficient experimental studies evaluating the effects of these programs. The Early Childhood Longitudinal…

This study explores the extent to which differences in research design explain variation in Head Start program impacts. We employ meta-analytic techniques to predict effect sizes for cognitive and achievement outcomes as a function of the type and rigor of research design, quality and type of outcome measure, activity level of control group, and…

Drawing on our five years of experience developing WWC evidence standards and reviewing studies against those standards as well as current literature on the design of impact studies, we highlight in this paper some of the most critical issues and common pitfalls in designing and conducting impact studies in education, and provide practical…

This article examines the power analyses for the first wave of group-randomized trials funded by the Institute of Education Sciences. Specifically, it assesses the precision and technical accuracy of the studies. The authors identified the appropriate experimental design and estimated the minimum detectable standardized effect size (MDES) for each…

A design for evaluation studies of educational programs should provide valid and defensible inferences. Goals of evaluation are the identity of major components of inferences and specific validity concerns. Design problems may be resolved by creatively utilizing features of specific evaluations in designing unique conditions that permit valid…

This article presents evidence from a national evaluation of the effects of 21st Century Community Learning Center after-school programs. The study was conducted in 12 school districts and 26 after-school centers, at which 2,308 elementary school students who were interested in attending a center were randomly assigned either to the treatment or…

The design and conduct of a national evaluation study is discussed, demonstrating that a control group may not provide the no-cause baseline information expected. Resolution of this problem requires reexamination of paradigms, research practices, and policies, as well as the underlying real world constraints and views that generate them. (PN)

LISREL V computer program is applied to a weak quasi-experimental design involving the Head Start program, as a multiple analysis attempt to assure that differences between nonequivalent control groups do not confound interpretation of a posteriori differences. (PN)

While policy analysis is currently the most popular research strategy for educational administration, it is more effective when used in conjunction with program evaluation. The relationship between these methods is illustrated by examining alternative research designs for a hypothetical school drug abuse program. (BS)

A model is described which illustrates two general uses of multiple analyses to evaluate quasi-experiments: obtaining estimates of the treatment effect, and checking the validity of the experiment. (MH)

ifferent evaluation models were applied to data from a high school career education program to investigate problems in statistical conclusion validity and program effectiveness judgments. If potential threats to internal validity are analyzed and protection strategies are developed, more confidence in unplanned ex post facto design using a…

Conclusion about site-specific determinants of educational outcomes of the controversial longitudinal evaluation of the Follow Through programs (1976) are challenged again by reexamining the Direct Instruction model data. Reanalysis confirms earlier critiques and supports more recent research funding on the relationship between types of…

Utilizing the regression-discontinuity research design, this article explores the effectiveness of a developmental English program in a large, multicampus community college. Routinely collected data were extracted from existing records of a cohort of first-time college students followed for approximately 6 years (N = 1,473). Results are consistent…

The internal validity of the RMC models, especially Model A, is examined. Concern centers on limiting evaluation to cognitive outcomes, using constant percentile as the no-treatment expectation, and using norms for one test to establish the expected no-treatment performance level for another test. (MH)

Seven considerations in designing the Title I Evaluation and Reporting System are reviewed: satisfying state, local, and federal needs; summative evaluation; differences in Title I subprograms; local use of different achievement tests; comparative analysis data; use of approved evaluation designs; and use of a common reporting metric. (MH)