Analysis of symmetrically predicted endogenous subgroups (ASPES) is an approach to assessing heterogeneity in an ITT effect from a randomized experiment when an intermediate variable (one that is measured after random assignment and before outcomes) is hypothesized to be related to the ITT effect, but is only measured in one group. For example,…

Several analytic strategies exist for opening up the "black box" to reveal more about what drives policy and program impacts. This article focuses on one of these strategies: the Analysis of Symmetrically-Predicted Endogenous Subgroups (ASPES). ASPES uses exogenous baseline data to identify endogenously-defined subgroups, keeping the…

Multisite trials, which are being used with increasing frequency in education and evaluation research, provide an exciting opportunity for learning about how the effects of interventions or programs are distributed across sites. In particular, these studies can produce rigorous estimates of a cross-site mean effect of program assignment…

Interest in the regression discontinuity (RD) design as an alternative to randomized control trials (RCTs) has grown in recent years. There is little practical guidance, however, on conditions that would lead to a successful RD evaluation or the utility of studies with underpowered RD designs. This article describes the use of RD design to…

Using exogenous characteristics to identify endogenous subgroups, the approach discussed in this method note creates symmetric subsets within treatment and control groups, allowing the analysis to take advantage of an experimental design. In order to maintain treatment--control symmetry, however, prior work has posited that it is necessary to use…

This document provides guidance about how to describe studies and report their findings in a way that is clear, complete, and transparent. This document does not include information about how studies are judged against What Works Clearinghouse evidence standards. For information about What Works Clearinghouse evidence standards, please refer to…

Response to Intervention (RTI) is a targeted programme that uses a tiered approach to identify the needs of low achieving pupils. The approach begins with whole class teaching (Tier 1), followed by small group tuition (Tier 2) for those who need more attention, and one to one tutoring (Tier 3) for those who do not respond to the small group…

This "What Works Clearinghouse Procedures and Standards Handbook (Version 3.0)" provides a detailed description of the standards and procedures of the What Works Clearinghouse (WWC). The remaining chapters of this Handbook are organized to take the reader through the basic steps that the WWC uses to develop a review protocol, identify…

Many out-of-school time programs want to learn more about how the children and youth they serve are faring. Outcome evaluations allow programs to do just that. This brief provides a basic review of outcome evaluations, discusses why they are important and when they are useful, and presents guidelines, strategies, and techniques for their use in…

Outcomes evaluation of community partnerships to prevent substance abuse poses many methodological challenges. The nine articles of this special section illustrate the variety of approaches evaluators have taken to studies that are often complicated because of small sample size. (SLD)

Discusses a technique for estimating sample size that does not require an a priori definition of desired precision. Presents five modifications that make the method easier to use and reduce the probability of estimating a larger sample size than necessary. The technique can guide the design of environmental sampling programs. (Contains 31…

A multistakeholder evaluation procedure is presented to address the many challenges in evaluating the performance of condominial sewer projects in Brazil. Condominial sewerage is a promising appropriate technology that is coproduced by users and public agencies, but little is known about project performance. This article shows that…

DETECT, the acronym for Dimensionality Evaluation To Enumerate Contributing Traits, is an innovative and relatively new nonparametric dimensionality assessment procedure used to identify mutually exclusive, dimensionally homogeneous clusters of items using a genetic algorithm ( Zhang & Stout, 1999). Because the clusters of items are mutually…

This report develops the sample design for evaluating five Federal manpower programs in terms of their effects upon trainees' incomes. The study involves a large 10-city sample of trainees in Job Corps, Neighborhood Youth Corps, Job Opportunities in the Business Sector, and Manpower and Development Act Institutional Training Programs (New Careers…

This paper examines issues related to the statistical power of impact estimates for experimental evaluations of education programs. The focus is on "group-based" experimental designs, because many studies of education programs involve random assignment at the group level (for example, at the school or classroom level) rather than at the student…