A randomized experiment that measures the impact of a social policy in a sample of the population reveals whether the policy will work on average with universal application. An experiment that includes only the subset of the population that volunteers for the intervention generates narrower "proof-of-concept" evidence of whether the…

Tiered evidence initiatives are an important federal strategy to incentivize and accelerate the use of rigorous evidence in planning, implementing, and assessing social service investments. The Social Innovation Fund (SIF), a program of the Corporation for National and Community Service, adopted a public-private partnership approach to tiered…

Traditionally, evaluation has primarily tried to answer the question "Does a program, service, or policy work?" Recently, more attention is given to questions about variation in program effects and the mechanisms through which program effects occur. Addressing these kinds of questions requires moving beyond assessing average program…

To answer "what works?" questions about policy interventions based on an experimental design, Peck (2003) proposes to use baseline characteristics to symmetrically divide treatment and control group members into subgroups defined by endogenously determined postrandom assignment events. Symmetric prediction of these subgroups in both…

In the current climate of increasing fiscal and clinical accountability, information is required about overall program effectiveness using clinical data. These requests present a challenge for programs utilizing single-subject data due to the use of highly individualized behavior plans and behavioral monitoring. Subsequently, the diversity of the…

Researchers have become increasingly interested in programs' main and interaction effects of two variables (A and B, e.g., two treatment variables or one treatment variable and one moderator) on outcomes. A challenge for estimating main and interaction effects is to eliminate selection bias across A-by-B groups. I introduce Rubin's causal model to…

Using exogenous characteristics to identify endogenous subgroups, the approach discussed in this method note creates symmetric subsets within treatment and control groups, allowing the analysis to take advantage of an experimental design. In order to maintain treatment--control symmetry, however, prior work has posited that it is necessary to use…

I define a treatment effect in terms of a comparison of outcomes and provide a typology of all possible comparisons that can be used to estimate treatment effects, including comparisons that are relatively unknown in both the literature and practice. I then assess the relative merit, worth, and value of all possible comparisons based on the…

D'Eon et al. concluded that change in performance self-assessment means from before to after a workshop can detect workshop success in their and other situations. In this commentary, their recommendation is refuted by showing that (a) self-assessments with balanced over- and underestimations are still biased and should not be used to evaluate…

This article argues with a literature review that a simplistic distinction between strong and weak evidence hinged on the use of randomized controlled trials (RCTs), the federal "gold standard" for generating rigorous evidence on social programs and policies, is not tenable with evaluative studies of complex, field interventions such as…

The current emphasis on best practices for school-based health and mental health programs brings with it the demand for evaluation efforts in schools. This article describes the challenges of launching a successful school program and evaluation, with lessons learned from three projects that focus on intimate partner violence. The authors discuss…