Evaluators are often called upon to assess the effects of programs. To assess a program effect, evaluators need a clear understanding of how a program effect is defined. Arguably, the most widely used definition of a program effect is the counterfactual one. According to the counterfactual definition, a program effect is the difference between…

A randomized experiment that measures the impact of a social policy in a sample of the population reveals whether the policy will work on average with universal application. An experiment that includes only the subset of the population that volunteers for the intervention generates narrower "proof-of-concept" evidence of whether the…

Policymakers and institutional leaders in higher education too often make decisions based on descriptive data analyses or even anecdote when better analysis options could produce more nuanced and more valuable results. Employing the setting of higher education program evaluation at a midwestern regional public university, for this study we…

The major aim of this manuscript is to bring together two important topics that have recently received much attention in child and adolescent research, albeit separately from each other: single-case experimental designs and statistical mediation analysis. Single-case experimental designs (SCEDs) are increasingly recognized as a valuable…

This article provides insight into how an adequate approach to selecting methods can establish credible and actionable evidence. The authors offer strategies to effectively support Extension professionals, including program developers and evaluators, in being more deliberate when selecting appropriate qualitative and quantitative methods. In…

Underlying all What Works Clearinghouse (WWC) products are WWC Study Review Guides, which are intended for use by WWC certified reviewers to assess studies against the WWC evidence standards. As part of an ongoing effort to increase transparency, promote collaboration, and encourage widespread use of the WWC standards, the Institute of Education…

This technical supplement to the Supplemental Nutrition Assistance Program (SNAP) Employment and Training (E&T) evaluation final reports presents details of the technical approach used for creating analysis variables and estimating impacts of treatment group services on outcomes, including employment, earnings, and SNAP participation. It also…

This article describes a modified importance-performance framework for use in evaluation of recreation-based experiential learning programs. Importance-performance analysis (IPA) provides an effective and readily applicable means of evaluating many programs, but the near universal satisfaction associated with recreation inhibits the use of IPA in…

School-based evaluations of interventions are increasingly common in education research. Ideally, the results of these evaluations are used to make evidence-based policy decisions for students. However, it is difficult to make generalizations from these evaluations because the types of schools included in the studies are typically not selected…

Traditionally, evaluation has primarily tried to answer the question "Does a program, service, or policy work?" Recently, more attention is given to questions about variation in program effects and the mechanisms through which program effects occur. Addressing these kinds of questions requires moving beyond assessing average program…

The U.S. Department of Education (ED) expects all Teacher Incentive Fund (TIF) grantees to conduct an evaluation of their programs. Experience with earlier rounds of TIF grants has shown that evaluations can provide valuable information for managing and improving TIF-supported activities, as well as evidence that these activities have had a…

Program evaluation is a process of carefully collecting information in order to make informed decisions to strengthen specific components of a given program. The type of evaluation an institution decides to undertake depends on the purpose as well as on the information the institution wants to find out about its program. Self-evaluation represents…

Evaluations of vocational education and training (VET) programs play a key role in informing training policy in Australia and elsewhere. Increasingly, such evaluations use observational data from surveys or administrative collections to assess the effectiveness of VET programs and interventions. The difficulty associated with using observational…

This paper develops a theoretical framework that could facilitate the application of the Autovalutazione, Valutazione periodica, Accreditamento (AVA) method in Italian universities, trying to simplify the use of this approach, and to cover the existing gap between Italy and others European academic institutions. The new competitive environment in…

In this article, the author talks about program evaluation, a strategy commonly used by instructional leaders that can help school business officials improve their budget process. As districts struggle to develop budgets in these challenging economic times, school business officials might consider turning to program evaluation for help. Program…