We assessed whether an adaptation of value-added analysis (VAA) can provide evidence on the relative effectiveness of interventions implemented in a large school district. We analyzed two datasets, one documenting interventions received by underperforming students, and one documenting interventions received by students in schools benefiting from…

Despite our best efforts as evaluators, program implementation failures abound. A wide variety of valuable methodologies have been adopted to explain and evaluate the "why" of these failures. Yet, typically these methodologies have been employed concurrently (e.g., project monitoring) or to the post-hoc assessment of program activities.…

In this Exemplars case, the fifth and final under the direction of the current coeditors, the authors present a reflective account of an ongoing, complex, multiyear, multinational monitoring and evaluation (M&E) system conducted for African Women in Agricultural Research and Development (AWARD), an international development program. The…

I define a treatment effect in terms of a comparison of outcomes and provide a typology of all possible comparisons that can be used to estimate treatment effects, including comparisons that are relatively unknown in both the literature and practice. I then assess the relative merit, worth, and value of all possible comparisons based on the…

Evaluators face challenges when programs consistently fail to meet expectations for performance or improvement and consequently, evaluators may recommend that closing a program is the most prudent course of action. However, the evaluation literature provides little guidance regarding when an evaluator might recommend program closure. Given…

Networks aiming for fundamental changes bring together a variety of actors who are part and parcel of a problematic context. These system innovation projects need to be accompanied by a monitoring and evaluation approach that supports and maintains reflexivity to be able to deal with uncertainties and conflicts while challenging current practices…

In the growing literature on the evaluation of program implementation, less has been said about evaluating program quality than about evaluating other aspects of program implementation. Furthermore, most articles and reports in the program-implementation evaluation literature have presented only brief descriptions of how implementation instruments…

Environment and sustainable development show how policies are becoming ever more complex and ambiguous. This trend calls for new evaluation approaches. They need to be more clearly focused on specific, explicit concerns. They must be driven by a strategic concept of use to overcome the vulnerability to manipulation of many integrative, essentially…

Most K-12 evaluations are designed to make inferences about how a program implemented at the classroom or school level affects student learning outcomes and such inferences inherently involve hierarchical data structure. One methodological challenge for evaluators is linking program implementation factors typically measured at the classroom or…

With the emphasis on the use of evidence-based practices has come a need to measure the fidelity of replications to the operations and principles of original models. Recent reviews have focused on methods to develop fidelity measures for evidence-based program models. However, the issue of how to validate such measures has been given scant…

The evaluation of an intervention delivered across multiple treatment sites presents a unique opportunity for evaluators to gauge the manner and degree to which the "impact" of treatment varies across implementation conditions and different target populations. However, the availability of implementation data for each treatment site, while…

The current emphasis on best practices for school-based health and mental health programs brings with it the demand for evaluation efforts in schools. This article describes the challenges of launching a successful school program and evaluation, with lessons learned from three projects that focus on intimate partner violence. The authors discuss…

In the present climate of public accountability, there is increasing demand to show "what works" and what return is gained for the public from investments to improve communities. This increasing demand for accountability is being met with growing confidence in the field of philanthropy during the past 10 years that the impact or effectiveness of…