The use of complexity-based evaluation methods remains relatively underexplored in the field of evaluation. While increasingly employed to assess complex interventions, Contribution Analysis (CA) continues to suffer from a lack of operationalization. In this article, we propose enhancing the implementation of CA by leveraging Participatory Systems…

Collective impact (CI) is a structured approach that helps drive multi-sector collaborations to address social problems through systems changes. While the CI approach is gaining popularity, practitioners experience challenges in evaluating its implementation and intended outcomes. We conducted a systematic scoping review to understand evaluation…

Large-scale education interventions aimed at diminishing disparities and generating equitable learning outcomes are often complex, involving multiple components and intended impacts. Evaluating implementation of complex interventions is challenging because of the interactive and emergent nature of intervention components. Methods that build from…

We present this brief reflection on key aspects of the 2030 Agenda for Sustainable Development and the implications of the worldwide focus on achieving the sustainable development goals as external observers of the evaluation endeavor around the world. We have conducted and participated in evaluations, but it is not our primary field of work and…

This article shares lessons learned in applying system evaluation theory (SET) to evaluate a Clinical and Translational Research Center (CTR) funded by the National Institutes of Health. After describing how CTR support cores are intended to work interdependently as a system, the case is made for SET as the best fit for evaluating this evaluand.…

This article presents an approach to use complex adaptive systems thinking to construct a theory of change to plan and evaluate transformational interventions. The article draws on the concepts of domains, scales, agents, and emergence to build a model of the system targeted by the intervention, to identify the chains of causality driving the…

Quality Assessment Frameworks (QAFs) are standardized templates that include pre-defined criteria, open questions, or a combination to assess the quality of evaluations. They support organizations to limit risks associated with poor-quality evaluation. Qualitative research focused on four QAFs shows that five dimensions of quality are assessed…

Marvin Alkin and Jean King published three AJE articles on evaluation use over four years, a coherent and comprehensive series covering the historical development of evaluation use, definitions and factors associated with use and misuse, and theories of evaluation use and influence, concluding with assessment of the first 50 years of use research.…

Fundamental systems transformations are needed to address the global emergency brought on by climate change and related global trends, including the COVID-19 pandemic, which, together, pose existential threats to the future of humanity. "Transformation" has become the clarion call on the global stage. Evaluating transformation requires…

Evaluators are becoming increasingly aware that, to provide maximum benefit to the programs they evaluate, they must address the systems within which the programs work. However, evaluation practice is often influenced by beliefs and assumptions that are rooted in an understanding of these systems as stable and predictable. We identify four traps…

Despite a wide body of literature on the importance of program theory and the need to tackle complexity to improve international development programming, the use of program theory to underpin interventions aimed at facilitating change in complex systems remains a challenge for many program practitioners. The actor-based change framework offers a…

While evaluation capacity building (ECB) may hold promise for fostering evaluation, little is known about how it is operationalized within a network. This article presents initial findings from a National Science Foundation-funded research project (Complex Adaptive Systems as a Model for Network Evaluations) that used concepts from complex…

Sustainable community health improvement often requires the implementation of complex interventions in complex systems. Drawing from the "Four Keys to Success" frame (theory, implementation, support, and evaluation), this article describes how we used a formative evaluation approach to foster a learning system capable of monitoring and…

In this article, we reflect on our experience applying a framework for evaluating systems change to an evaluation of a statewide West Virginia alliance funded by the National Science Foundation (NSF) to improve the early persistence of rural, first-generation, and other underrepresented minority science, technology, engineering, and mathematics…

Evaluation is defined by its central task of valuing--the process and product of judging the merit, worth, or significance of a policy or program. However, there are no clear-cut ways to consider values and render value judgments in evaluation practice. There remains contention in the evaluation field about whether and how to make value judgments.…