The Campbellian validity typology has been used as a foundation for outcome evaluation and for developing evidence-based interventions for decades. As such, randomized control trials were preferred for outcome evaluation. However, some evaluators disagree with the validity typology's argument that randomized controlled trials as the best design…

Large-scale education interventions aimed at diminishing disparities and generating equitable learning outcomes are often complex, involving multiple components and intended impacts. Evaluating implementation of complex interventions is challenging because of the interactive and emergent nature of intervention components. Methods that build from…

The use of complexity-based evaluation methods remains relatively underexplored in the field of evaluation. While increasingly employed to assess complex interventions, Contribution Analysis (CA) continues to suffer from a lack of operationalization. In this article, we propose enhancing the implementation of CA by leveraging Participatory Systems…

When a researcher tests an educational program, product, or policy in a randomized controlled trial and detects a significant effect on an outcome, the intervention is usually classified as something that "works." When expected effects are not found, there is seldom an orderly and transparent analysis of plausible reasons why.…

A multitude of global challenges that society grapples with, including climate change, social injustices, and economic disparities, persist largely due to the shortcomings of effectively responding to complex systems. In this article, we consider adopting systems literacy as a comprehensive educational approach to navigate in complex systems. We…

STEM teacher shortage remains a serious problem in secondary education in many European countries, despite many interventions and many research studies over the last decades. The observation that past interventions and research outcomes have not significantly contributed to reduce teacher shortage implies that places to effectively intervene in…

Background: Whole-of-community interventions have been recommended and show promise for preventing obesity; however, research to understand the mechanisms underlying their success or failure is lacking. Complex systems approaches may be useful to address this gap. Purpose: To describe the evolution and utilization of qualitative and quantitative…

Many geoscience education initiatives now involve cross-departmental or multi-institutional programs. However, the geoscientists who lead such programs typically have little experience or training in program design, leadership, or evaluation. In this commentary, we make the case that geoscientists taking on these ambitious leadership roles can…

This paper describes a new leadership coaching model that was delivered as part of Manchester city region's delivery of the Department for Education's Early Outcomes Fund. The coaching model explicitly paralleled the relational practices that are increasingly shaping early intervention policy and practice. Goodwin's theory of professional vision…

The intent of this report is to contribute to discussions about what is involved in efforts to implement the type of broad-based, multifaceted system changes seen as essential for fundamentally improving schools. The report begins by stressing four interrelated considerations involved in making multifaceted and complex sustainable changes at a…

This article describes the development and use of a rapid evaluation approach to meet program accountability and learning requirements in a research for development program operating in five developing countries. The method identifies clusters of outcomes, both expected and unexpected, happening within areas of change. In a workshop, change agents…

Despite a wide body of literature on the importance of program theory and the need to tackle complexity to improve international development programming, the use of program theory to underpin interventions aimed at facilitating change in complex systems remains a challenge for many program practitioners. The actor-based change framework offers a…

One core feature of Positive Behavioral Interventions and Support (PBIS) is a systems-level teaming process for coordinating staff implementation of evidence-based practices and monitoring student progress across all three tiers. Prior research has shown schools that report regular teaming and team-based data use are more likely to successfully…

The objective of this project was to learn what elements of a systemic-focused implementation guide are key to facilitating the involvement of families and faculty in rural southeastern North Carolina elementary schools as an intervention to reduce the use of exclusionary discipline for children's externalized behavior in school. Guided by the…

Community college students are an untapped reservoir of talent and including them in social work education aligns with the values of the profession. Yet these students encounter formidable obstacles on the path from a 2-year degree to an affordable social work degree. In this article, we describe the systems thinking underlying a newly instituted…