The use of complexity-based evaluation methods remains relatively underexplored in the field of evaluation. While increasingly employed to assess complex interventions, Contribution Analysis (CA) continues to suffer from a lack of operationalization. In this article, we propose enhancing the implementation of CA by leveraging Participatory Systems…

Collective impact (CI) is a structured approach that helps drive multi-sector collaborations to address social problems through systems changes. While the CI approach is gaining popularity, practitioners experience challenges in evaluating its implementation and intended outcomes. We conducted a systematic scoping review to understand evaluation…

In this article, we develop and evaluate a novel system and computing platform to structure, measure, and improve student development using points. We define student development broadly as the achievement of learning to do, know, live together, and be. The system leverages individual agency, social influences, content generation and sharing,…

The current research sought to explore the utility of the Assessment for Intervention (AFI) model in bridging the gap between assessment and intervention, through the conceptual lens of Bio-Ecological Systems Theory (BEST) and Social Constructivist Theory (SCT). AFI is a five-stage assessment model that aims to provide useful and evidence-based…

The current research study utilized a case study research method to test the theory that the creativity and innovative processes are a complex adaptive system. Through qualitative interviews with renowned creators, the current study categorized the data into eight characteristics of complex adaptive systems: path dependent, systems have a history,…

The Campbellian validity typology has been used as a foundation for outcome evaluation and for developing evidence-based interventions for decades. As such, randomized control trials were preferred for outcome evaluation. However, some evaluators disagree with the validity typology's argument that randomized controlled trials as the best design…

Background/purpose: The purposes of this research are to develop a reliable and valid assessment tool for measuring systems thinking skills in upper primary students in Thailand and to establish a normative criterion for evaluating their systems thinking abilities based on educational standards. Materials/methods: The study followed a three-phase…

Assessment has become an important aspect of almost every academic or extracurricular program within institutions of higher education in the United States. This article defines internationalization and its related activities while discussing educational outcomes and offering a model for developing assessment strategies for said activities.…

Quality Assessment Frameworks (QAFs) are standardized templates that include pre-defined criteria, open questions, or a combination to assess the quality of evaluations. They support organizations to limit risks associated with poor-quality evaluation. Qualitative research focused on four QAFs shows that five dimensions of quality are assessed…

Although mentoring interventions have helped to advance diversity, equity, and inclusion (DEI) goals within STEM fields, even the most efficacious programs can be undermined without intentional cultivation of the broader mentoring ecosystem. This study examines participants from eight campus teams that engaged in a multi-session process of…

The study of educational effectiveness has become increasingly complex. Alongside methodological advancements in the investigation and measurement of educational effectiveness, meta-analyses conducted by leading researchers have shown that the field has been suffering from a significant lack of theory or from a weak theoretical basis. The present…

This article aims to provide a systemic instrument to evaluate the functioning of higher education systems. Despite systemic instruments have had a strong impact on the management of public policy systems in fields such as health and innovation, higher education has not been widely discussed in applying this type of instrument. Herein lies the…

Evaluators are becoming increasingly aware that, to provide maximum benefit to the programs they evaluate, they must address the systems within which the programs work. However, evaluation practice is often influenced by beliefs and assumptions that are rooted in an understanding of these systems as stable and predictable. We identify four traps…

While evaluation capacity building (ECB) may hold promise for fostering evaluation, little is known about how it is operationalized within a network. This article presents initial findings from a National Science Foundation-funded research project (Complex Adaptive Systems as a Model for Network Evaluations) that used concepts from complex…

Powerful arguments have been made recently to advocate for the introduction of systems thinking in chemistry education to equip graduates to address sustainability challenges. This contribution describes systems thinking activities and an assessment tool that enables meaningful learning in first-year organic chemistry as students engage with the…