Most program evaluation efforts concentrate on assessments of "program implementation" and "program outcomes." However, another area of programs that has not received sufficient attention in the literature is evaluating the plan of the program. Since the quality of the plan and planning process can influence program…

Randomized experiments are a strong design for establishing impact evidence because the random assignment mechanism theoretically allows confidence in attributing group differences to the intervention. Growth of randomized experiments within educational studies has been widely documented. However, randomized experiments within education have…

Evidence-based programs and grassroots programs are often adapted during implementation. Adaptations are often hidden, ignored, or punished. Although some adaptations stem from lack of organizational capacity, evaluators report other adaptations happen in good faith or are efforts to better fit the local context. Program implementers, facilitators…

Formative evaluation provides stakeholders with timely feedback to support an intervention's improvement during implementation to maximize its effectiveness. We describe the qualitative methods that guided one study within a formative evaluation of a multicomponent care delivery intervention. We then describe the challenges and lessons learned…

Large-scale education interventions aimed at diminishing disparities and generating equitable learning outcomes are often complex, involving multiple components and intended impacts. Evaluating implementation of complex interventions is challenging because of the interactive and emergent nature of intervention components. Methods that build from…

Implementation fidelity is the degree of compliance with which the core elements of program or intervention practices are used as intended. The scientific literature reveals gaps in defining and assessing implementation fidelity in early intervention: lack of common definitions and conceptual framework as well as their lack of application. Through…

Intervention scientists have used program theory-driven evaluation to design, implement, and assess the success of intervention programs for decades. However, interventions often are designed without the input of the community for which they are intended. The lack of incorporation of community members' voices that participate in various…

We assessed whether an adaptation of value-added analysis (VAA) can provide evidence on the relative effectiveness of interventions implemented in a large school district. We analyzed two datasets, one documenting interventions received by underperforming students, and one documenting interventions received by students in schools benefiting from…

This brief report describes the conception, development, and use of a rubric in evaluating the feasibility of a new program. The evaluators searched for a meta-analytic tool to help organize ideas about what data to collect, and why, in order to create a detailed story of feasibility of implementation for the client. The main advantage of using…

Sustainable community health improvement often requires the implementation of complex interventions in complex systems. Drawing from the "Four Keys to Success" frame (theory, implementation, support, and evaluation), this article describes how we used a formative evaluation approach to foster a learning system capable of monitoring and…

This study systematically reviewed evidence regarding health program and service evaluations in Indigenous contexts. Following the PRISMA guidelines and combining terms for 'Indigenous populations' and 'health programs and services'. Eight principles emerged: Principle 1: Adopting Indigenous led or co-led approaches is vital to balance power…

Evaluation practices are continuing to evolve, particularly in those areas related to formative, participatory, and improvement approaches. Improvement science is one of the evaluative practices. Its strength is that it seeks to embrace stakeholders' and frontline workers' knowledge and experience, who are often tasked with leading improvement…

Despite the increase in federal tiered evidence initiatives that require the use of rigorous evaluation designs, such as randomized experiments, there has been limited guidance in the evaluation literature on practical strategies to implement such studies successfully. This paper provides lessons learned in executing experiments in applied…

The growing emphasis on using evidence to inform decisions in public sector policy making has created both opportunities for and demands on the American Evaluation Association (AEA) in pursuing its mission "to improve evaluation practices and methods, increase evaluation use, promote evaluation as a profession, and support the contribution of…

This case study examined the functional role activities of developmental evaluators in a school and community partnership employing a collaborative program model to improve an urban underperforming public school. Program implementers' assessments of program developments and their experience with developmental evaluators were also examined. Data…