Evidence-based programs and grassroots programs are often adapted during implementation. Adaptations are often hidden, ignored, or punished. Although some adaptations stem from lack of organizational capacity, evaluators report other adaptations happen in good faith or are efforts to better fit the local context. Program implementers, facilitators…

In this reflective case, we examine the role and function of collaboration in use-oriented evaluation approaches. Reflecting on our experience evaluating a program that sought to integrate new technologies into K-12 teaching and pre-service teacher education, we found that the desire and pursuit of making evaluation useful can influence why, with…

Background: Mentoring programs pair non-familial adults with children and adolescents for the purposes of promoting positive youth development. Although these programs are widely popular, evaluations tend to show that mentoring programs have, on average, modest effects on youth outcomes. Some researchers have suggested that mentoring programs…

Since 2007, two taxonomies have been proposed to identify the components of evaluation practice that may be specified in an evaluation policy. Little is known, however, about how these taxonomies align with evaluation policies developed by philanthropic foundations. Through thematic analysis, this article first compares 12 foundation evaluation…

Over 2000 years ago, Aristotle wrote of four distinct causes at play in the world we know. Those causes, the material cause, the formal cause, the efficient cause, and the final cause, were meant to refer to ontological and, by extension, epistemological concerns, and were powerful enough to be seized upon and used in some form by those of very…

The Campbellian validity typology has been used as a foundation for outcome evaluation and for developing evidence-based interventions for decades. As such, randomized control trials were preferred for outcome evaluation. However, some evaluators disagree with the validity typology's argument that randomized controlled trials as the best design…

Several influential reviews and two meta-reviews have converged on the position that teacher professional development (PD) is more effective when it is sustained, collaborative, subject specific, draws on external expertise, has buy-in from teachers, and is practice based. This consensus view has now been incorporated in government policy and…

Meaningfully engaging faculty in assessment presents a continual challenge for general education administrators. Faculty perceptions of assessment--including that assessment serves no purpose other than "checking a box" for accreditation and that it conveys little benefit to the instructor--often hinder instructor engagement. A low-cost…

Social return on investment (SROI), an evaluation method that compares monetized social value generated to costs invested, is in ascendance. Conceptually akin to cost-benefit analysis, it shares some of its challenges; however, these are heightened due to the expressed promise of using SROI to compare programs and inform philanthropic and public…

Analysis of symmetrically predicted endogenous subgroups (ASPES) is an approach to assessing heterogeneity in an ITT effect from a randomized experiment when an intermediate variable (one that is measured after random assignment and before outcomes) is hypothesized to be related to the ITT effect, but is only measured in one group. For example,…

The George Mason University (Mason) offices responsible for institutional reporting and assessment were merged to strengthen and solidify their functions and expertise. The organizational structure creating the Office of Institutional Effectiveness and Planning (OIEP) was intended to engineer a new paradigm where assessment would inform the…

Propensity score matching and weighting methods are applied to balance covariates and reduce selection bias in the analysis of observational study data, and ultimately estimate a treatment effect. We wish to evaluate the impact of a Supplemental Instruction (SI) program on student success in an Introductory Statistics course. In such student…

Dr. Hal Abeles is a foundational leader in developing music performance assessments, conducting program evaluations of arts partnerships, and publishing basic research on the effects of arts instruction on learning in and across cognitive, social and personal domains. This article explores the theoretical and practical issues in music performance…

Programme evaluation has developed tremendously over the past 50 years, with a proliferation of evaluation research, an increase in the institutionalization of evaluation, and growth in the professionalization of evaluation. However, existing research and developments are still largely in North America, Europe, Australia, and New Zealand, with…

Amidst diminishing federal investment in Adult Basic Education (ABE), there is growing interest in return on investment (ROI) as an economic rationale to support ABE funding. Against this backdrop, we provide an overview of the ROI concept and methods and the empirical evidence on ABE program impacts to broaden the discourse among practitioners…