Evaluators are often called upon to assess the effects of programs. To assess a program effect, evaluators need a clear understanding of how a program effect is defined. Arguably, the most widely used definition of a program effect is the counterfactual one. According to the counterfactual definition, a program effect is the difference between…

In October of 2019, I presented at the IUPUI Assessment Institute in Indianapolis on the topic of academic program review. While almost every campus conducts program reviews, the definition, purpose, structure, and integration of assessment in program reviews is not universally understood, accepted, or implemented. This paper summarizes essential…

The terms "outcome" and "impact" are ubiquitous in evaluation discourse. However, there are many competing definitions that lack clarity and consistency and sometimes represent fundamentally different meanings. This leads to profound confusion, undermines efforts to improve learning and accountability, and represents a…

Purpose: This study presents a novel methodology/process for assessing inclusion of theoretically-based implementation factors within available adoption-ready health promotion programs. Methods: Classroom-based physical activity (CBPA) programs were used as an example to describe the process. Our team selected an implementation science framework…

Purpose: While the number of entrepreneurship education programmes offered around the world is on the rise, research into the assessment of entrepreneurship education programmes is still lacking. The purpose of this paper is to take the stance that entrepreneurship education has to focus on a set of transversal competences aimed at teaching…

This article draws on previously employed methods for conducting fidelity studies and applies them to an evaluation of an unprescribed intervention. We document the process of assessing the fidelity of implementation of the Math Recovery first-grade tutoring program, an unprescribed, diagnostic intervention. We describe how we drew on recent…

The continuously growing demand for program results has produced an increased need for evaluation capacity building (ECB). The "Integrative ECB Model" was developed to integrate concepts from existing ECB theory literature and to structure a synthesis of the empirical ECB literature. The study used a broad-based research synthesis method with…

Background: Since the late 1970s participatory approaches have been widely promoted to evaluate international development programs. However, there is no universal agreement of what is meant by participatory evaluation. For some evaluators, participatory evaluations involve the extensive participation of all stakeholder groups (from donor to…

Good evaluation requires that evaluation efforts themselves be evaluated. Many things can and often do go wrong in evaluation work. Accordingly, it is necessary to check evaluations for problems such as bias, technical error, administrative difficulties, and misuse. Such checks are needed both to improve ongoing evaluation activities and to assess…

Given that the development of treatment fidelity assessment protocol is an integral but too frequently ignored aspect of clinical trials for psychological treatments, the Intensive Mental Health Program (IMHP) sought to build fidelity activities into training, program evaluation, and clinical recordkeeping from the outset of a 3 year study period.…

The chapter explores critical roles of internal evaluators in contemporary organizational settings. The need is highlighted for an expanded, reconfigured, unorthodox set of roles and styles of work to meet the needs of the emerging learning organizations effectively. A discussion of major categories of internal evaluator roles emerged from the…

The web is playing a central role in distance education. The word "usability" is usually synonymous with functionality of the system for the user. Also, usability of a website is defined as something that can be used by a specific group of people to carry out specific objectives in an effective way, with efficiency and satisfaction.…

Local assessment systems are being marketed as formative, benchmark, predictive, and a host of other terms. Many so-called formative assessments are not at all similar to the types of assessments and strategies studied by Black and Wiliam (1998) but instead are interim assessments. In this article, we clarify the definition and uses of interim…

This practical guide to outcomes-based assessment in student affairs is designed to help readers meet the growing demand for accountability and for demonstrating student learning. The authors offer a framework for implementing the assessment of student learning and development and pragmatic advice on the strategies most appropriate for the…

Three primary stances on the wisdom of mixing evaluation models while mixing evaluation methods frame the challenges to defensible mixed-method evaluative inquiry. These challenges are addressed by shifting the mixed-method controversy from models toward other critical features of disparate traditions of inquiry. (Author/SLD)