Background: Treatment implementation is not just one thing but rather is a multifaceted process that includes treatment delivery, treatment receipt, and treatment adherence. As such, local variations in implementation and service delivery of interventions are an inevitable. Purpose: To assess implementation fidelity of a multi-site experiential…

Background: Educational researchers have long espoused the virtues of writing with regard to student cognitive skills. However, research on the reliability of the grades assigned to written papers reveals a high degree of contradiction, with some researchers concluding that the grades assigned are very reliable whereas others suggesting that they…

Background: Empathy is an individual's capacity to understand the behavior of others, to experience their feelings, and to express that understanding to them. Empathic ability is an asset professionally for individuals, such as teachers, physicians and social workers, who work with people. Being empathetic is also critical to our being able to…

The purpose of this paper is to provide a basis for judging the adequacy of evaluation plans or, as they are commonly called, evaluation designs. The authors assume that using the procedures suggested in this paper to determine the adequacy of evaluation designs in advance of actually conducting evaluations will lead to better evaluation designs,…

Background: A music education program with a goal of enhancing cognitive development of preschool-aged children enrolled in local preschools is evaluated by The Evaluation Center at Western Michigan University. The budget for the evaluation was small, and therefore presented several challenges to the evaluation team. Purpose: Through a case study…

Background: In program evaluation interventions are usually examined for global effects but not always for differential effects. The reasoning behind the focus on global effects is that most of the concepts applied in data analyses, and most of the hypotheses being assessed, are variable-oriented. Consequently, existing differential effects will…

Background: The philosophical underpinnings of evaluation guidelines set forth by a funding agency can sometimes seem inconsistent with that of the intervention. Purpose: Our purpose is to introduce questions pertaining to the contrast between the instructional program's underlying philosophical beliefs and assumptions and those underlying our…

Good evaluation requires that evaluation efforts themselves be evaluated. Many things can and often do go wrong in evaluation work. Accordingly, it is necessary to check evaluations for problems such as bias, technical error, administrative difficulties, and misuse. Such checks are needed both to improve ongoing evaluation activities and to assess…

This paper presents a retrospective explanation as to how evaluation culture has been slowly incorporated into politics and public institutions in Spain during the last three decades. Evaluation, quality, and excellence are relevant concepts embedded in the design of current political programs. At this moment in history, it is important to ask,…

Background: Social workers entering the profession typically receive little, if any, content or training on evaluation practice. This is, in part, due to limited course offerings outside of the typical courses in most schools of social work. In addition, practicing social workers who often serve in the role as field instructors have not fully…

This review focuses on what the author terms a reconsideration of the working credentials of the randomly controlled trial (RCT) design, and includes a discussion of popularly accepted aspects as well as some new perspectives. The author concludes that there is nothing either Imperative or superior about the need for RCT designs, and that an…

The constraints of conducting evaluations in real-world settings often necessitate the implementation of less than ideal designs. Unfortunately, the standard method for estimating the precision of a result (i.e., confidence intervals [CI]) cannot be used for evaluative conclusions that are derived from multiple indicators, measures, and data…

The purpose of this article is to extend the discussion of issues currently being debated on the need for more rigorous program evaluation in educational and other sectors of research, to the field of international development evaluation, reviewing the different approaches which can be adopted to rigorous evaluation methodology and their…

This article outlines the Results-Based Evaluation (RBE) framework proposed for the ex-post assessment of the National Environmental Management Strategies (NEMS) in 12 small-island developing states (SIDS) in the South Pacific. It gives an overview of the methods and basis of developing an evaluation framework in the context of SIDS in the region.…