Logic modeling, the process that explicates how programs are constructed and theorized to bring about change, is considered to be standard evaluation practice. However, logic modeling is often experienced as a transactional, jargon-laden, discrete task undertaken to produce a document to comply with the expectations of an external entity, the…

Premised on the idea that evaluators should be familiar with a range of approaches to program modifications, I review several existing approaches and then describe another, less well-recognized option. In this newer option, evaluators work with others to identify potentially needed adaptations for select program aspects "in advance." In…

In Australia, a review of the higher education sector is usually triggered by a change in government leadership, followed by the development and implementation of the government's response in the form of a reform package to enact change. The aim of this study was to conduct an independent evaluation of a large-scale national government policy…

The aim of the article is to describe the assessment model adopted by the SCINTILLA Project, a project in Italy aimed at the online vocational training of young, seriously-disabled subjects and their subsequent work inclusion in smart-work mode. It will thus describe the model worked out for evaluation of the training program conceived for the…

This article presents the results of a national study of 39 higher education institutions that collected information about their evaluation procedures and outcome measures for faculty development for online teaching conducted during 2011-2012. The survey results found that over 90% of institutions used measures of the faculty person's…

To date, evaluation research and policy analysis have focused mainly on average program impacts and paid little systematic attention to their variation. Recently, the growing number of multi-site randomized trials that are being planned and conducted make it increasingly feasible to study "cross-site" variation in impacts. Important…

This paper presents the methods and the main results of the 6 studies which were conducted in order to test the validity of the comprehensive model of educational effectiveness. The synthesis of these studies reveals that some support to the model has been provided. It is also demonstrated that 2 characteristics of the comprehensive model…

This special series focused on treatment integrity in the child mental health and education field is timely. The articles do a laudable job of reviewing (a) the current status of treatment integrity research and measurement, (b) existing conceptual models of treatment integrity, and (c) the limitations of prior research. Overall, this thoughtful…

The aim of this study was to examine the underlying structure of the Evaluation Scale of Educational Programme Implementation (ESEPI). The ESEPI consists of six factors that play an important role in educational programme implementation, namely: training, educational material, administration, facilities, relationships and educational procedure.…

The Comparing Options in Secondary Mathematics: Investigating Curriculum (COSMIC) project is a longitudinal study of student learning from two types of mathematics curricula: integrated and subject-specific. Previous large-scale research studies such as the National Assessment of Educational Progress (NAEP) indicate that numerous variables are…

Describes the a-e-I-o-u program evaluation approach, a framework for organizing key evaluation questions that allows for many models of evaluation and/or methods of data collection. Evaluations by users of the approach have indicated that it provides a practical way to organize evaluation questions and collect appropriate data. (GCP)

Kirkpatrick's highly popular model of training evaluation criteria proposed four ascending, causally related, and positively intercorrelated levels: reactions, learning, behavior, and results. This article examines the validity of the three problematic assumptions of the model, the frequency of each level in published evaluation studies,…

Research and evaluation activities have gradually evolved from the selection of an improvement and evaluation model, as well as data collection using an attitudinal survey and application of case study procedures along with focus group interviews, to the preparation of summary documents and specific policy recommendations. This document presents…

This document provides some guidance to those who are exploring the connections between service-learning and evaluation. Categories are defined in the document by highlighting the following seven questions: (1) What do I need to know about the important issues involved in evaluating my service-learning program? (2) Do you have a model of…

The Office of Management and Budget has recommended the termination of numerous federal programs, citing a lack of program results as the primary reason for this decision. In response to this recommendation, several federal agencies have turned to logic modeling to demonstrate that programs are on the path to results accountability. However,…