In large-scale assessments, such as state-wide testing programs, national sample-based assessments, and international comparative studies, there are many steps involved in the measurement and reporting of student achievement. There are always sources of inaccuracies in each of the steps. It is of interest to identify the source and magnitude of…

Recent work has shown that many state supported school counseling programs have not developed working statewide program evaluation schemas. This study examined two exemplary examples of state level program evaluation. Mixed-method case studies were created and then analyzed across cases to reveal common themes and best practices. The findings…

This article presents an approach to systematic reviews that combines the Campbell Collaboration Crime and Justice standards and the realist notion of contexts-mechanisms-outcomes (CMO) configurations. Both approaches have their advantages and drawbacks, and the authors will make a case for combining both approaches to profit from their advantages…

As public and private funders have focused on evaluating the effectiveness of workforce development programs, a myriad of data collection systems and reporting processes have taken shape. Navigating these systems takes significant time and energy and often saps frontline providers' capacity to use data internally for program improvement.…

Discusses the potential that program theory evaluation has to provide knowledge about the parts of a program that should be retained and the parts that can be changed or should be abandoned. Considers why program theory evaluation may be able to give some guidance about how a program should be changed to function in a new and different…

Teacher evaluation has emerged as a major focus for reform at the highest levels of education policymaking, and for good reason. Most evaluations are based on scant evidence of actual effectiveness, produce inflated ratings, and provide teachers with little useful feedback. This paper offers policymakers and practitioners important "lessons…

Systems of accountability are "second order" ways of using evidence of the standard to which "first order" tasks are carried out for a great variety of purposes. However, more accountability is not always better, and processes of holding to account can impose high costs without securing substantial benefits. At their worst,…

The impact of No Child Left Behind (NCLB) is usually understood in relation to schools and districts, but the legislation has also affected community-based organizations that operate school-linked programs. This case study of an after-school program in California demonstrates how educational accountability systems that emphasize students' academic…

There have been many competent evaluations of youth civic engagement projects. Our work differs from those in assumptions, purposes, goals, methods, activities, and findings for use in evaluation capacity building and program improvement, theoretical and conceptual clarity, and decision-making. Typically, evaluations have focused on predetermined…

The use of evaluation to examine and improve the quality of teaching and courses is now a component of most universities. However, despite the various methods and opportunities for evaluation, a lack of understanding of the processes, measures and value are some of the major impediments to effective evaluation. Evaluation requires an understanding…

"Direct evaluation," as a research concept, is based on the idea of "research as an intermediate technology," developed by Steven and Rachel Kaplan in "Cognition and Environment" (1982: Prager). A description of this evaluative technique and its major contributions to program justification, program evaluation, and…

This article discusses how to make process use an independent variable in evaluation practice: the purposeful means of building an organization's capacity to conduct and use evaluations in the long run. The goal of evaluation capacity building (ECB) is to strengthen and sustain effective program evaluation practices through a number of activities:…

The purpose of this study was to develop a model for evaluating a professional development school program to enact an evidence-based model for a continuous cycle of program improvement. Guided by the logic model for program evaluation, we developed three survey instruments based on the professional development school standards of the National…

This document supplements the advice given in "How good is our school?" (HGIOS)--third edition and "The Child at the Centre"--second edition which set out comprehensive quality frameworks and provide helpful advice on the process of improvement through self-evaluation. In particular, this document provides advice on the…

This review of a research institute's evaluation of the national Follow Through program asserts that both the contracting office and the research institute failed to adequately specify the kind of study intended and the functions which it proposed to serve. One of several kinds of evaluation studies might have been undertaken: (1) a research…