Drawing on a qualitative study of an educational reform and its external evaluation, I describe how a well-intentioned but poorly conceptualized evaluation helped perpetuate asymmetries in the generation and use of evaluation findings. This article explores this project's failure to clarify evaluator roles, identify intended users and expected…

Program evaluation often suffers due to time constraints, imperfect instruments, incomplete data, and the need to report standardized metrics. This article about the evaluation process for the Wi$eUp financial education program showcases the difficulties inherent in evaluation and suggests best practices for assessing program effectiveness. We…

Understanding the influence of multisite evaluation requires careful consideration of cultural context. The author illustrates dimensions of influence and culture with excerpts from four National Science Foundation evaluation case studies and summarizes what influence teaches everyone about culture and what culture teaches everyone about…

A cross-case analysis of four National Science Foundation (NSF) case studies identified both unique details and common themes related to promoting the use and influence of multisite evaluations. The analysis provided evidence of diverse evaluation use by stakeholders and suggested that people taking part in the multisite evaluations perceived…

Current economic conditions and the increasing competition for government funding are leading to an increased focus on the impact of research. Measuring the impact of research is difficult because not all impacts are direct and some can be negative or result from the identification of problems that require a non-research response. The time between…

While legislative oversight offices, like many evaluation and policy analysis units, face substantial challenges in promoting use of their work by policymakers, they often have not taken steps to overcome these challenges by adopting the strategies suggested by the evaluation literature. Although the offices seek utilization, they have not fully…

Different dimensions of evaluation utilization and evaluator strategies for stimulating use are discussed, in terms of designing a formal evaluation utilization system. While the need exists to improve the design process, there is no simple formula. Evaluators should see part of their function as creating a repository of knowledge. (SLD)

Studied the nature of barriers to the implementation of external program evaluation through interviews with 18 staff members of a university outreach program. Findings reinforce the importance of a trusting relationship to avoid barriers to evaluation implementation. (SLD)

The nine articles of this theme issue provide overviews of legislative program evaluation and demonstrate the tensions caused by the conflict between utility and rigor in such evaluations. Legislative evaluation is concerned with evaluation utilization because legislators commission evaluations they expect to use. (SLD)

This digest describes some of the basic logic-of-evaluation skills and some of the basic methodological skills that need to be mastered in order to practice the art and science of evaluation. Some topics from the logic of evaluation that must be dealt with include understanding : (1) the differences and connections between evaluation and other…

Chelimsky, former head of the Program Evaluation and Methodology Division of the General Accounting Office, suggested that advocacy by evaluators destroys their credibility. Evaluators should, this author argues, be advocates for democracy and the public interest, with the question being how explicitly and how defensibly. (SLD)

Strategies for use by clients who commission evaluation studies in the private, university, or governmental sector are outlined. Topics covered include establishing the purpose of the evaluation, assembling in-house management staff, specifying work and expertise requirements, establishing confidentiality and project control, and managing results.…

Anecdotal evidence from interviews with some program managers gives evidence that unintended negative consequences of evaluations are more than a theoretical concern; they are a real problem deserving an evaluator's attention and energy. Negative consequences usually include: (1) wasted resources; (2) demotivation; and (3) program destruction and…

Twelve specific suggestions for evaluators concerning development, presentation, and followup of improved evaluation recommendations are discussed. Offering effective recommendations requires special skills that need to be recognized, studied carefully, and passed along to practitioners and students. (TJH)

This paper outlines purposes of program evaluation and provides practical information to help individuals from a wide spectrum of backgrounds use evaluation to move their program ahead in a purposeful manner. Wisely moving ahead requires an assessment of an organization's past and present as well as an examination of its assumptions about the…