Sound evaluation planning requires numerous decisions about how constructs in a program theory will be translated into measures and instruments that produce evaluation data. This article, the first in a dialogue exchange, examines how decisions about measurement are (and should be) made, especially in the context of small-scale local program…

Carol Weiss did much to enhance the role of evaluation in her writings. Her work shows evaluators what affects their roles as they evaluate programs. Furthermore, her theory of change spells out the complexities involved in program evaluation. There are various processes involved in the evaluation of programs. The paper looks at some of the…

This study addresses the central question "How do evaluators' background characteristics relate to their evaluation design choices?" Evaluators were provided with a fictitious description of a school-based program and asked to design an evaluation of that program. Relevant background characteristics such as level of experience,…

In this paper, the author considers certain aspects of the problem of obtaining unbiased information about the merits of a program or product, whether for purposes of decision making or for accountability. The evaluation of personnel, as well as the evaluation of proposals and evaluations, generally involves a different set of problems than those…

A simulation study was conducted in an attempt to examine how evaluators modify their evaluation design in response to differing stakeholder groups. In this study, evaluators were provided with a fictitious description of a school-based program. They were then asked to design an evaluation of the program. After the evaluation design decisions were…

Good evaluation requires that evaluation efforts themselves be evaluated. Many things can and often do go wrong in evaluation work. Accordingly, it is necessary to check evaluations for problems such as bias, technical error, administrative difficulties, and misuse. Such checks are needed both to improve ongoing evaluation activities and to assess…

As the number of large federal programs increases, so, too, does the need for a more complete understanding of how to conduct evaluations of such complex programs. The research literature has documented the benefits of stakeholder participation in smaller-scale program evaluations. However, given the scope and diversity of projects in multi-site…

Drug studies are often cited as the best exemplars of evaluation design. However, many of these studies are seriously biased in favor of positive findings for the drugs evaluated, even to the point where dangerous effects are hidden. In spite of using randomized designs and double blinding, drug companies have found ways of producing the results…

The third-year evaluation of the federally funded Washington, D.C. voucher program shows that low-income students offered vouchers in the first two years of the program had modestly higher reading scores after three years but showed no significant difference in mathematics. Students were randomly assigned to treatment and control groups, and the…

Some of the most important evaluation questions cannot be answered through experimental research. The "assessment" of the St. Louis Leadership Program (SLLP), sponsored by the Danforth Foundations, is used as an illustration of a "connoisseur-based study" approach to evaluation. The project team, given limited time and funds, decided that it could…

The purpose of this article is to discuss the problem of illusion in research, especially in program evaluation. Currently, there are multiple research traditions in program evaluation. Nevertheless, all research orientations are concerned with the role of researchers, for example, how they interpret their data and what evidence they offer for…

The author compares his experiences evaluating Peace Corps volunteers in Kenya and evaluating a health promotion program for adolescents in Orange County, California. Doing international evaluations can improve domestic evaluations by heightening awareness of often unrecognized cultural variations in the domestic setting. (BS)

Can this evaluation be saved? More precisely, can this evaluation be saved in such a way that both evaluator and client feel satisfied that their points of view were respected and both agree that the evaluation itself provides valid information obtained in a principled manner? Because the scenario describes a preliminary discussion and no contract…

Reflecting on his experiences as an evaluation consultant at the Suez Canal University Faculty of Medicine in Egypt the author discusses: (1) implications of the Egyptian political context; (2) adapting to the host culture; and (3) maintaining distance from one's own culture. Personal qualities to complement the recommended naturalistic methods…

A good formative evaluator, who helps programs become as effective as possible, behaves remarkably like a good personal coach. Nine ways in which a formative evaluator can help a program manager are listed, and implications for practicing evaluators and the evaluation profession are detailed. (SLD)