A technique developed to monitor the quality of manufactured products, statistical process control (SPC), incorporates several features that may prove attractive to evaluators. This paper reviews the history of SPC, suggests how the approach can enrich program evaluation, and illustrates its use in a hospital-based example. (SLD)

The closing of the Program Evaluation and Methodology Division of the U.S. General Accounting Office marks the end of an era of innovative and important evaluation work. The loss of leadership with the retirement of E. Chelimsky contributed to the decline in the division's fortunes which was brought about by budget constraints. (SLD)

It is argued that reports of evaluation activities often leave much to be desired because they fail to portray the real trial-and-error processes of the research study and do not report on the logic in use at the time of the inquiry. (SLD)

The use of focus groups by the Federal Bureau of Investigation as a tool of internal evaluation is described. Focus groups are used in an environment where credibility is key to achieving meaningful cooperation. Issues for consideration by other evaluators interested in the approach are summarized. (SLD)

The various arts and sciences that comprise the field of program evaluation are discussed. It is argued that emphasis on rigor and expressive content has left other aspects of evaluation unexplored. Educational evaluators need to consider what programs mean and how they contribute to understanding. (SLD)

In spite of the scarcity of empirical studies of evaluation, such studies are critical to development of evaluation theory. Illustrative examples are given of studies of evaluation practice that operationalize theory. Although practitioners should continue public reflection, they must also focus on the independent and empirical study of…

This article discusses four Australian case studies in which photographs and videos formed an important source of program evaluation data. In each case, background information is presented, along with ways the visual data were handled and reported. Where available, outcomes are presented to demonstrate the impact of the visual image. (SLD)

Proposals satisfy a number of functions in evaluation work. This article examines how these functions differ depending on whether the proposal concerns a preordinate study in which methods are prespecified or an emergent study in which decisions about methods are made as the study proceeds. (Author/SLD)

Reviews synthesis as an important task of evaluation, putting evidence together for a coherent assessment. Types of synthesis are described from the case of two variables to more complex situations, ending with metanalysis. Syntheses help identify common patterns and regularities and can help progress toward positive social change. (SLD)

The use of the vignette method with a small group of respondents over a two-year teacher enhancement project sponsored by the National Science Foundation is described, and the usefulness of the method is reviewed. Vignette data provide material sufficient to answer evaluation questions. (SLD)

The evaluation of the preschool program conducted by the High/Scope Educational Research foundation is an example of a program evaluation that employed a randomized design and also examined long-term outcomes. The study, which employed a 22-year follow-up study, is important for its findings and its methodology. (SLD)

Assessing results of grants programs is illustrated with evaluation activities of the National Science Foundation and other federal agencies. Methods considered are ex-ante peer review of proposed work, ex-post peer review of completed work, publications-based methods, and "look-back" methods tracing contributions of past work and…

Continuing the interchange of ideas about evaluation methodology, the authors argue the advisability of using the entire array of methods to produce useful and persuasive findings. There is, ultimately, a truth to be sought, but no one approach has a monopoly on finding it. (SLD)

Unintended effects of evaluation are discussed, with reference to the Hawthorne Effect, where unintended effects overshadow the manipulations controlled by researchers. Unintended effects are categorized as (1) affective reactions; (2) political dimensions; (3) importance of individual differences; and (4) expectations concerning evaluation.…

In spite of many inherent problems, impact evaluation should remain an integral part of program evaluation, both because impact evaluation brings its budget justification with it, and because its focus on causal hypotheses is an essential part of evaluation. Methodology based on the author's body of work is reviewed. (SLD)