Sound evaluation planning requires numerous decisions about how constructs in a program theory will be translated into measures and instruments that produce evaluation data. This article, the first in a dialogue exchange, examines how decisions about measurement are (and should be) made, especially in the context of small-scale local program…

Attention to evaluation quality is commonplace, even if sometimes implicit. Drawing on her 2010 Presidential Address to the American Evaluation Association, Leslie Cooksy suggests that evaluation quality depends, at least in part, on the intersection of three factors: (a) evaluator competency, (b) aspects of the evaluation environment or context,…

This study addresses the central question "How do evaluators' background characteristics relate to their evaluation design choices?" Evaluators were provided with a fictitious description of a school-based program and asked to design an evaluation of that program. Relevant background characteristics such as level of experience,…

A simulation study was conducted in an attempt to examine how evaluators modify their evaluation design in response to differing stakeholder groups. In this study, evaluators were provided with a fictitious description of a school-based program. They were then asked to design an evaluation of the program. After the evaluation design decisions were…

In contrast with nonindigenous workers, to what extent do unique ethical problems arise when indigenous field-workers participate in field studies? Three aspects of study design and operation are considered: data integrity issues, risk issues, and protection issues. Although many of the data quality issues that arise with the use of indigenous…

This paper describes a process-oriented approach to culturally competent evaluation, focusing on a case study of an evaluation of an HIV/AIDS educational program in Bulawayo, Zimbabwe. We suggest that cultural competency in evaluation is not a function of a static set of prescribed steps but is achieved via ongoing reflection, correction, and…

Drug studies are often cited as the best exemplars of evaluation design. However, many of these studies are seriously biased in favor of positive findings for the drugs evaluated, even to the point where dangerous effects are hidden. In spite of using randomized designs and double blinding, drug companies have found ways of producing the results…

This article addresses ethical dilemmas experienced by street-level research and evaluation workers recruiting and gathering data in community-based research projects. The authors focus on a subgroup of street-level research workers, whom they call research extenders (REs), employed because they share important characteristics with the target…

This is a statement on advocacy, activism, confluence of interest, and uncertainty, perhaps with a surprise ending. No two professional evaluators are the same but many use similar methods. Still, each person will use a method in a somewhat idiosyncratic way. Especially in the interpretation of data, personality and experience have a play.…

Can this evaluation be saved? More precisely, can this evaluation be saved in such a way that both evaluator and client feel satisfied that their points of view were respected and both agree that the evaluation itself provides valid information obtained in a principled manner? Because the scenario describes a preliminary discussion and no contract…