The emergence of a new, naturalistic, paradigm of inquiry has led to a demand for rigorous criteria that meet traditional standards of inquiry. Two sets are suggested, one of which, the "trustworthiness" criteria, parallels conventional criteria, while the second, "authenticity" criteria, is implied directly by new paradigm…

Several validity typologies are overviewed then integrated. The integrated framework is used to contrast the positions of Campbell and Cronback. The current practice and logic of quasi-experimentation is critiqued, and expansions beyond the primary focus of dominant validity typologies are suggested. (BS)

Considered fundamentally a scientific enterprise, evaluation research also requires artistic skills. The latter are essential to in-house evaluators, more so than to extrainstitutional evaluators. A training model is suggested. (Author/GK)

The role of human judgment in the development and synthesis of evidence has not been adequately developed or acknowledged within quasi-experimental analysis. Corrective solutions need to confront the fact that causal analysis within complex environments will require a more active assessment that entails reasoning and statistical modeling.…

Secondary analysis is a situation that argues for explicit training in methodology. Evaluative researchers are often required to extend analysis conducted by original evaluators of a program for reasons of dissatisfaction or incompleteness. (Author/GK)

In no-difference research, no differences are found among groups or conditions. This article summarizes the existing commentary on such research. The characteristics of no-difference research, its acceptance by the research community, strategies for conducting such studies, and its centrality within the experimental and nonexperimental paradigms…

Since usually no defensible option for performing a task within quasi-experimentation is unbiased, it is desirable to select several options that reflect biases in different directions. The benefits of applying a critical multiplism approach to causal hypotheses, group nonequivalence, and units of analysis in quasi-experimentation is discussed.…

Methods useful in identifying and assessing outcomes of relatively simple programs may not be practical or appropriate for complex programs and systems. Comparative case study methods are recommended for dealing with such global outcomes of planned organizational change. (Author)

Situations in which mailed questionnaires are most appropriate are identified. Population variables, characteristics of questionnaires, and social desirability variables are examined in depth. (Author)

The dilemma inherent in formal standards results from the tension between maintaining quality control and constraining creativity. This dilemma is nowhere more apparent than in matters of method: measurement, sampling, research design, and statistical analysis. The author offers suggestions for revision of the standards to allow for specific…

A program can be evaluated, and a cost-benefit ratio can be calculated to five decimal places, without ever directly examining outcomes. A useful vocabulary in economics and an understanding of the concepts involved is provided evaluators by an economist. (Author/RL)

Using examples from government program evaluation studies, six areas where research synthesis is more effective than individual studies are presented. Research synthesis can: (1) help match treatment type with recipient type; (2) explain which treatment features matter; (3) explain conflicting results; (4) determine critical outcomes; (5) assess…

Recent methodological advances are described that enable the analyst to extract additional information from the data envelopment analysis (DEA) methodology, including goal programming to develop cross-efficiencies, cluster analysis, analysis of variance, and pooled cross section time-series analysis. Some shortcomings of DEA are discussed. (LMO)

These authors argue that evaluation in medical settings is best taught using experimental and quasi-experimental paradigms as a basic framework. Three types of health evaluation courses at the master's, doctoral, and professional doctoral levels at the University of Michigan that stress the crucial connection between evaluation and policy are…

This theme issue discusses the misuse of evaluation. The eight articles all discuss instances in which the evaluation process or evaluation findings were misused. Misuse is usually not the result of methodological issues but rather is an issue of human relations and sometimes of political pressure. (SLD)