The emergence of a new, naturalistic, paradigm of inquiry has led to a demand for rigorous criteria that meet traditional standards of inquiry. Two sets are suggested, one of which, the "trustworthiness" criteria, parallels conventional criteria, while the second, "authenticity" criteria, is implied directly by new paradigm…

What a realistic training program should require, given a broad scope of research methodology, is discussed. Considering the core of evaluation research training, the diversity of methodologies raises issues about what is to be taught and how the involvement of students in actual projects is discussed. (Author/GK)

Considered fundamentally a scientific enterprise, evaluation research also requires artistic skills. The latter are essential to in-house evaluators, more so than to extrainstitutional evaluators. A training model is suggested. (Author/GK)

The role of human judgment in the development and synthesis of evidence has not been adequately developed or acknowledged within quasi-experimental analysis. Corrective solutions need to confront the fact that causal analysis within complex environments will require a more active assessment that entails reasoning and statistical modeling.…

In no-difference research, no differences are found among groups or conditions. This article summarizes the existing commentary on such research. The characteristics of no-difference research, its acceptance by the research community, strategies for conducting such studies, and its centrality within the experimental and nonexperimental paradigms…

A practical theory is offered for probing external validity, the generalization of causal relationships. This alternative theory is built around five principles abstracted from research on construct validation. It emphasizes purposive sampling for theoretical ends rather than random sampling to represent a population. (SLD)

Ten functions of program theory are reviewed: contributing to social science knowledge; assisting policymakers, discriminating between theory failure and program failure, identifying the problem and target group, providing program implementation description, uncovering unintended effects, specifying intervening variables, improving formative use…

Two forms of naturalistic evaluation have appeared within the past decade: (1) a collection of qualitative techniques that are complimentary with conventional quantitative methods; and (2) an alternative paradigm that emphasizes the negotiation of multiple socially constructed realities, interdependence of facts and values, and the emergent…

Modeling the process by which participants are selected into groups, rather than adjusting for preexisting group differences, provides the basis for several new approaches to the analysis of data from nonrandomized studies. Econometric approaches, the propensity scores approach, and the relative assignment variable approach to the modeling of…

Causal models often omit variables that should be included, use variables that are measured fallibly, and ignore time lags. Such practices can lead to severely biased estimates of effects. The discussion explains these biases and shows how to take them into account. (Author)

The adequacy of traditional effect size measures for research synthesis is challenged. Analogues to analysis of variance and multiple regression analysis for effect sizes are presented. The importance of tests for the consistency of effect sizes in interpreting results, and problems in obtaining well-specified models for meta-analysis are…

The need for and applicability of general systems theory in human services evaluation is discussed. The role of evaluation in human services agencies is described and the necessity of combining the programs to be evaluated and the evaluation process itself in a single unifying conceptual model is demonstrated. (Author/AL)

This special issue deals with the preparation of professional evaluators. Evaluation is considered as a profession that warrants specialized training. Aspects of necessary training and descriptions of current programs are discussed. (SLD)

Program theory and implementation process theory contribute, respectively, to specifying the what and the why of program delivery. This article discusses the relationship between these two theories and the types of data collection methods that can be derived from each in order to contribute to more comprehensive evaluation designs. (JAZ)

To enable decision makers to become more aware of the levels of confidence that can be placed in the results of particular computerized decision models, methods for establishing confidence levels are discussed. The U.S. General Accounting Office has developed criteria and review procedures for evaluating computerized policy analysis models.…