Current economic conditions and the increasing competition for government funding are leading to an increased focus on the impact of research. Measuring the impact of research is difficult because not all impacts are direct and some can be negative or result from the identification of problems that require a non-research response. The time between…

This article summarizes critical evaluation needs, challenges, and lessons learned in translational research. Evaluation can play a key role in enhancing successful application of research-based programs and tools as well as informing program refinement and future research. Discussion centers on what is unique about evaluating programs and…

Presents an overview of the concepts and methods employed by outcome researchers evaluating intervention effectiveness in child welfare, and the problems associated with these methods that make their use by agencies impractical. Discusses concepts of statistical significance, effect size, and the meaningfulness of effects. Emphasizes the need for…

Stakeholder-based studies modify the relationship between evaluator and user, producing conflict that can cause evaluators to regress to familiar, traditional patterns that leave interactive evaluation strategies only partially implemented. (Author/PN)

A study of school district uses of evaluation and test data is used to illustrate the variety of ambiguities that can appear in qualitative data. Rules of evidence to distinguish valid and potentially nonvalid testimony are proposed. Uses of testimony in evaluation are discussed. (BS)

In this context, ethics refers to a set of normative standards for resolving dilemmas that policy analysts or evaluators often face with regard to what they should do in performing their roles in the interests of societal desirability. The dilemmas relate to: (1) prediction versus prediction plus prescription; (2) evaluation versus evaluation plus…

Evaluations of special education interventions are different from research on medical interventions. Educational evaluations would be more useful if they applied an actuarial, or probability approach. Information should be collected over time and using various measures, thus increasing the available data. (GDC)

The promise of the stakeholder approach to evaluation lies in its potential to counter criticism that evaluation is too narrow, unrealistic in its standards for success, unfair to program staff and participants, and irrelevant to decision makers. (Author)

Critiquing the stakeholder idea, the author states: if government chooses to take account of competing views in social program evaluation, it can get a better result if it encourages the competing views to find a voice of their own, not to speak through the government's chosen instrument. (Author/PN)

One problem faced by researchers is the selection of the most appropriate research method or methods, given the context of the research setting. This dilemma does not concern the choice of the best research method given the program objectives or research questions, nor the selection of the best state-of-the-art methods. The issue concerns the…

Deliberately involving urban education stakeholders in the Cities-in-Schools evaluation study improved the utility of the findings for formative evaluation stakeholders but not for summative evaluation stakeholders. (Author/PN)

Point of entry problems are faced by evaluators when asked to start an evaluation at an inappropriate time or to perform an inappropriate study. The fundamental purpose of evaluation is to help improve services. Guidelines are presented for choosing when to do a context, input, process, or product evaluation. (GDC)

Strengths and weaknesses of three types of semiquantitative methods used by the federal government in research impact assessment are presented, and examples of their use are reviewed. These include the classic retrospective method, another retrospective approach, and accomplishments books used by selected research-sponsoring organizations. (SLD)

The implementation of research design in studies on substance abuse education is discussed in this meta-analysis. Findings indicate that the quality of research design is constrained by negative feedback from evaluation stakeholders. This finding is applied to an evaluation of the Here's Looking at You, 2000 drug education program currently…

Articles in this special issue deal with the assessment of the impact of research and are divided into segments concerning semiquantitative approaches; qualitative approaches; and quantitative and fiscal approaches. These articles illustrate the importance of the role of motivation and associated incentives. (SLD)