As the number of large federal programs increases, so, too, does the need for a more complete understanding of how to conduct evaluations of such complex programs. The research literature has documented the benefits of stakeholder participation in smaller-scale program evaluations. However, given the scope and diversity of projects in multi-site…

The purpose of this study was to assess staff and client perspectives on the Internet-based Journey Mapping program evaluation tool. A drug court program was chosen for a case study research design. Six staff and 10 clients participated in interviews and observations, and also responded to a questionnaire. A staff survey provided additional data.…

This paper presents information on re-developing an outcome evaluation for a state-funded program providing service coordination utilizing wraparound to youth with severe emotional disturbance (SED) and their families. Originally funded by the Robert Wood Johnson Foundation, the Kentucky IMPACT program has existed statewide since 1990. Changing…

Although the Delphi is widely used, research on certain methodological issues is somewhat limited. After a brief introduction to the strengths, limitations, and methodological challenges of the technique, we share our experiences (as well as problems encountered) with an electronic Delphi of educational program evaluation (EPE) in the Asia-Pacific…

This article describes the results of a qualitative case study evaluating a research mentorship program in community care settings in Ontario, Canada. The purpose of the program was to build evaluation and research capacity among staff of community care agencies through a mentorship program. Data were collected through in-depth, semi-structured…

A theory-driven approach to validity is proposed. The central argument is that a model or theory should be formulated in a program evaluation and the modeling process should include the identification of potential threats to validity in research. (Author/LMO)

Empowerment evaluation is defined as a process that encourages self-determination among the recipients of a program evaluation. This article contains a theoretical framework of empowerment evaluation, a discussion of methodological issues, and suggestions for operational considerations for evaluators who wish to implement empowerment evaluation.…

The five articles of this special section focus on the quantitative-qualitative debate in program evaluation. This section focuses on philosophical aspects of program evaluation, making it clear that the issues are far more complex than the simple quantitative-qualitative dichotomy described by T. S. Kuhn (1970) implies. (SLD)

Introduces the special section that focuses on four major studies under the treatment improvement protocols (TIPs) evaluation project. Provides an overview of each article, and addresses the value of using a theory-driven approach to dissemination research. (SLD)

Suggests that there is a tendency in social science and program evaluation to adhere to some methodological practices by force of custom rather than because of their reasoned applicability. These ideas include regression artifacts, random measurement error, and change or gain scores. (Author/SLD)

Agrees with L. Mohr that researchers are too quick to assume that measurement error is random, but disagrees that the idea of regression toward the mean has been a distraction and the notion that change scores analysis should be avoided in favor of regression analysis. (SLD)

Responds to C. S. Reichardt's discussion of regression artifacts, random measurement error, and change scores. Emphasizes that attention to regression artifacts in program evaluation is almost bound to be problematic and proposes some arguments in support of this position. (SLD)

Discusses how a time-series approach was used in the evaluation of the impact of California Senate Bill 2669 on child maltreatment reports. Describes the factors that influenced decisions about evaluation methods and the selection of outcome and exogenous measures. Recommends actions that would facilitate the implementation of evaluations of this…

Outcomes evaluation of community partnerships to prevent substance abuse poses many methodological challenges. The nine articles of this special section illustrate the variety of approaches evaluators have taken to studies that are often complicated because of small sample size. (SLD)

Provides the conceptual framework for the Treatment Improvement Protocols (TIPs) evaluation project, using the diffusion of innovations theory as the theoretical foundation to understand and assess the development of TIPs. Summarizes principles of diffusion theory, and discusses how the model was used to structure the TIPs studies. (SLD)