The use of concept mapping in research and evaluation has expanded dramatically over the past 20 years. Researchers in academic, organizational, and community-based settings have applied concept mapping successfully without the benefit of systematic analyses across studies to identify the features of a methodologically sound study. Quantitative…

As the number of large federal programs increases, so, too, does the need for a more complete understanding of how to conduct evaluations of such complex programs. The research literature has documented the benefits of stakeholder participation in smaller-scale program evaluations. However, given the scope and diversity of projects in multi-site…

The purpose of this study was to assess staff and client perspectives on the Internet-based Journey Mapping program evaluation tool. A drug court program was chosen for a case study research design. Six staff and 10 clients participated in interviews and observations, and also responded to a questionnaire. A staff survey provided additional data.…

This paper presents information on re-developing an outcome evaluation for a state-funded program providing service coordination utilizing wraparound to youth with severe emotional disturbance (SED) and their families. Originally funded by the Robert Wood Johnson Foundation, the Kentucky IMPACT program has existed statewide since 1990. Changing…

Although the Delphi is widely used, research on certain methodological issues is somewhat limited. After a brief introduction to the strengths, limitations, and methodological challenges of the technique, we share our experiences (as well as problems encountered) with an electronic Delphi of educational program evaluation (EPE) in the Asia-Pacific…

This article describes the results of a qualitative case study evaluating a research mentorship program in community care settings in Ontario, Canada. The purpose of the program was to build evaluation and research capacity among staff of community care agencies through a mentorship program. Data were collected through in-depth, semi-structured…

Agrees with the power of randomized clinical trials, but makes the case that random assignment raises other, unanticipated threats to internal validity as a result of failing to consider treatment preference in research participant behavior. Reviews benefits and limitations of research strategies that augment randomization while respecting…

A theory-driven approach to validity is proposed. The central argument is that a model or theory should be formulated in a program evaluation and the modeling process should include the identification of potential threats to validity in research. (Author/LMO)

A two-step procedure is proposed for detecting behavioral patterns in observational data in evaluation research. Latent class analysis is used to derive a classification of single observations at the variable level. The second step uses the probabilities that each observation falls into latent classes to derive behavioral preference scores. (SLD)

Empowerment evaluation is defined as a process that encourages self-determination among the recipients of a program evaluation. This article contains a theoretical framework of empowerment evaluation, a discussion of methodological issues, and suggestions for operational considerations for evaluators who wish to implement empowerment evaluation.…

Studied the reliability of the Addiction Severity Index (1980) by using eight trained professional actors as client surrogates to examine the reliability of the instrument when administered in the field. With careful surrogate selection and training, the method proves to be an effective means of assessment. (SLD)

Reports on the use of the tailored design method (TDM) to achieve high survey response in two separate studies of the dissemination of Treatment Improvement Protocols (TIPs). Findings from these two studies identify six factors may have influenced nonresponse, and show that use of TDM does not, in itself, guarantee a high response rate. (SLD)

A fuzzy set model is offered for interpreting vague frequency expressions, such as "rarely" and "sometimes." Results with 152 undergraduates reporting marijuana use reflect different frequency expressions for the same level of use and suggest that self-report validity may be enhanced by analyzing frequency expressions as fuzzy…

The five articles of this special section focus on the quantitative-qualitative debate in program evaluation. This section focuses on philosophical aspects of program evaluation, making it clear that the issues are far more complex than the simple quantitative-qualitative dichotomy described by T. S. Kuhn (1970) implies. (SLD)

Compared two methods for evaluating youth empowerment programs. One program used a traditional, open-ended style of service planning and program tracking, and the other used precoded measurement procedures that invited the direct participation of youth in goal setting and evaluation. Discusses the relatively greater usefulness of the precoded…