Focus groups have an established history in applied research and evaluation. The fundamental methods of the focus group technique have been well discussed, as have their potential advantages. Less guidance tends to be provided regarding the analysis of data resulting from focus groups or how to organize and defend conclusions drawn from the…

Schools are held more responsible for evaluating, quality assuring and improving their student assessments. Teachers' lack of understanding of new, competence-based assessments as well as the lack of key stakeholders' involvement, hamper effective and efficient self-evaluations by teachers of innovative, competence-based assessments (CBAs). While…

The goal of this article is to contribute to the validation of a self-evaluation method, which can be used by schools to evaluate the quality of their Competence Assessment Program (CAP). The outcomes of the self-evaluations of two schools are systematically compared: a novice school with little experience in competence-based education and…

The ethical work of program evaluators is based on a covenant of honesty and transparency among stakeholders. Yet even under the most favorable evaluation conditions, threats to ethical standards exist and muddle that covenant. Unfortunately, ethical issues associated with different evaluation structures and contracting arrangements have received…

As the number of large federal programs increases, so, too, does the need for a more complete understanding of how to conduct evaluations of such complex programs. The research literature has documented the benefits of stakeholder participation in smaller-scale program evaluations. However, given the scope and diversity of projects in multi-site…

The purpose of this study was to assess staff and client perspectives on the Internet-based Journey Mapping program evaluation tool. A drug court program was chosen for a case study research design. Six staff and 10 clients participated in interviews and observations, and also responded to a questionnaire. A staff survey provided additional data.…

Although staff training in public child welfare agencies is thought to be a critical component of effective practice, little is known about the delivery and effectiveness of training interventions. Few evaluations are published in the literature and it is uncertain how much evaluation activity is being conducted and utilized. This study collected…

Although the Delphi is widely used, research on certain methodological issues is somewhat limited. After a brief introduction to the strengths, limitations, and methodological challenges of the technique, we share our experiences (as well as problems encountered) with an electronic Delphi of educational program evaluation (EPE) in the Asia-Pacific…

Educational needs assessments (hereafter NAs) are inattentive to cost considerations and are frequently dominated by elite stakeholder groups. In this article I make a case for adopting a cost-utility approach, illustrating the argument with data generated in a NA of central library services in a Canadian school district. Using survey data from…

An introduction is provided to the use of confirmatory factor analysis to test measurement invariance and stability in longitudinal research. The approach is illustrated through examples representing one or two constructs in one to three measurement waves. Basic issues in establishing measurement invariance are discussed. (SLD)

Application of evaluability assessment by the Cooperative Extension Service of the Department of Agriculture is reviewed. The effort involved five states with six major programs since 1984. Underlying philosophy and methodological considerations are discussed, and background information is provided on the assessment processes and studies…

There is increasing recognition of the need for evaluations that identify program processes or mediators and assess degree of program implementation rather than focusing solely on outcome evaluation. This paper describes the application of complementary qualitative and quantitative evaluation procedures to assess the degree of implementation of…

Twelve specific suggestions for evaluators concerning development, presentation, and followup of improved evaluation recommendations are discussed. Offering effective recommendations requires special skills that need to be recognized, studied carefully, and passed along to practitioners and students. (TJH)

The Communities That Care Youth Survey measures risk and protective factors shown in prior studies to predict adolescent problem behaviors such as drug use, delinquency, and violence. This paper describes the development and validation of cut points for the risk and protective factor scales in the Communities That Care Youth Survey that…

This paper describes the process of developing a participatory monitoring and evaluation strategy for a Kenyan youth-based NGO. The iterative nature of the study including the process of narrowing down indicators to measure and methods to monitor/evaluate these is well documented. A discussion on the extent to which the process achieved…