This article describes the on-going evaluation of a pilot program characterized by: (1) cooperation between program managers and evaluators; (2) first-hand knowledge about the implementation of the pilot; and (3) a focus on summative and formative evaluations. Evaluators participated actively in the design of the pilot. (Author/SLD)

Discusses three models of consultation and their assumptions regarding the roles and responsibilities of evaluation consultants, suggests collaborative approaches as appropriate for evaluating the internal functioning of distance education programs, and presents the application of a collaborative model to the selection of evaluation issues at…

Identifies a series of questions asked by language program personnel in the early stages of planning an evaluation of a language program. Answers to the questions offer working definitions of language programs and components, audience, evaluation purposes and priorities, evaluators and their roles, internal and external evaluation, and evaluation…

It is assumed that evaluators of the future will still be interested in case study methodology. Scenarios that ignore a case study method, that look back to a distinctive case study method, and that see the case study method as an integrating force in the qualitative-quantitative debate are explored. (SLD)

Several extensions to the Angoff method of standard setting are described that can accommodate characteristics of performance-based assessment. A study involving 12 panelists supported the effectiveness of the new approach but suggested that panelists preferred an approach that was at least partially conjunctive. (SLD)

The articles of this special issue propose two methods of deriving an initial standard and one method for determining the extent to which the standard should include compensation. Much work remains to be done on further development of the methods and the larger issues of policy regarding performance assessment. (SLD)

Discusses the development of a national accreditation scheme for volunteer assessors in the expeditions section of the Duke of Edinburgh's Award program for young people. Stages of accreditation include wild country experience, introductory learning and preparation, accreditation weekend, and induction assessments. Performance criteria are…

Two studies involving a total of 99 experts examined the reproducibility of standards for 2 medical certifying examinations set under different conditions. Together, results of both studies provide evidence that a modified version of the Angoff method is quite reliable and produces stable results under varying conditions. (SLD)

Using results for 1,487 West German eleventh graders from the International Study of Achievement in Written Composition, problems related to interrater and intrarater reliability and generalizability of composition ratings were assessed. Results indicate that numerous writing assignments are required to assure reliability. (TJH)

The following models which provide an estimate of the reliability of a rating scale are described: (1) disattenuation; (2) common factor; (3) external criterion; (4) treatment effects; and (5) regression. Such models are especially useful in selecting, evaluating, and training participants in clinical research. (SLD)

The validity of vignettes used to elicit subject responses in social science research is examined in a study of the validity of a group of patient assault vignettes seen by 12 persons with experience in patient care. A model for establishing the validity of vignettes is proposed. (SLD)

Four key entities participating in foreign assistance projects are (1) donors; (2) technical advisors; (3) host country counterparts; and (4) beneficiaries. Resistance to collaborative evaluation that each may exhibit is discussed, along with ways evaluators can overcome resistance and enhance evaluation outcomes, credibility, and utilization.…

An approach to evaluation initiated by the Kellogg Foundation called cluster evaluation (not to be confused with cluster sampling) is described. Cluster evaluation provides a skilled, credible evaluator to work with a cluster of projects to assure that useful and defensible information is obtained for program evaluation. (SLD)

Two studies were conducted to determine the stability of self-assessment of evaluation performance and use by 276 evaluators of community mental health centers over a 6-week period. The test-retest reliability of two instruments developed to assess internal evaluation was examined. The value of these instruments is discussed. (SLD)

An editorial and two articles on quality control in evaluation are presented. Topics covered include definition and assessment of evaluation quality and development of standards for judging quality. Programs inspections activities of the Office of Inspector General of the Department of Health and Human Services are discussed. (TJH)