In Comparative Judgement (CJ) exercises, examiners are asked to look at a selection of candidate scripts (with marks removed) and order them in terms of which they believe display the best quality. By including scripts from different examination sessions, the results of these exercises can be used to help with maintaining standards. Results from…

This literature review attempts to survey representative studies within the context of L2 speaking assessment that have contributed to the conceptualization of rater cognition. Two types of studies are looked at: 1) studies that examine "how" raters differ (and sometimes agree) in their cognitive processes and rating behaviors, in terms…

This paper reports on the factors that characterize high-utilization evaluations. It is based on materials submitted to an American Educational Research Association (AERA) Division H competition for outstanding examples of evaluation utilization. The paper is organized into three sections. The first section outlines the background of the study:…

Bertram S. Brown, former director of the National Institute of Mental Health, discusses conflicts between scientific and value dimensions of evaluation. He relates this conflict to: usefulness of mental health program evaluations; conduct of internal evaluations; release of findings; training; rating of reports; and planned change consultation.…

Decision Support Systems (DSS) are automated information systems designed to aid administrative decision making. A literature review on the design, implementation, and evaluation of DSS, suggests that evaluators act as liasons between designers and managers, identify and collect data for DSS, and evaluate DSS. (Author/EGS)

This paper focuses on evaluation-related behaviors of school principals. The role of principals as clients of professional evaluation is examined. Research that views principals as evaluators is reported. Concepts drawn from the literature are used to explore ways of studying the rationality that principals employ to make decisions. (Author/JAZ)

An overview of educational evaluation literature is followed by summaries of research on both the perceptions of school principals about their awareness and use of evaluation in decision-making and their evaluation behavior. New lines of research on evaluation for decision-making as a learning process are offered. (MLF)

The literature on the use of evaluative information in local education agencies is summarized. A general structure for studying evaluation use is presented. It is organized after the commonplaces of education (teacher, student, and subject matter). (BW)

A brief summary of standard setting knowledge is presented, derived from about 20 methods that utilize a judgmental review process, the approach most relevant to the standard-setting strategies proposed in this special issue. Criteria for judging effectiveness and critiques of the methods discussed in the issue are offered. (SLD)

The articles of this special issue propose two methods of deriving an initial standard and one method for determining the extent to which the standard should include compensation. Much work remains to be done on further development of the methods and the larger issues of policy regarding performance assessment. (SLD)

The three standard-setting approaches described in this special issue are summarized and contrasted: (1) judgmental policy capturing; (2) the extended Angoff method; and (3) the dominant profile method. An integrative summary of findings is followed by recommendations for modifying the methods. (SLD)

Evaluation can serve a variety of educational management questions, but an administrator's ability to use evaluation as a management tool depends not only upon his or her own perception of evaluation but also on the perceptions of other potential evaluation users in the system. Research has uncovered a variety of factors influencing an…

Factors associated with the use of program evaluation results are examined. Evaluation can serve a variety of educational management needs if (1) these needs are organized around a central concern, and if (2) stakeholders use evaluation information so that their decision making resolves the central concern. (GDC)

Thirty-two content evaluation studies of teacher licensure tests were reviewed; and methodological differences were noted regarding evaluation focus, the stringency of the charge to expert panelists, and the manner in which panelists' ratings were analyzed. Normative expectations for such content ratings are prepared. (SLD)

The author advances the argument that evaluation is unique not in purpose, but in environment. The evaluator operates within ethical, organizational, and political contexts in order to achieve utilization of a context-responsive evaluation. (LMO)