Problems inherent in setting standards/passing scores for criterion-referenced tests are discussed; and traditional methods of setting standards are reviewed. Three acceptable methods based on judgments of questions are discussed; their authors include, respectively: (1) W. H. Angoff (1971); (2) R. L. Ebel (1972); and (3) L. Nedelsky (1954). (SLD)

Discusses panels for standard setting and presents 10 guidelines for the selection of panel members for such studies. Panel members should themselves hold the license for which they are producing a cutting score, and they must be familiar with the requirements of the profession and the characteristics of the candidates. (SLD)

The recent increase in the use of constructed-response items in educational assessment and the dissatisfaction with the nature of the decision that the judges must make using traditional standard-setting methods created a need to develop new and effective standard-setting procedures for tests that include both multiple-choice and…

The Bookmark standard-setting procedure was developed to address the perceived problems with the most popular method for setting cut-scores: the Angoff procedure (Angoff, 1971). The purposes of this article are to review the Bookmark procedure and evaluate it in terms of Berk's (1986) criteria for evaluating cut-score setting methods. The…

Two studies of medical certification examinations were undertaken to assess standard setting using Angoff's Method. Results indicate that (1) specialization within broad content areas does not affect an expert's estimates of the performance of the borderline group; and (2) performance data should be provided during the standard-setting process.…

This paper reports the results of using several alternative methods of setting cut scores. The methods used were: (1) a variation of the Angoff method (1971); (2) a variation of the borderline group method; and (3) an advanced impact method (G. Dillon, 1996). The results discussed are from studies undertaken to set the cut scores for fourth grade…

Presents a procedure for standard setting that involves the cluster analysis of test takers to discover examinee groups that are useful for envisioning marginally competent performance or defining borderline or contrasting groups. Illustrates use of the procedure with a statewide mathematics test, and concludes that cluster analysis is useful in…

When setting standards, administrators of small-scale OSCEs often face several challenges, including a lack of resources, a lack of available expertise in statistics, and difficulty in recruiting judges. The Modified Borderline-Group Method is a standard setting procedure that compensates for these challenges by using physician examiners and is…

While standard setting procedures are typically discussed in terms of deriving a reasonable cutting score for a given form of a test, the situation may be structured such that the standard has been mandated without regard to the test form itself. This situation may result either through legislative or policy actions and may be a fait accompli by…

Research indicating that different cut-off points result from the use of different standard-setting techniques leaves decision makers with a disturbing dilemma: Which standard-setting method is best? This investigation of the reliability and validity of 10 different standard-setting approaches was designed to provide information that might help…

The ways in which standard-setting procedures are carried out in the medical specialty certification area are described. Three experiments that have been conducted with alternatives to currently used normative standards are examined, and reasons why these experiments have had limited success are suggested in this speech. The defensibility of…

Multiple matrix sampling is applied to a variation of Angoff's standard setting method. Thirty-six experts (internists) and 190 items were divided into five groups, and borderline examinee performance was estimated. There was some variability in the cutting scores produced by the individual groups, but various components were well estimated. (SLD)

Considers how educators can best prepare their students to meet the demands of test standards without sacrificing what they know to be effective teaching. Provides examples of how addressing the standards improved both the author's teaching and student learning by providing a focus and a challenge that moved her out of her comfort zone. (SG)

A new approach for examining the quality of judgments from standard-setting judges using a Binomial Trials Model (BTM) is presented and illustrated with 26 judges from the Georgia High School Graduation Test. Results suggest that the BTM provides information not available from other methods. (SLD)

Responds to a negative evaluation of the National Assessment of Educational Progress (NAEP) by the National Academy of Sciences (NAS) and asserts that a review of the evidence for the NAEP performance standards indicates that there is support for the current approach to NAEP standard setting. Considers the scholarship of the NAS evaluation…