Discusses panels for standard setting and presents 10 guidelines for the selection of panel members for such studies. Panel members should themselves hold the license for which they are producing a cutting score, and they must be familiar with the requirements of the profession and the characteristics of the candidates. (SLD)

Two studies of medical certification examinations were undertaken to assess standard setting using Angoff's Method. Results indicate that (1) specialization within broad content areas does not affect an expert's estimates of the performance of the borderline group; and (2) performance data should be provided during the standard-setting process.…

This paper reports the results of using several alternative methods of setting cut scores. The methods used were: (1) a variation of the Angoff method (1971); (2) a variation of the borderline group method; and (3) an advanced impact method (G. Dillon, 1996). The results discussed are from studies undertaken to set the cut scores for fourth grade…

Presents a procedure for standard setting that involves the cluster analysis of test takers to discover examinee groups that are useful for envisioning marginally competent performance or defining borderline or contrasting groups. Illustrates use of the procedure with a statewide mathematics test, and concludes that cluster analysis is useful in…

While standard setting procedures are typically discussed in terms of deriving a reasonable cutting score for a given form of a test, the situation may be structured such that the standard has been mandated without regard to the test form itself. This situation may result either through legislative or policy actions and may be a fait accompli by…

Research indicating that different cut-off points result from the use of different standard-setting techniques leaves decision makers with a disturbing dilemma: Which standard-setting method is best? This investigation of the reliability and validity of 10 different standard-setting approaches was designed to provide information that might help…

The ways in which standard-setting procedures are carried out in the medical specialty certification area are described. Three experiments that have been conducted with alternatives to currently used normative standards are examined, and reasons why these experiments have had limited success are suggested in this speech. The defensibility of…

Multiple matrix sampling is applied to a variation of Angoff's standard setting method. Thirty-six experts (internists) and 190 items were divided into five groups, and borderline examinee performance was estimated. There was some variability in the cutting scores produced by the individual groups, but various components were well estimated. (SLD)

Considers how educators can best prepare their students to meet the demands of test standards without sacrificing what they know to be effective teaching. Provides examples of how addressing the standards improved both the author's teaching and student learning by providing a focus and a challenge that moved her out of her comfort zone. (SG)

A new approach for examining the quality of judgments from standard-setting judges using a Binomial Trials Model (BTM) is presented and illustrated with 26 judges from the Georgia High School Graduation Test. Results suggest that the BTM provides information not available from other methods. (SLD)

Responds to a negative evaluation of the National Assessment of Educational Progress (NAEP) by the National Academy of Sciences (NAS) and asserts that a review of the evidence for the NAEP performance standards indicates that there is support for the current approach to NAEP standard setting. Considers the scholarship of the NAS evaluation…

Takes issue with support of the standard setting process of the National Assessment of Educational Progress (NAEP) advanced by critics of the evaluation of the standard setting process by the National Academy of Sciences and asserts that the achievement level descriptions developed for the NAEP no longer adhere to policy statements defining the…

Evaluated Angoff ratings from eight different occupational licensing examinations through generalizability theory to estimate the optimal number of raters. Results indicate that approximately 10 to 15 raters is an optimal target range. (SLD)

The authors used a 16-item Ethical Intent Checklist, developed from the American Counseling Association's (1999) Ethical Standards for Internet Online Counseling, to assess the current practices of 88 online counseling Web sites. Results showed fewer than half of online counselors were following the accepted practice on 8 of the 16 items. Online…

When one wants to set data reliability standards for a class of scientific inquiries or when one needs to compare and select among many different kinds of data with reliabilities that are crucial to a particular research undertaking, then one needs a single reliability coefficient that is adaptable to all or most situations. Work toward this goal…