The authors observe in this commentary that construct maps can help standard-setting panels to make realistic and internally consistent recommendations for performance-level descriptions (PLDs) and cut-scores, but the benefits may not be realized if policymakers do not fully understand the rationale for the recommendations provided by the…

Compares Angoff and Nedelsky methods for setting passing scores on tests. Using one of the methods, 84 college students were taught to estimate their probable scores on a vocabulary test. Estimates were compared to their later actual scores. The Nedelsky method was considerably less accurate under certain conditions, and both methods…

The use of item response theory models for analyzing the results of judgmental standard setting studies (the Angoff technique) for establishing minimum pass levels is discussed. A comparison of three methods indicates the traditional approach may not be best. A procedure based on generalizability theory is suggested. (GDC)

Describes examinee-centered method for setting multiple cutscores on a test involving both objective and extended-response items. Judges evaluate a representative sample of examinee performance using a rating scale that is defined in terms of performance standards, and these ratings are linked to examinee's test scores to generate a functional…

Several methods for determining a passing score on an examination from individual raters' estimates of minimal pass levels were compared through simulation. The methods used differed in the weighting estimates for each item received in the aggregation process. Reasons why the simplest procedure is most preferred are discussed. (SLD)

A large number of seemingly diverse coefficients have been proposed as indices of dependability, or reliability, for domain-referenced and/or mastery tests. In this paper, it is shown that most of these indices are special cases of two generalized indices of agreement: one that is corrected for chance, and one that is not. The special cases of…