The "Standards for Educational and Psychological Testing" (1985) and the "Uniform Guidelines on Employee Selection Procedures" (1978) provide no clear guidance on choosing methods for setting standards on teacher certification tests, although they require that practices be well-documented and produce reasonable results, and…

In most judgmental methods for setting performance standards, content experts evaluate test items based on a conceptualization of a borderline examinee or a group of examinees. An alternative method is proposed by which judges evaluate items based on the degree of criticality regardless of content area. This classification of items serves to link…

The American Nurses' Association certification provides professional recognition beyond licensure to nurses who pass an examination. To determine the passing score as it would be set by a representative peer group, a survey was mailed to a random sample of 200 recently certified nurses. Three questions were asked: (1) what percentage of examinees…

In order to evaluate standard setting procedures, apart from the more commonly applied approach of simply comparing the derived standards or failure rates across various techniques, this study investigated the errors of classification associated with the contrasting groups procedures. Monte Carlo simulations were employed to produce…

Kansas law requires setting passing scores for the reading and mathematics competency test for 2nd, 4th, 6th, 8th, and 11th grade students, administered annually since April of 1980. New objective-referenced tests are prepared each year. Both judgmental (Angoff, Ebel, and Nedelsky) methods and empirical (contrasting groups and borderline) methods…

This study examined whether two variations on the typical Angoff group standard-setting process would produce sufficiently consistent results to recommend their use. The results imply that judgments gathered after an initial traditional group-process session can provide an efficient alternative mechanism for setting cutting scores using the Angoff…

Thirty-eight methods are presented for either setting standards or adjusting them based on an analysis of classification error rates. A trilevel classification scheme is used to categorize the methods, and 10 criteria of technical adequacy and practicability are proposed to evaluate them. (Author/LMO)

A new standard setting approach is introduced, called the cognitive components approach. Like the Angoff method, the cognitive components method generates minimum pass levels (MPLs) for each item. In both approaches, the item MPLs are summed for each judge, then averaged across judges to yield the standard. In the cognitive components approach,…

The use of latent class analysis for establishing student performance standards was studied. Latent class analysis (LCA) is an established procedure for investigating the latent structure of a set of data. LCA presumes that groups, classes, or respondents differ qualitatively from one another, and that these differences account for all of the…

Suggestions are provided for future research in item bias detection, reduction of essay-reader variation in setting cut-score levels, and limitations of equating theory. (TJH)

Ten factors relevant to the standard-setting decision on important tests (i.e., high school diploma tests) are proposed as a basis for a standard-setting preparation program. It is suggested that, typically, those who set standards have had little or no experience and would benefit from formal preparation. (SLD)

Issues concerning the selection of judges for standard setting are discussed. Determining the consistency of judges' recommendations, or their congruity with other expert recommendations, would help in selection. Enough judges must be chosen to allow estimation of recommendations by an entire population of judges. (SLD)

Training judges to generate item ratings in standard setting once the reference group has been defined is discussed. It is proposed that sensitivity to the factors that determine difficulty can be improved through training. Three criteria for determining when training is sufficient are offered. (SLD)

Debate about the levels-setting process for criterion-referenced tests of achievement is reviewed. An alternative framework is proposed for conceptualizing the problem and practice of setting performance standards on such tests. Standard setting is reconceptualized as the development, description, and implementation of reasonable rules for…

Describes two studies at Brigham Young University which investigated whether the Pre-Professional Skills Test can identify teacher education candidates who lack essential skills in reading, writing, and mathematics. Demonstrates that the test is relevant to the kinds of skills and knowledge that teacher education students need to succeed.…