Possible sources of intrajudge inconsistency in standard setting are reviewed, and approaches are presented to improve the accuracy of rating. Procedures for providing judges with feedback through discussion or computerized communication are discussed. Monitoring and maintaining judges' consistency throughout the rating process are essential. (SLD)

An approach is presented to the definition of minimal competence for judges to use in standard setting. Panelists in standard setting must receive training to ensure that differences in rating result from differences in perceptions of item difficulty, not in differences of opinion about the definition of minimal competence. (SLD)

Training judges to generate item ratings in standard setting once the reference group has been defined is discussed. It is proposed that sensitivity to the factors that determine difficulty can be improved through training. Three criteria for determining when training is sufficient are offered. (SLD)

This document presents information to trainers already trained in the Registered Holistic scoring method. It has been developed to assist these trainers in training others in the procedures and the criteria of the new scoring method. The Registered Holistic scoring method was designed and field tested for the ninth grade writing sample component…

The impact of overall test length and difficulty on the expert judgments of item performance by the Nedelsky method were studied. Five university-level instructors predicting the performance of minimally competent candidates on a mathematics examination were fairly consistent in their assessments regardless of length or difficulty of the test.…

A technical advisory committee assisted the Alabama State Department of Education in establishing passing scores and equating test forms of the Alabama High School Graduation Examination (AHSGE). The committee examined four methods of setting cutting scores: (1) Jensen's theoretical method--probability theory is used to set standards that are…