Issues concerning the selection of judges for standard setting are discussed. Determining the consistency of judges' recommendations, or their congruity with other expert recommendations, would help in selection. Enough judges must be chosen to allow estimation of recommendations by an entire population of judges. (SLD)

This study explores the use of the modified caution index (MCI) for identifying judges whose patterns of recommendations suggest that their judgments might be based on incomplete information, flawed reasoning, or inattention to their standard-setting tasks. It also examines the effect on test standards and passing rates when the test standards of…

A performance-standard setting procedure termed judgmental policy capturing (JPC) and its application are described. A study involving 12 panelists demonstrated the feasibility of the JPC method for setting performance standards for classroom teachers seeking certification from the National Board for Professional Teaching Standards. (SLD)

The implicit definition of competence and the inferential chain that links the standard-setting process to the decision outcomes of the method are considered for two classes of standard-setting procedures: those involving data-free judgments of items and those involving data-based judgment of items. The major underlying assumptions of competence…

A structured Delphi procedure was used to secure the informed judgments of three types of adult participants (n=170) on appropriate standards for the North Carolina High School Competency Tests in Reading and Mathematics. The feasibility of the method and the amount of agreement between groups are discussed. (Author/PN)

This study addressed seven questions regarding the methods used in setting passing scores on the essay subtest of the National Teacher Examinations (NTE) Communication Skills test for the North Carolina State Board of Education. North Carolina uses these tests to screen prospective applicants to teacher education programs. The judges (five college…

Issues in collecting data on the classroom performance of teachers are discussed, with a focus on written descriptions of teacher behavior in contrast with videotaped observations. The central issue in this comparison is the comparability of the information when it is observed live and when it is simply recorded on videotape. The fundamental…