A newly developed performance standard-setting procedure, termed iterative judgmental policy capturing (JPC), is applicable to assessments composed of distinct multidimensional exercises. The procedure is described, and results are reported from the application of JPC in a study involving a panel of 20 teachers and 6 performance exercises. (SLD)

A performance-standard setting procedure termed judgmental policy capturing (JPC) and its application are described. A study involving 12 panelists demonstrated the feasibility of the JPC method for setting performance standards for classroom teachers seeking certification from the National Board for Professional Teaching Standards. (SLD)

The implicit definition of competence and the inferential chain that links the standard-setting process to the decision outcomes of the method are considered for two classes of standard-setting procedures: those involving data-free judgments of items and those involving data-based judgment of items. The major underlying assumptions of competence…

The modified caution index's use in identifying judges whose patterns of item judgment appear aberrant when compared with the pattern produced by the entire group (N=158) of judges was studied. Effects on test standards and passing rates of removing test standards of these judges were also assessed. (TJH)

A standard-setting experiment was conducted in which one group of participants was instructed to refrain from discussing their recommended test standards and another group was requested to report the reasons underlying their recommendations. The procedures used were designed to induce convergence of recommendations, but did not require consensus.…

This article reviews developments in the field of educational measurement over the preceding 20 years, focusing on criterion-referenced measurement, item response theory, standard-setting methods, and equating methods. (TJH)

A structured Delphi procedure was used to secure the informed judgments of three types of adult participants (n=170) on appropriate standards for the North Carolina High School Competency Tests in Reading and Mathematics. The feasibility of the method and the amount of agreement between groups are discussed. (Author/PN)

The "Standards for Educational and Psychological Testing" (1985) and the "Uniform Guidelines on Employee Selection Procedures" (1978) provide no clear guidance on choosing methods for setting standards on teacher certification tests, although they require that practices be well-documented and produce reasonable results, and…

Issues concerning the selection of judges for standard setting are discussed. Determining the consistency of judges' recommendations, or their congruity with other expert recommendations, would help in selection. Enough judges must be chosen to allow estimation of recommendations by an entire population of judges. (SLD)

The effects of using recommended data collection procedures on median recommended test standards, variability of recommended test standards, and reliability of recommended standards for 7 subtests of the National Teacher Examinations Communications Skills and General Knowledge Tests were explored, using 236 evaluators (75 public school teachers…

This study examined the contribution of five categories of variables to the standard-setting recommendations of a representative sample of 241 judges recommending standards for the General Knowledge and Communication Skills Tests of the National Teacher Examinations (NTE) for the state of North Carolina. It was based on the assumption that…

Methods that might be used to establish standards of performance that will permit the National Board for Professional Teaching Standards to classify candidate teachers as highly accomplished (worthy of National Board certification) or less than highly accomplished (not worthy of certification) are contrasted. Plans for research on these issues are…

This study explores the use of the modified caution index (MCI) for identifying judges whose patterns of recommendations suggest that their judgments might be based on incomplete information, flawed reasoning, or inattention to their standard-setting tasks. It also examines the effect on test standards and passing rates when the test standards of…

In this final report of an 18-month series of projects designed to investigate the validity of the General Knowledge (GK) and Communications Skills (CS) tests of the National Teacher Examinations (NTE) for North Carolina, highlights and major conclusions are summarized and appropriate standards of performance are recommended. Evidence was…

Criteria for the selection of judges (evaluators) for setting item-based standards involved in tests for which cutting scores must be established are investigated. Focus is on cases in which test standards are based on specialists' judgments concerning the difficulty of test items in tests used to determine who will be awarded a diploma, admitted…