Teachers who can adapt and be ready for all changes will also be able to provide a balance to increase the competence of vocational high school students. This is also not denied when teachers become assessors in student competency tests. The objectives of this study were to produce an instrument for the readiness of teachers as assessors; to…

This article presents the results of a survey conducted in an attempt to examine evaluators' perceptions of the contributions that different educational experiences make to the development of their evaluation skill sets. Self-identified new and graduate student evaluators were asked to describe their evaluation training experiences and to indicate…

Policy-making problems being faced in Louisiana regarding the testing of handicapped students within a state mandated minimum competency testing program are dealt with. The decision-making process is complicated by the fact that two groups (staff administering the test and staff administering programs for the handicapped) with differing…

Eleven judges representing 11 different geographic regions in the United States participated in a standard-setting session designed to determine the possibility of obtaining interjudge consensus and intrajudge consistency simultaneously. Each judge had experience in the field for which standards were being set. The judges rated 65 multiple-choice…

The modified caution index's use in identifying judges whose patterns of item judgment appear aberrant when compared with the pattern produced by the entire group (N=158) of judges was studied. Effects on test standards and passing rates of removing test standards of these judges were also assessed. (TJH)

Possible sources of intrajudge inconsistency in standard setting are reviewed, and approaches are presented to improve the accuracy of rating. Procedures for providing judges with feedback through discussion or computerized communication are discussed. Monitoring and maintaining judges' consistency throughout the rating process are essential. (SLD)

The present study examines the influence of judges' item-related knowledge on setting standards for competency tests. Seventeen judges from different professions took a 122-item teacher-certification test in economics while setting competency standards for the test using the Angoff procedure. Judges tended to set higher standards for items they…

An approach is presented to the definition of minimal competence for judges to use in standard setting. Panelists in standard setting must receive training to ensure that differences in rating result from differences in perceptions of item difficulty, not in differences of opinion about the definition of minimal competence. (SLD)

Training judges to generate item ratings in standard setting once the reference group has been defined is discussed. It is proposed that sensitivity to the factors that determine difficulty can be improved through training. Three criteria for determining when training is sufficient are offered. (SLD)

This document presents information to trainers already trained in the Registered Holistic scoring method. It has been developed to assist these trainers in training others in the procedures and the criteria of the new scoring method. The Registered Holistic scoring method was designed and field tested for the ninth grade writing sample component…

The Palos Verdes Peninsula Unified School District Office of Program Evaluation and Research is responsible for providing information for program development and improvement; providing test information to special programs coordinators; and acting as a clearinghouse for all information concerning tests, evaluation methodology, and educational…

The impact of overall test length and difficulty on the expert judgments of item performance by the Nedelsky method were studied. Five university-level instructors predicting the performance of minimally competent candidates on a mathematics examination were fairly consistent in their assessments regardless of length or difficulty of the test.…

The writing examination package is divided into three volumes: Examination Handbook (present volume), Training Manual, and a Ratings Manual. The volumes are interdependent, and all three must be understood before using the Training Manual to train raters. The Examination Handbook provides a context for the other two volumes. It gives a general…

A technical advisory committee assisted the Alabama State Department of Education in establishing passing scores and equating test forms of the Alabama High School Graduation Examination (AHSGE). The committee examined four methods of setting cutting scores: (1) Jensen's theoretical method--probability theory is used to set standards that are…

Criteria for the selection of judges (evaluators) for setting item-based standards involved in tests for which cutting scores must be established are investigated. Focus is on cases in which test standards are based on specialists' judgments concerning the difficulty of test items in tests used to determine who will be awarded a diploma, admitted…