The continuous method of scoring a performance test composed of standardized patients was compared with a derivative method that assigned each of the 131 examinees (medical residents) a dichotomous score, and use of Angoff's method with these scoring methods was studied. Both methods produce reasonable means and distributions of scores. (SLD)

Examines the appropriateness of a cut-off score derived from the Angoff procedure for a reading test in the United Kingdom. Shows that the recommended cut-off score is too low. Suggests ways that standard setting might draw on a range of information to produce appropriate and rationally defensible cut-off scores. (DSK)

This paper reports on a newly designed judgmental method for setting test performance standard that: (1) overcome many of the practical and psychometric problems associated with the Angoff and Ebel methods; (2) can be used to set multiple cut points on a score scale; (3) may be readily and efficiently implemented with assessments that use…

Understanding the behavior of those evaluating a speech is important for a complete understanding of the public communication process. Ratings of public speaking were submitted to a Rasch analysis to determine whether objective measurement can create and maintain a standard of speech evaluation. Data used were 1,022 ratings of 168 speeches given…

This study used generalizability theory and other statistical concepts to assess the application of the Angoff method to setting cutoff scores on two professional certification tests. A panel of ten judges gave pre- and post-feedback Angoff probability ratings of items of two forms of a professional certification test, and another panel of nine…

This study was conducted to: (1) determine the validity of five Educational Testing Service (ETS) tests as measures of the knowledge and academic skills required for specific endorsements of professional public school personnel in Arkansas; and (2) formulate recommendations on minimum qualifying scores for tests that are valid to use for…

In light of the widespread use of competency testing, the authors consider that it is important to determine ways of developing and using competency testing to insure that it achieves its full potential. The paper, in three parts, introduces a model for the development and validation of competency tests, reviews several methods for setting…

An experiment was designed that varied cutting score procedures, instructions, and types of judges in order to address the following questions concerning the Real Estate Licensing Examination: (1) Will the cutting score levels produced by groups of judges from differing backgrounds (academicians vs. practitioners vs. lawyers) using the same method…

Based on the standard setting procedures or the American Board of Preventive Medicine for their Core Test, this article describes how Rasch measurement can facilitate using test content judgments in setting a standard. Rasch measurement can then be used to evaluate and improve the precision of the standard and to hold it constant across time.…

Before using the National Teacher Examinations (NTE) for teacher certification, states are required to conduct a state-wide validity study. This paper describes approaches used in 35 studies for 18 states to establish (NTE) content validity through curriculum and job relatedness and to establish minimum performance standards. (BS)

This study evaluated procedures for establishing a minimum performance standard for the essay subtest of the National Teacher Examinations Communications Skills test. Results indicated the preferred procedure for setting standards on essays should involve a blind review followed by an informed review. (Author/DWH)

In the current climate of accountability in American public education, tests get more attention and carry more importance than ever before. Both state accountability systems and the federal No Child Left Behind Act (NCLB) hold schools accountable for whether students pass standardized state tests. NCLB requires that schools and school districts…

Although arbitrary, whenever multiple judgmental standard-setting procedures are utilized by different groups concurrently, stability across raters can be achieved and decisions can be made in a relatively judicious manner. Greater stability across methods (Ebel, Nedelsky, Angoff) may be effected by slightly modifying the Ebel approach. (Author/PN)

A socially moderated link was established between statewide achievement results and the National Assessment of Educational Progress (NAEP) by using the same achievement level descriptions in an Iowa Test of Basic Skills standard-setting and an NAEP standard setting study. A statistically moderated link was established through an equipercentile…

A brief history of developments in the assessment of the reliability of credentialing examinations is presented, and some new results are outlined that highlight the interactions among scoring, standard setting, and the reliability and validity of pass-fail decisions. Decision consistency is an important concept in evaluating credentialing…