Two studies of medical certification examinations were undertaken to assess standard setting using Angoff's Method. Results indicate that (1) specialization within broad content areas does not affect an expert's estimates of the performance of the borderline group; and (2) performance data should be provided during the standard-setting process.…

This study retrospectively applies aggregate scoring to the cardiology questions from a recertifying examination in general internal medicine. Three groups of experts were identified among the examinees and the remainder of the examinees were the participants (N=1887) of the study. Generalizability theory is applied to these data. (MLW)

The major forms of evidence that support a standard's credibility are reviewed, and what can be done over time and for different forms of an examination to enhance its comparability in a credentialing setting is outlined. Pass-fail decisions must be consistent to ensure a standard's credibility. (SLD)

Scores from four samples each of 250 and 1,000 physicians demonstrated that a linear procedure for equating scores and rescaling judges' standards for a certification test could be applied to individual item data gathered through the Angoff standard-setting method. Equated and rescaled values were close to those actually assigned. (SLD)

An agenda for research on standards for professional licensure and certification examinations is outlined. The relationship between scores and standards is explored, and major classes of standards are described. Topics for future research are discussed in the areas of judgment basis, process efficiency, group effects, and content and expertise…

This study compares physician performance on the Computer-Aided Simulation of the Clinical Encounter with peer ratings and performance on multiple choice questions and patient management problems. Results indicate that all formats are equally valid, although multiple choice is the most reliable method of assessment per unit of testing time.…

Whether cutting score equivalents (CSEs) based on examinee performance are the same as CSEs based on expert judgment was examined using data from 3,262 examinees taking an internal medicine certification examination. CSEs produced by 40 physicians/experts were closer to the criteria than were standards derived from examinee performance. (SLD)