Two investigations involving 700 candidates for medical specialty certification suggest that test items with only 4 options perform as well as the same items with 5 options. Results also suggest that five-option multiple-choice items can be reduced to four-option items by removing a nonfunctioning item. (SLD)

A group-process approach to standard setting was compared to an independent approach for a medical specialty certification examination. Both approaches used the Angoff (1971) standard-setting method. In the group-process method, reviewers discussed items and their ratings during the rating process; in the independent condition, reviewers provided…

Results from 627 examinees taking a health sciences certification examination suggest that test items written purposefully to assess higher order cognitive skills do not provide evidence of assessing different levels of cognitive processing. Results do not support continuing use of a hierarchical cognitive classification dimension for the test…

Performance of a common set of test items on an examination in which the order of options for one test form was experimentally manipulated. Results for 759 medical specialty board examinees find that reordering item options results in significant but unpredictable effects on item difficulty. (SLD)

A commonly accepted rule for developing equated examinations using the common-items non-equivalent groups (CINEG) design is that items common to the two examinations being equated should be identical. The CINEG design calls for two groups of examinees to respond to a set of common items that is included in two examinations. In practice, this rule…

This National Council on Measurement in Education module describes standard setting for achievement measures used in education, licensure, and certification. The purposes, design, and conduct of standard-setting procedures are reviewed, and ways to calculate a cutting score are demonstrated. A self-test and annotated bibliography are included.…