Even though the impact of the position of response options on answers to multiple-choice items has been investigated for decades, it remains debated. Research on this topic is inconclusive, perhaps because too few studies have obtained experimental data from large-sized samples in a real-world context and have manipulated the position of both…

Recent research has shown how the statistical bias in Rasch model difficulty estimates induced by guessing in multiple-choice items can be eliminated. Using vertical scaling of a high-profile national reading test, it is shown that the dominant effect of removing such bias is a nonlinear change in the unit of scale across the continuum. The…

When missing values are present in item response data, there are a number of ways one might impute a correct or incorrect response to a multiple-choice item. There are significantly fewer methods for imputing the actual response option an examinee may have provided if he or she had not omitted the item either purposely or accidentally. This…

In aptitude and achievement tests, different response formats are usually used. A fundamental distinction must be made between the class of multiple-choice formats and the constructed response formats. Previous studies have examined the impact of different response formats applying traditional statistical approaches, but these influences can also…

There is much debate on the merits and pitfalls of standardized tests for college admission, with questions regarding the format (multiple-choice vs. constructed response), cognitive complexity, and content of these assessments (achievement vs. aptitude) at the forefront of the discussion. This study addressed these questions by investigating the…

Two factors, language difficulty and option set convergence, were experimentally manipulated and their effects on item difficulty assessed. Option convergence was found to have a significant effect on item difficulty while the effect of language difficulty was not significant. (Author/BW)

This study compared the item difficulty, item discrimination, and test reliability of three forms of multiple-choice items: (1) one correct answer; (2) "none of the above" as a foil; and (3) "none of the above" as the correct answer. Twelve items in the three formats were administered in a college statistics examination. (BS)

The effects of statistical and cognitive difficulty ordering of test items, gender, and explicit labeling of item type on college students' test performance were assessed. A total of 137 male and 265 female undergraduates were administered two 40-item multiple choice tests. Results indicate significant effects for gender and use of labels, but…

The effect of removing nonfunctioning items from multiple-choice tests was studied by examining change in difficulty, discrimination, and dimensionality. Results provide additional support for the benefits of eliminating nonfunctioning options, such as enhanced score reliability, reduced testing time, potential for broader domain sampling, and…

First year medical students answered parallel multiple-choice questions at different taxonomic levels as part of their diagnostic examinations. The results show that when content is held constant, students perform as well on interpretation and problem-solving questions as on recall questions. (Author/BW)

Two investigations involving 700 candidates for medical specialty certification suggest that test items with only 4 options perform as well as the same items with 5 options. Results also suggest that five-option multiple-choice items can be reduced to four-option items by removing a nonfunctioning item. (SLD)

A meta-analysis of the difficulty and discrimination of the "none-of-the-above" (NOTA) test option was conducted with 12 articles (20 effect sizes) for difficulty and 7 studies (11 effect sizes) for discrimination. Findings indicate that using the NOTA option does not result in items of lesser quality. (SLD)

This study was concerned with the reliability and validity of subjective judgments about five characteristics of multiple-choice test items from an introductory college-level astronomy test: (1) item difficulty, (2) language complexity, (3) content importance or relevance, (4) response set convergence, and (5) process complexity. (Author)

Multiple-choice tests composed entirely of two-, three-, or four-choice items were investigated. Results indicated that number of alternatives per item was inversely related to item difficulty, but directly related to item discrimination. Reliability and standard error of measurement of three-choice item tests was equivalent or superior.…

Performance of a common set of test items on an examination in which the order of options for one test form was experimentally manipulated. Results for 759 medical specialty board examinees find that reordering item options results in significant but unpredictable effects on item difficulty. (SLD)