This paper outlines a technique for differentially weighting options of a multiple choice test in a fashion that maximizes the item predictive validity. The rule can be applied with different number of categories and the "optimal" number of categories can be determined by significance tests and/or through the R2 criterion. Our theoretical analysis…

For (0, 1) scored multiple-choice tests, a formula giving test reliability as a function of the number of item options is derived, assuming the "knowledge or random guessing model," the parallelism of the new and old tests (apart from the guessing probability), and the assumptions of classical test theory. It is shown that the formula is a more…

In the past, several latent structure models have been proposed for handling problems associated with measuring the achievement of examinees. Typically, however, these models describe a specific examinee in terms of an item domain or they describe a few items in terms of a population of examinees. In this paper, a model is proposed which allows a…

Multilinear formula scoring (MFS) is reviewed, with emphasis on estimating option characteristic curves (OCSs). MFS was used to estimate OCSs for the arithmetic reasoning subtest of the Armed Services Vocational Aptitude Battery for 2,978 examinees. A second analysis obtained OCSs for simulated data. The use of MFS is discussed. (SLD)

One means of learning about the processes operating in a multiple choice test is to include some test items, called nonsense items, which have no correct answer. This paper compares two versions of a mathematical model of test performance to interpret test data that includes both genuine and nonsense items. One formula is based on the usual…

These studies in test adequacy focus on two problems: procedures for estimating reliability, and techniques for identifying ineffective distractors. Fourteen papers are presented on recent advances in measuring achievement (a response to Molenaar); "an extension of the Dirichlet-multinomial model that allows true score and guessing to be…

Current Scoring practices for multiple-choice tests are rooted in early Associationist Theory and are based on a two-step procedure: (1) right answers counted as ones and wrong answers are zeros, and (2) number of right answers form a total-correct score. The author contends that if either step is invalid, the use of the general linear model (GLM)…

During 1969 the International Association for the Evaluation of Educational Achievement began a series of cross-cultural studies to investigate the workings of multiple-choice achievement tests and student guessing behaviors. Empirical models to correct for guessing are discussed in terms of test item difficulty, number of response choices,…

This paper deals with test fairness regarding a test consisting of two parts: (1) a "common" section, taken by all students; and (2) a "variable" section, in which some students may answer a different set of questions from other students. For example, a test taken by several thousand students each year contains a common multiple-choice portion and…

Formula score theory (FST) associates each multiple choice test with a linear operator and expresses all of the real functions of item response theory as linear combinations of the operator's eigenfunctions. Hard measurement problems can then often be reformulated as easier, standard mathematical problems. For example, the problem of estimating…