Potential solutions to applied measurement problems are commonly evaluated using simulated data, largely because it is often impossible, difficult, or prohibitively expensive to supply the needed volumes of real data. However, answers proved by simulation studied generalize real-world problems only to the extent that the simulated data resembles…

This study investigated the number-correct scoring method based on different theories (classical true-score theory and multidimensional item response theory) when a standardized test requires more than one ability for an examinee to get a correct response. The number-correct scoring procedure that is widely used is the one that is defined in…

In this study, the effect of appended pretesting was evaluated with regard to item statistics and examinee scores for groups of items that were pretested as part of a large-scale operational testing program. In appended pretesting, items are administered in a separately timed section at the end of an operational test battery. Two evaluations were…

Previous studies that investigated the effect of unequal ability distributions on the Type I error (TIE) of the Mantel-Haenszel chi-square test for detecting differential item functioning (DIF) simulated ability distributions that differed only in means. This simulation study suggests that the magnitude of TIE inflation is increased, and the type…

A mathematical programming approach is presented for computer adaptive testing (CAT) with many constraints on the item and test attributes. Because mathematical programming problems have to be solved while the examinee waits for the next item, a fast implementation of the Branch-and-Bound algorithm is needed for this approach. Eight modifications…

Minimum trace factor analysis has been used to find the greatest lower bound to reliability. This technique, however, fails to be scale free. A solution to the scale problem is proposed through the maximization of the greatest lower bound as the function of weights. (Author/JKS)

An important relationship is given for two generalizations of coefficient alpha: (1) Rajaratnam, Cronbach, and Gleser's generalizability formula for stratified-parallel tests, and (2) Raju's coefficient beta. (Author/CTM)

Two alternative procedures to Rogers' method of using control charts to display item statistics are discussed. The data itself determines limit and centerline values, thus permitting these values to be compared to any criterion difficulty level(s) deemed appropriate for a given set of test items. (Author/RL)

Describes a study investigating the comparative effects of objective (multiple choice and true-false) and subjective (short answer essay) quiz items on student academic performance in a personalized system of instruction course. Results showed no substantial differences between the two experimental groups. (Author/CK)

Methods for evaluating the consistency of responses to test items were compared. When a researcher is unwilling to make the assumptions of classical test theory, has only a small number of items, or is in a tailored testing context, Cliff's dominance indices may be useful. (Author/CTM)

Some practical guidelines for developing multiple choice tests are offered. Included are three steps: (1) test design; (2) proper construction of test items; and (3) item analysis and evaluation. (JMF)

Examines four factors involved in the determination of how many test items should be constructed or sampled for a set of objectives: (1) the type of decision to be made with results, (2) importance of objectives, (3) number of objectives, and (4) practical constraints. Specific guidelines that teachers and evaluators can use and an illustrative…

Differential item functioning (DIF) assessment procedures for items with more than two ordered score categories, referred to as polytomous items, were evaluated. Three descriptive statistics (standardized mean difference and two procedures based on the SIBTEST computer program) and five inferential procedures were used. Conditions under which the…

Recent simulations have shown that, for a given sample size, the Mantel-Haenszel (MH) variances tend to be larger when items are administered to randomly selected examinees than when they are administered adaptively. Results suggest that adaptive testing may lead to more efficient application of MH differential item functioning analyses. (SLD)