A multiple choice test item is identified as flawed if it has no single best answer. In spite of extensive quality control procedures, the administration of flawed items to test-takers is inevitable. Common strategies for dealing with flawed items in conventional testing, grounded in the principle of fairness to test-takers, are reexamined in the…

A study was undertaken to determine the effects on essay scores of intermingling handwritten and word-processed versions of student essays. A sample of 32 examinees, each of whom had produced both a handwritten and a word-processed essay, was drawn from a larger group who had participated in a pilot study of item types for The Praxis Series:…

The purpose of this study was to determine whether it is both possible and cost-effective to revise middle-difficulty Graduate Record Examinations (GRE) discrete items in order to produce items of higher or lower difficulty. The basic procedure was to select items of a given difficulty and, by revising the distractors, make them easier or more…

The interest in the application of large-scale computerized adaptive testing has served to focus attention on issues that arise when theoretical advances are made operational. Some of these issues stem less from changes in testing conditions and more from changes in testing paradigms. One such issue is that of the order in which questions are…

Generative response modeling is an approach to test development and response modeling that calls for the creation of items in such a way that the parameters of the items on some response model can be anticipated through knowledge of the psychological processes and knowledge required to respond to the item. That is, the computer would not merely…

The accuracy of procedures that are used to compare the performance of different groups of examinees on test items obviously depends on the correct classification of members in each examinee group. The significance of this dependence is determined by the sensitivity of the statistical procedure and the proportion of examinees who are unidentified.…

A study compared the influence of two forms of discourse completion test (DCT) on the elicitation of rejection of advice. An open questionnaire providing scenarios alone was compared with a classic dialogue completion task in which a conversational turn is provided. The tasks were given to 32 graduate students, 19 native and 13 non-native…

The first of two studies reported examined the factors that predict differences in item responses for black and white matched examinees to analogies on the Graduate Record Examinations (GRE). Data were taken from 13 forms of the GRE Verbal Test, with a median sample size of 21,000 whites and a median sample size of 1,400 blacks for the purpose of…

In an attempt to determine the level of questioning according to Bloom's taxonomy, generated by test items that accompany selected fifth-grade science textbooks, an analysis of the tests provided by three of the most commonly used elementary science textbooks was completed. Additionally this analysis examined the extent to which each series…

Errors in item parameter estimates have a negative impact on the accuracy of item and test information functions. The estimation errors may be random, but because items with higher levels of discriminating power are more likely to be selected for a test, and these items are most apt to contain positive errors, the result is that item information…

Two studies investigated factors affecting the scores of Hawaii students taking the verbal subtest of the Scholastic Aptitude Test (SAT). For the past several years, the mean verbal scores of Hawaii students have consistently been among the lowest 10% of all states. The first study addressed the identification of items and types of items that have…

In the context of Phase Four of the National Assessment of Educational Progress (NAEP) Science Attribute Study, this report includes a discussion of item attributes, an overview of the item attributes used in the study, some psychometric characteristics of the blocks analyzed, a general description of the rule-space methodology, the results…

Often, educational and psychological measurement instruments must be translated from one language to another when they are administered to different cultural groups. The translation process often necessarily introduces measurement inequivalence. Therefore, an examination may be said to exhibit differential functioning if the test provides a…

In computerized adaptive testing, updating parameter estimates using adaptive testing data is often called online calibration. In this paper, how to evaluate whether the adaptive testing model used for online calibration fits the item response model used sufficiently is studied. Three approaches are investigated, based on a Lagrange multiplier…

R. J. Owen (1975) proposed an approximate empirical Bayes procedure for item selection in adaptive testing. The procedure replaces the true posterior by a normal approximation with closed-form expressions for its first two moments. This approximation was necessary to minimize the computational complexity involved in a fully Bayesian approach, but…