Alternatives to the standard statewide assessment often are necessary for valid measurement of students with significant disabilities. These alternate assessments must be carefully developed and evaluated with respect to generally accepted psychometric standards. Ideally, these measures should be sensitive to growth and scaled to state tests that…

The Bologna Declaration brought reforms into higher education that imply changes in teaching methods, didactic materials and textbooks, infrastructures and laboratories, etc. Statistics and mathematics are disciplines that traditionally have the worst success rates, particularly in non-mathematics core curricula courses. This research project,…

A special case of examinee choice, the Optional Essay Problem, is examined from the point of view of test equating. The Optional Essay Problem involves equating essay scores when the examinees are required to select an optional essay topic from a list of topics in addition to taking a mandatory test required of all examinees. The conditions that…

The detection of differential item functioning (DIF) has become an important psychometric research topic in recent years. A number of item response theory (IRT) methods for solving this problem have been suggested. A common approach is to calculate some function of the area between item response curves estimated from the subpopulations of…

At the Educational Testing Service, the Mantel-Haenszel procedure is used for differential item functioning (DIF) detection, and the standardization procedure is used to describe DIF. This report describes these procedures. First, an important distinction is made between DIF and impact, pointing to the need to compare the comparable. Then, these…

The Johnson O'Connor Research Foundation, which produces vocabulary instructional materials for test takers, is in the process of determining the difficulty values of nontechnical words in the English language. To this end, the Foundation writes test items for vocabulary words and tests them in schools. The items are then calibrated using the…

This study examined to what extent, if any, estimates of reliability for a multiple choice test are affected by the presence of large item sets where each set shares common reading material. The purpose of this research was to assess the effect of local item dependence on estimates of reliability for verbal portions of seven forms of the old and…

In this study, the ability of test development staff to predict the difficulty of analogy items was explored. The nature of the item attributes that contributed to test writers' predictions of difficulty as well as actual item difficulty was also investigated. The two expert test writers studied were quite good at predicting item difficulty. Item…

Two different methods of obtaining three parameter logistic item response theory (IRT) pretest item parameter estimated for the Graduate Management Admissions Testing Program. The first method consisted of calibrating pretest and operational items simultaneously in a LOGIST run, that is a concurrent calibration design. The second approach entailed…

Marks made by students on test item booklets were analyzed as a clue to better understanding of the metacognitive strategies employed during the completion of a 100-question multiple-choice final examination. Test item booklets of 56 undergraduates were scrutinized for the frequency of the following item markings; (1) no markings at all; (2)…

This study compares logistic discriminant function analysis for differential item functioning (DIF) with a technique for the detection of DIF that is based on item response theory rather than the Mantel-Haenszel procedure. In this study, the areas between the two item characteristic curves, also called the item characteristic curve method is…

The effects of anchor test characteristics in the accuracy and precision of test equating in the "common items, nonequivalent groups design" were studied. The study also considered the effects of nonparallel based and new forms on the equating solution, and it investigated the effects of differential weighting on the success of equating…

An integer programming approach to item pool design is presented that can be used to calculate an optimal blueprint for an item pool to support an existing testing program. The results are optimal in the sense that they minimize the efforts involved in actually producing the items as revealed by current item writing patterns. Also, an adaptation…

A version of sequential mastery testing is studied in which response behavior is modeled by an item response theory (IRT) model. First, a general theoretical framework is sketched that is based on a combination of Bayesian sequential decision theory and item response theory. A discussion follows on how IRT based sequential mastery testing can be…

The capability of the DIMTEST statistical test to assess essential dimensionality of the model underlying item responses of real tests as opposed to simulated tests was investigated. A variety of real test data from difference sources was used to assess essential dimensionality. Based on DIMTEST results, some test data are assessed as fitting an…