A common procedure for obtaining multiple readings (ratings) for a constructed response item, especially in high-stakes tests, is to have two readers read the papers independently, with a third reading if the results differ by more than one point. This necessitates a scoring rule that specifies how the ratings will be aggregated into a single item…

Investigated how well the generalized and Rasch models described item and test performance across a broad range of mixed-item-format test configurations (six tests from two state proficiency testing programs). Evaluating the impact of model assumptions on the predictions of item and test information permitted a delineation of the implications of…

Evaluated the equivalence of scores and one-parameter logistic model item difficulty estimates obtained from computer-based and paper-and-pencil forms of a licensure examination taken by 418 examinees. There was no effect of either order or mode of administration on the equivalences. (SLD)

Equating multiple test forms is frequently desired. When multiple forms are linked in a chain of equating, error tends to build up in the process. This paper compares three procedures for equating multiple forms in a common-form design where each school administered, in a spiraled fashion, only a subset of multiple forms. Data used were from a…

This study investigated the effect on student scores of using anchor sets that differed in dimensionality in item response theory (IRT) scaled tests. Real data from a mathematics achievement test that had been documented to have dimensions aligned with item format were used. Item responses were available from a representative sample of…

Explanations for an observed change in Rasch item parameters ("b" values) from consecutive administrations of a professional licensing examination were investigated. Analysis of covariance indicated that the change was not related to item position or type. It is hypothesized that the change is attributable to shifts in curriculum…

A part-form methodology was used to study the effect of varying degrees of multidimensionality on the consistency of pass/fail classification decisions obtained from simulated unidimensional item response theory (IRT) based licensure examinations. A control on the degree of form multidimensionality permitted an assessment throughout the range of…

Whether the presence of bidimensionality has any effect on the adaptive recalibration of test items was studied through live-data simulation of computer adaptive testing (CAT) forms. The source data were examinee responses to the 298 scored multiple choice items of a licensure examination in a health care profession. Three 75-item part-forms,…

To investigate the psychometric feasibility of replacing a paper-and-pencil licensing examination with a computer-administered test, a validity study was conducted. The computer-administered test (Cadm) was a common set of items for all test takers, distinct from computerized adaptive testing, in which test takers receive items appropriate to…

The purpose of this research was to study the effect of the three different ways of increasing the number of points contributed by constructed response (CR) items on the reliability of test scores from mixed-item-format tests. The assumption of unidimensionality that underlies the accuracy of item response theory model-based standard error…

The sources of multidimensionality found in several different forms of a licensure examination were studied. The relationship between one source of multidimensionality, differential item functioning (DIF) (or factors producing DIF), and content characteristics was explored in an attempt to isolate aspects of training or curriculum that could…