The purpose of this study was to compare and evaluate five online pretest item calibration/scaling methods in computerized adaptive testing (CAT): (1) the marginal maximum likelihood estimate with one-EM cycle (OEM); (2) the marginal maximum likelihood estimate with multiple EM cycles (MEM); (3) Stocking's Method A (M. Stocking, 1988); (4)…

Reviews research literature on comparability issues in computerized adaptive testing (CAT) and synthesizes issues specific to comparability and test security. Develops a framework for evaluating comparability that contains three categories of criteria: (1) validity; (2) psychometric property/reliability; and (3) statistical assumption/test…

The interest in the application of large-scale computerized adaptive testing has served to focus attention on issues that arise when theoretical advances are made operational. Some of these issues stem less from changes in testing conditions and more from changes in testing paradigms. One such issue is that of the order in which questions are…

Two-stage and multistage test designs provide a way of roughly adapting item difficulty to test-taker ability. All test takers take a parallel stage-one test, and, based on their scores, they are routed to tests of different difficulty levels in subsequent stages. These designs provide some of the benefits of standard computerized adaptive testing…

Computerized adaptive testing procedures (CATPs) based on the graded response method (GRM) of F. Samejima (1969) and the partial credit model (PCM) of G. Masters (1982) were developed and compared. Both programs used maximum likelihood estimation of ability, and item selection was conducted on the basis of information. Two simulated data sets, one…

Methods to control the test construct and the efficiency of a computerized adaptive test (CAT) were studied in the context of a reading comprehension test given as a part of a battery of tests for college admission. A goal of the study was to create test scores that were interchangeable with those from a fixed form paper and pencil test. The first…

The effects of a fixed and variable entry procedure on bias and information of a Bayesian adaptive test were compared. Neither procedure produced biased ability estimates on the average. Bias at the distribution extremes, efficiency curves, item subsets generated for administration, and items required to reach termination are discussed. (TJH)

A college administered the computer-adaptive ACCUPLACER (College Board, 1995) reading placement test to 399 entering students and its paper-and-pencil version, COMPANION, to 481 students. When the age of the two groups was held constant, no differences were found between the groups. (SLD)

This paper presents a strategy for measuring cognitive gains in reading during the early school years. It is argued that accurate estimates of gain scores and their appropriate interpretation requires the use of adaptive tests with multiple criterion referenced points that mark learning milestones. It is further argued that two different measures…

A study involving 209 Spanish high school students compared computer-based English vocabulary tests: (1) a self-adapted test (SAT); (2) a computerized adaptive test (CAT); (3) a conventional test; and (4) a test combining SAT and CAT. No statistically significant differences were found among test types for estimated ability or posttest anxiety.…

Findings of studies examining the effects of presentation media on item characteristics are reviewed. The effect of medium of presentation independent of adaptive methodology in computer-assisted testing was studied through tests of 763 Marine trainees. Conditions necessary for score equivalence between item presentation media are described. (SLD)

F. M. Lord's flexilevel, computerized adaptive testing (CAT) procedure was compared to an item-response theory-based CAT procedure that uses Bayesian ability estimation with various standard errors of estimates used for terminating the test. Ability estimates of flexilevel CATs were as accurate as were those of Bayesian CATs. (TJH)

Discussion of expert systems and computerized adaptive tests describes two versions of EXSPRT, a new approach that combines uncertain inference in expert systems with sequential probability ratio test (SPRT) stopping rules. Results of two studies comparing EXSPRT to adaptive mastery testing based on item response theory and SPRT approaches are…

A study involving 1,000 simulated examinees compared the partial credit and graded response models in computerized adaptive testing (CAT). The graded response model fit the data well and provided slightly more accurate ability estimates than those of the partial credit model. Benefits of polytomous model-based CATs are discussed. (SLD)

Using one simulated and two real data sets, the effects of the systematic variation of the item-selection procedure and the stepsize method on the operating characteristics of computerized adaptive testing (CAT) for instruments with polychotomously scored rating scale items were studied. The six rating scale CAT procedures used performed well.…