Reviewed methods of estimating theta suitable for computerized adaptive testing (CAT) and discussed the differences between Fisher and Kullback-Leibler information criteria for selecting items. Examined the accuracy of different CAT algorithms using samples from the National Assessment of Educational Progress. Results show when correcting for…

The purpose of this Monte Carlo study was to evaluate the relative accuracy of T. Warm's weighted likelihood estimate (WLE) compared to maximum likelihood estimate (MLE), expected a posteriori estimate (EAP), and maximum a posteriori estimate (MAP), using the generalized partial credit model (GPCM) and graded response model (GRM) under a variety…

Item scores that do not fit an assumed item response theory model may cause the latent trait value to be estimated inaccurately. Several person-fit statistics for detecting nonfitting score patterns for paper-and-pencil tests have been proposed. In the context of computerized adaptive tests (CAT), the use of person-fit analysis has hardly been…

Two methods for estimating the reliability of a computerized adaptive test (CAT) without using item response theory are presented. The data consist of CAT and paper-and-pencil scores from identical or equivalent samples, and scores for all examinees on one or more covariates, using the Armed Services Vocational Aptitude Battery. (TJH)

Performed a simulation study to demonstrate the dramatic impact of capitalization on estimation errors on ability estimation in adaptive testing. Discusses four different strategies to minimize the likelihood of capitalization in computerized adaptive testing. (SLD)

Test security has often been a problem in computerized adaptive testing (CAT) because the traditional wisdom of item selection overly exposes high discrimination items. The a-stratified (STR) design advocated by H. Chang and his collaborators, which uses items of less discrimination in earlier stages of testing, has been shown to be very…

In the a-stratified method, a popular and efficient item exposure control strategy proposed by H. Chang (H. Chang and Z. Ying, 1999; K. Hau and H. Chang, 2001) for computerized adaptive testing (CAT), the item pool and item selection process has usually been divided into four strata and the corresponding four stages. In a series of simulation…

A study was conducted to investigate whether augmenting the calibration of items using computerized adaptive test (CAT) data matrices produced estimates that were unbiased and improved the stability of existing item parameter estimates. Item parameter estimates from four pools of items constructed for operational use were used in the study to…

Proposes a model for constrained computerized adaptive testing in which the information in the test at the trait level (theta) estimate is maximized subject to the number of possible constraints on the content of the test. Test assembly relies on a linear-programming approach. Illustrates the approach through simulation with items from the Law…

Compared five item selection rules with respect to the efficiency and precision of trait (theta) estimation at the early stages of computerized adaptive testing (CAT). The Fisher interval information, Fisher information with a posterior distribution, Kullback-Leibler information, and Kullback-Leibler information with a posterior distribution…

In the process of item calibration for a computerized adaptive test (CAT), many well-established calibrating packages show weakness in the estimation of item parameters. This paper introduces an on-line calibration algorithm based on the convexity of likelihood functions. This package consists of: (1) an algorithm that estimates examinee ability…

In a computerized adaptive test (CAT), it would be desirable to obtain an acceptable precision of the proficiency level estimate using an optimal number of items. Decreasing the number of items is accompanied, however, by a certain degree of bias when the true proficiency level differs significantly from the a priori estimate. G. Raiche (2000) has…

This paper suggests several item selection criteria for adaptive testing that are all based on the use of the true posterior. Some of the ability estimators produced by these criteria are discussed and empirically criticized. (SLD)

A simulation study was conducted to investigate the application of expected a posteriori (EAP) trait estimation in computerized adaptive tests (CAT) based on the partial credit model and compare it with maximum likelihood estimation (MLE). Results show the conditions under which EAP and MLE provide relatively accurate estimation in CAT. (SLD)

The purposes of this research study were to develop and field test anchor-based judgmental methods for enabling test specialists to estimate item difficulty statistics. The study consisted of three related field tests. In each, researchers worked with six Law School Admission Test (LSAT) test specialists and one or more of the LSAT subtests. The…