Computerized adaptive testing (CAT) and multistage testing (MST) have become two of the most popular modes in large-scale computer-based sequential testing. Though most designs of CAT and MST exhibit strength and weakness in recent large-scale implementations, there is no simple answer to the question of which design is better because different…

This article extends the Bonett (2003a) approach to testing the equality of alpha coefficients from two independent samples to the case of m [greater than or equal] 2 independent samples. The extended Fisher-Bonett test and its competitor, the Hakstian-Whalen (1976) test, are illustrated with numerical examples of both hypothesis testing and power…

A computer simulation study was conducted to determine the feasibility of using logistic regression procedures to detect differential item functioning (DIF) in polytomous items. Results indicate that logistic regression is powerful in detecting most forms of DIF, although it requires large amounts of data manipulation and careful interpretation.…

The effect of violating the assumption of nonspeededness on ability and item parameter estimates in item response theory was studied through simulation under three speededness conditions. Results indicate that ability estimation was least affected by speededness but that substantial effects on item parameter estimates were found. (SLD)

Exact testing was evaluated as a method for conducting Mantel-Haenszel differential item functioning (DIF) analyses with relatively small samples. A series of computer simulations found that the asymptotic Mantel-Haenszel and the exact method yielded very similar results across sample size, levels of DIF, and data sets. (SLD)

The goal of this study was the development of a procedure to predict the equating error associated with the long-term equating method of Tate (2003) for mixed-format tests. An expression for the determination of the error of an equating based on multiple links using the error for the component links was derived and illustrated with simulated data.…

Parameter recovery in the graded-response model was investigated using the MULTILOG computer program under default conditions. Results from 36 simulated data sets suggest that at least 500 examinees are needed to achieve adequate calibration under the graded model. Sample size had little influence on the true ability parameter's recovery. (SLD)

It is demonstrated, theoretically and empirically, that item sets can be selected that meet the unidimensionality assumption of most item response theory models, even though they require more than one ability for a correct response. A method for identifying such item sets for test development purposes is presented. (SLD)

In a simulation study of ability and estimation of differential item functioning (DIF) in computerized adaptive tests, Rasch-based DIF statistics were highly correlated with generating DIF, but DIF statistics tended to be slightly smaller than in the three-parameter logistic model analyses. (SLD)

Robustness of the school-level item response theoretic (IRT) model to violations of distributional assumptions was studied in a computer simulation. In situations where school-level precision might be acceptable for real school comparisons, expected a posteriori estimates of school ability were robust over a range of violations and conditions.…

The phenomenon of simultaneous differential item functioning (DIF) amplification and cancellation and the role of the SIBTEST approach in detecting DIF are investigated with a variety of simulated test data. The effectiveness of SIBTEST is supported, and the implications of DIF amplification and cancellation are discussed. (SLD)

A logistic regression model for characterizing differential item functioning (DIF) between two groups is presented. A distinction is drawn between uniform and nonuniform DIF in terms of model parameters. A statistic for testing the hypotheses of no DIF is developed, and simulation studies compare it with the Mantel-Haenszel procedure. (Author/TJH)

Item parameter estimation errors in test development are highlighted. The problem is illustrated with several simulated data sets, and a conservative solution is offered for addressing the problem in item response theory test development practice. Steps that reduce the problem of capitalizing on chance in item selections are suggested. (SLD)

Computer simulations were run to measure the relationship between testlet validity and factors of item pool size and testlet length for both adaptive and linearly constructed testlets. Making a testlet adaptive yields only modest increases in aggregate validity because of the peakedness of the typical proficiency distribution. (Author/SLD)

Six procedures for combining sets of item response theory (IRT) item parameter estimates from different samples were evaluated using real and simulated response data. Results support use of covariance matrix-weighted averaging and a procedure using sample-size-weighted averaging of estimated item characteristic curves at the center of the ability…