This paper evaluated multidimensional linking procedures with which multidimensional test data from two separate calibrations were put on a common scale. Data were simulated with known ability distributions varying on two factors which made linking necessary: mean vector differences and variance-covariance (v-c) matrix differences. After the…

A person-fit method that allows researchers to identify nonfitting response vectors is the l(z) statistic. Simulation results show that l(z) may not perform as expected when estimated person parameters are used rather than true person parameters. Other considerations in using true and estimated person parameters are discussed. (SLD)

The utility of the three-form design coupled with maximum likelihood methods for estimation of missing values was evaluated. Simulation studies demonstrate that maximum likelihood estimation and multiple imputation methods produce the most efficient and least biased estimates of variances and covariances for normally distributed and slightly…

It is suggested that using an unbiased estimate of the weight matrix may eliminate the small or intermediate sample size bias of the asymptotically distribution-free (ADF) test statistic. Results of simulations show that test statistics based on the biased estimator or the unbiased estimate are highly similar. (SLD)

A method of comparing one-step M-estimates of location for heavy tailed distributions is proposed and investigated. Simulations indicate that the new procedure provides good control over Type I errors and has more power than do some other methods for dealing with heavy tailed distributions. (SLD)

The extent to which properties of the ability scale and the form of the latent trait distribution influence the estimated item parameters of item response theory (IRT) was investigated using real and simulated data. Simulated data included 5,000 ability values randomly drawn from the standard normal distribution. Real data included the results for…

Develops and evaluates the linkage of the Armed Services Vocational Aptitude Battery to the mathematics scale of the National Assessment of Educational Progress. The accuracy of the proficiency distribution estimated from the projection was close to the accuracy of the distribution estimated from the large scale assessment. (SLD)

The main goal of this study was to assess how sensitive unidimensional parameter estimates derived from BILOG were when the unidimensionality assumption was violated and the underlying ability distribution was not multivariate normal. A multidimensional three-parameter logistic distribution that was a straightforward generalization of the…

This paper describes a study, now in progress, of new methods for representing the sampling variability of Mantel-Haenszel differential item functioning (DIF) results, based on the system for categorizing the severity of DIF that is now in place at the Educational Testing Service. The methods, which involve a Bayesian elaboration of procedures…

Validity generalization meta-analysis (VG) examines the extent to which the validity of an instrument can be transported across settings. VG offers correction and summarization procedures designed in part to remove the effects of statistical artifacts on estimates of association between criterion and predictor. By employing a random effects model,…

A method of item pool design is proposed that uses an optimal blueprint for the item pool calculated from the test specifications. The blueprint is a document that specifies the attributes that the items in the computerized adaptive test (CAT) pool should have. The blueprint can be a starting point for the item writing process, and it can be used…

Analysis of a two-way sample of means is considered when corresponding population means are believed a priori to satisfy a partial order restriction. Simulation and the Gibbs sampler are used to summarize posterior distributions, and the posterior distribution is used to predict GPAs of first-year students at University of Iowa. (SLD)

Hierarchical Bayes procedures were compared for estimating item and ability parameters in item response theory. Simulated data sets from the two-parameter logistic model were analyzed using three different hierarchical Bayes procedures: (1) the joint Bayesian with known hyperparameters (JB1); (2) the joint Bayesian with information hyperpriors…

Responses to previously calibrated items administered in a computerized adaptive testing (CAT) mode may be used to recalibrate the items. This live-data simulation study investigated the possibility, and limitations, of on-line adaptive recalibration of precalibrated items. Responses to items of a Rasch-based paper-and-pencil licensure examination…