In recent guidelines for fair educational testing it is advised to check the validity of individual test scores through the use of person-fit statistics. For practitioners it is unclear on the basis of the existing literature which statistic to use. An overview of relatively simple existing nonparametric approaches to identify atypical response…

The assessment of the number of dimensions and the dimensionality structure of questionnaire data is important in scale evaluation. In this study, the authors evaluate two dimensionality assessment procedures in the context of Mokken scale analysis (MSA), using a so-called fixed lowerbound. The comparative simulation study, covering various…

This study investigated statistical properties of the K-index (Holland, 1996) that can be used to detect copying behavior on tests. A simulation study was conducted to investigate the applicability of the K-index for small, medium, and large datasets. In addition, the Type I error rate and the detection rate of this index were compared with the…

Studied the statistical properties of the K-index (P. Holland, 1996) that can be used to detect copying behavior on a test through a simulation study of the use of the K-statistic with small, medium, and large datasets. Also compared the Type I error rate and detection rate of this index with those of the copying index (J. Wollack, 1997).…

Proposed two new indices to detect answer copying on a multiple choice test and conducted a simulation study to investigate the usefulness of both indexes. Discusses conditions under which the proposed indexes can be useful. (SLD)

In this study a cumulative-sum (CUSUM) procedure from the theory of Statistical Process Control was modified and applied in the context of person-fit analysis in a computerized adaptive testing (CAT) environment. Six person-fit statistics were proposed using the CUSUM procedure, and three of them could be used to investigate the CAT in online test…

Proposed person-fit statistics that are designed for use in a computerized adaptive test (CAT) and derived critical values for these statistics using cumulative sum (CUSUM) procedures so that item-score patterns can be classified as fitting or misfitting. Compared nominal Type I errors with empirical Type I errors through simulation studies. (SLD)

Studied the effect on criterion-related validity of nonfitting response vectors (NRVs) on a predictor test using simulation. Results suggest that NRVs can influence the validity of a test if the type of misfit is severe, the correlation between the predictor and criterion scores is "p"=0.3 or 0.4, and there are 15% or higher NRVs. (SLD)

Person-fit research in the context of paper-and-pencil tests is reviewed, and some specific problems regarding person fit in the context of computerized adaptive testing (CAT) are discussed. Some new methods are proposed to investigate person fit in a CAT environment. These statistics are based on Statistical Process Control (SPC) theory. A…

Studied whether the theoretical sampling distribution of the U3 person-fit statistic is in agreement with the simulated sampling distribution under different item response theory models and varying item and test characteristics. Simulation results suggest that the use of standard normal deviates for the standardized version of the U3 statistic may…

Studied the use of the person response function (PRF) for identifying nonfitting item score patterns. Proposed a person-fit method reformulated in a nonparametric item response theory (IRT) context. Conducted a simulation study to compare the use of the PRF with a person-fit statistic, resulting in the conclusion that the PRF can be used as a…

A Bayesian approach to the evaluation of person fit in item response theory (IRT) models is presented. In a posterior predictive check, the observed value on a discrepancy variable is positioned in its posterior distribution. In a Bayesian framework, a Markov Chain Monte Carlo procedure can be used to generate samples of the posterior distribution…

Through simulation, the power of the U3 statistic was compared with the power of one of the simplest person-fit statistics, the sum of the number of Guttman errors. In most cases, a weighted version of the latter statistic performed as well as the U3 statistic. (SLD)

The effect of person misfit to an item response theory (IRT) model on a mastery/nonmastery decision was investigated. Also investigated was whether the classification precision can be improved by identifying misfitting respondents using person-fit statistics. A simulation study was conducted to investigate the probability of a correct…

Two new indices to detect answer copying on a multiple-choice test, S(1) and S(2) (subscripts), are proposed. The S(1) index is similar to the K-index (P. Holland, 1996) and the K-overscore(2), (K2) index (L. Sotaridona and R. Meijer, in press), but the distribution of the number of matching incorrect answers of the source (examinee s) and the…