Most existing diagnostic models are developed to detect whether students have mastered a set of skills of interest, but few have focused on identifying what scientific misconceptions students possess. This article developed a general dual-purpose model for simultaneously estimating students' overall ability and the presence and absence of…

Van der Linden's hierarchical model for responses and response times can be used in order to infer the ability and mental speed of test takers from their responses and response times in an educational test. A standard approach for this is maximum likelihood estimation. In real-world applications, the data of some test takers might be partly…

Cognitive diagnosis models (CDMs) typically assume skill attributes with discrete (often binary) levels of skill mastery, making the existence of skill continuity an anticipated form of model misspecification. In this article, misspecification due to skill continuity is argued to be of particular concern for several CDM applications due to the…

For a certification, licensure, or placement exam, allowing examinees to take multiple attempts at the test could effectively change the pass rate. Change in the pass rate can occur without any change in the underlying latent trait, and can be an artifact of multiple attempts and imperfect reliability of the test. By deriving formulae to compute…

When cut scores for classifications occur on the total score scale, popular methods for estimating classification accuracy (CA) and classification consistency (CC) require assumptions about a parametric form of the test scores or about a parametric response model, such as item response theory (IRT). This article develops an approach to estimate CA…

As item response theory has been more widely applied, investigating the fit of a parametric model becomes an important part of the measurement process. There is a lack of promising solutions to the detection of model misfit in IRT. Douglas and Cohen introduced a general nonparametric approach, RISE (Root Integrated Squared Error), for detecting…

The DINA (deterministic input, noisy, and gate) model has been widely used in cognitive diagnosis tests and in the process of test development. The outcomes known as slip and guess are included in the DINA model function representing the responses to the items. This study aimed to extend the DINA model by using the random-effect approach to allow…

DIMTEST is a nonparametric statistical test procedure for assessing unidimensionality of binary item response data that uses the T-statistic of W. F. Stout (1987). This study investigates the performance of the T-statistic with respect to different shapes of ability distributions and confirms its nonparametric nature. (SLD)

The effect of violating the assumption of nonspeededness on ability and item parameter estimates in item response theory was studied through simulation under three speededness conditions. Results indicate that ability estimation was least affected by speededness but that substantial effects on item parameter estimates were found. (SLD)

The simultaneous assessment of differential item functioning (DIF) for a collection of test items through an index that measures the variance of DIF on a test as an indicator of the degree to which different items show DIF in different directions is proposed and evaluated through simulations. (SLD)

As an alternative to equally weighted difference scores, examined an orthogonal reliable component analysis (RCA) solution and an oblique principal components analysis (PCA) solution for the standardization sample of the Kaufman Assessment Battery for Children (KABC; A. Kaufman and N. Kaufman, 1983). Discusses the practical implications of the…

Investigated the effectiveness of H. Wainer's (1993) strategy for obtaining positively biased ability estimates when examinees can review and change answers on computerized adaptive tests. Results, based on simulation and testing data from 87 college students, show that the Wainer strategy sometimes yields inflated ability estimates and sometimes…

This article describes a method of incorporating two ability estimates into a single regression analysis so that substantially fewer items exhibit differential item functioning (DIF). Comparative analysis with the Mantel-Haenszel procedure also shows more accurate matching of examinees, preventing identifying multidimensional item impact as DIF.…

Used simulations of computerized adaptive tests to evaluate results yielded by four commonly used ability estimation methods: maximum likelihood estimation (MLE) and three Bayesian approaches. Results show clear distinctions between MLE and Bayesian methods. (SLD)

In a simulation study of ability and estimation of differential item functioning (DIF) in computerized adaptive tests, Rasch-based DIF statistics were highly correlated with generating DIF, but DIF statistics tended to be slightly smaller than in the three-parameter logistic model analyses. (SLD)