A classic topic in the fields of psychometrics and measurement has been the impact of the number of scale categories on test score reliability. This study builds on previous research by further articulating the relationship between item response theory (IRT) and classical test theory (CTT). Equations are presented for comparing the reliability and…

In the typical application of a cognitive diagnosis model, the Q-matrix, which reflects the theory with respect to the skills indicated by the items, is assumed to be known. However, the Q-matrix is usually determined by expert judgment, and so there can be uncertainty about some of its elements. Here it is shown that this uncertainty can be…

The problem of factor score indeterminacy implies that the factor and the error scores cannot be completely disentangled in the factor model. It is therefore proposed to compute Harman's factor score predictor that contains an additive combination of factor and error variance. This additive combination is discussed in the framework of classical…

This article extends the work by Armstrong and Shi on CUmulative SUM (CUSUM) person-fit methodology. The authors present new theoretical considerations concerning the use of CUSUM person-fit statistics based on likelihood ratios for the purpose of detecting cheating and random guessing by individual test takers. According to the Neyman-Pearson…

Various authors have proposed agreement indices for measuring nominal scale response agreement between two judges. Two situations may occur. Either the categories of the nominal scale are defined in advance and both raters use the same categories, or the categories are not defined in advance and the number of categories used by each rater is…

This article proposes several statistics for assessing individual fit based on two unidimensional models for continuous responses: linear factor analysis and Samejima's continuous response model. Both models are approached using a common framework based on underlying response variables and are formulated at the individual level as fixed regression…

In a typical study comparing the relative efficiency of two item selection rules in computerized adaptive testing, the common result is that they simultaneously differ in accuracy and security, making it difficult to reach a conclusion on which is the more appropriate rule. This study proposes a strategy to conduct a global comparison of two or…

The authors propose a method to compare the influence of a treatment on different properties within subjects. The properties are measured by several Likert-type-scaled items. The results show that many existing approaches, such as repeated measurement analysis of variance on sum and mean scores, a linear partial credit model, and a graded response…

Longitudinal studies are permeating clinical trials in psychiatry. Therefore, it is of utmost importance to study the psychometric properties of rating scales, frequently used in these trials, within a longitudinal framework. However, intrasubject serial correlation and memory effects are problematic issues often encountered in longitudinal data.…

Raju and Oshima (2005) proposed two prophecy formulas based on item response theory in order to predict the reliability of ability estimates for a test after change in its length. The first prophecy formula is equivalent to the classical Spearman-Brown prophecy formula. The second prophecy formula is misleading because of an underlying false…

This study explores the effects of ignoring item interaction on item parameter estimation and the efficiency of using the local dependence index Q[subscript 3] and the SAS NLMIXED procedure to detect item interaction under the three-parameter logistic model and the generalized partial credit model. Through simulations, it was found that ignoring…

Derives equations for computing weights that maximize the reliability of a test with multiple parts using a congeneric model. Presents a direct derivation for the three-part case and a two-step derivation for the "n"-part case. Gives examples that show the computations and the usefulness of the equations. (SLD)

Gulliksen's matched random subtests method is a graphical method to split a test into parallel test halves, allowing maximization of coefficient alpha as a lower bound to the classical test reliability coefficient. This problem is formulated as a zero-one programing problem solvable by algorithms that already exist. (TJH)

Presents results from two methods of evaluating monotonicity in item response models: regressing individual item scores on the total test score and on the "rest" score obtained by omitting the selected items from the total test score. Discusses implications for exploratory analysis of dichotomous item response data and the application of…

PARAFAC is a generalization of principal components analysis in a factor score matrix and in a factor loadings matrix. How PARAFAC behaves when applied to positive manifold data is examined, and a constrained PARAFAC method is offered for use when PARAFAC does not produce a positive manifold solution. (SLD)