A critical discussion of the assumption of uncorrelated errors in classical psychometric theory and its applications is provided. It is pointed out that this assumption is essential for a number of fundamental results and underlies the concept of parallel tests, the Spearman-Brown's prophecy and the correction for attenuation formulas as well as…

The frequently neglected and often misunderstood relationship between classical test theory and item response theory is discussed for the unidimensional case with binary measures and no guessing. It is pointed out that popular item response models can be directly obtained from classical test theory-based models by accounting for the discrete…

The accuracy of the results of the national tests in English, mathematics and science taken by 11-year olds in England has been a matter of much debate since their introduction in 1994, with estimates of the proportion of students incorrectly classified varying from 10 to 30%. Using live data from the 2009 and 2010 administration of the national…

Subscores are reported for several operational assessments. Haberman (2008) suggested a method based on classical test theory to determine if the true subscore is predicted better by the corresponding subscore or the total score. Researchers are often interested in learning how different subgroups perform on subtests. Stricker (1993) and…

A common problem in predictive validity studies in the educational and psychological fields, e.g. in educational and employment selection, is restriction in range of the predictor variables. There are several methods for correcting correlations for restriction of range. The aim of this paper was to examine the usefulness of two approaches to…

Uses classical test theory to show that it is the nature of the range restriction, rather than the nature of the available reliability coefficient, that determines the sequence for applying corrections for range restriction and unreliability. Shows how the common rule of thumb for choosing the sequence is tenable only when the correction does not…

Evaluated coefficient alpha under violations of two classical test theory assumptions: essential tau-equivalence and uncorrelated errors through simulation. Discusses the interactive effects of both violations with true and error scores. Provides empirical evidence of the derivation of M. Novick and C. Lewis (1993). (SLD)

This article studies the ordinal reliability of (total) test scores. This study is based on a classical-type linear model of observed score (X), true score (T), and random error (E). Based on the idea of Kendall's tau-a coefficient, a measure of ordinal reliability for small-examinee populations is developed. This measure is extended to large…

In many instances it is reasonable to assume that the population covariance matrix has positive elements. This assumption implies for the single factor analysis model that the loadings and regression weights for best linear factor prediction are positive. For the multiple factor analysis model where each variable loads on a single factor and a…

It has often been asserted that the reliability of a measure places an upper limit on its validity. This article demonstrates in theory that validity can rise when reliability declines, even when validity evidence is a correlation with an acceptable criterion. Whether empirical examples can actually be found is an open question. (SLD)

E. Roskam's (1985) conjecture that steeper item characteristic curve (ICC) "a" parameters (slopes) (and higher item total correlations in classical test theory) would be found with more concretely worded test items was tested with results from 925 young adults on the Eysenck Personality Questionnaire (H. Eysenck and S. Eysenck, 1975).…

The reliability of a gain is determined by the reliabilities of the components, the correlation between them, and their standard deviations. Reliability is not inherently low, but the components of gains in many investigations make low reliability likely and require caution in the use of gain scores. (SLD)

The critiques by L. Collins and L. Humphreys in this issue illustrate problems with the use of gain scores. Collins' examples show that familiar formulas for the reliability of differences do not reflect the precision of measures of change. Additional examples demonstrate flaws in the conventional approach to reliability. (SLD)

People can use a variety of different strategies to perform tasks and these strategies all have two characteristics in common. First, they can be evaluated in comparison with either an absolute or a relative standard. Second, they can be used at varying levels of consistency. In the present article, the authors develop a general theory of task…

This paper points out that there are certain generalizations about directions for guessing and methods of scoring that require that data be derived from random groups design. It supports the viewpoint that it is neither sufficient nor appropriate to make such generalizations on the basis of an analysis of scores obtained from the answer sheets of…