Test-takers, policymakers, teachers, and institutions are increasingly demanding that testing programs provide more detailed feedback regarding test performance. As a result, there has been a growing interest in the reporting of subscores that potentially provide such detailed feedback. Haberman developed a method based on classical test theory…

The purpose of this study was to investigate a new way of evaluating interrater reliability that can allow one to determine if two raters differ with respect to their rating on a polytomous rating scale or constructed response item. Specifically, differential item functioning (DIF) analyses were used to assess interrater reliability and compared…

This study develops a theoretical model for the costs of an exam as a function of its duration. Two kind of costs are distinguished: (1) the costs of measurement errors and (2) the costs of the measurement. Both costs are expressed in time of the student. Based on a classical test theory model, enriched with assumptions on the context, the costs…

Building on prior research on the relationships between key concepts in item response theory and classical test theory, this note contributes to highlighting their important and useful links. A readily and widely applicable latent variable modeling procedure is discussed that can be used for point and interval estimation of the individual person…

Spearman's correction for attenuation (measurement error) corrects a correlation coefficient for measurement errors in either-or-both of two variables, and follows from the assumptions of classical test theory. Spearman's equation removes all measurement error from a correlation coefficient which translates into "increasing the reliability of…

There are well-defined theoretical differences between the classical test theory (CTT) and item response theory (IRT) frameworks. It is understood that in the CTT framework, person and item statistics are test- and sample-dependent. This is not the perception with IRT. For this reason, the IRT framework is considered to be theoretically superior…

Multilevel modeling (MLM) is frequently used to detect cluster-level group differences in cluster randomized trial and observational studies. Group differences on the outcomes (posttest scores) are detected by controlling for the covariate (pretest scores) as a proxy variable for unobserved factors that predict future attributes. The pretest and…

It has been argued that item response theory trait estimates should be used in analyses rather than number right (NR) or summated scale (SS) scores. Thissen and Orlando postulated that IRT scaling tends to produce trait estimates that are linearly related to the underlying trait being measured. Therefore, IRT trait estimates can be more useful…

Classical test theory methods are derived to allow statistical evaluation of all KT(KT - 1)/2 unique, non-trivial difference scores that occur by comparing one or more variables (K) at two or more testings (T) for a single individual (or object of measurement). Future research needs are discussed. (Author)

A simple statistical test procedure for assessing questionnaire response validity is proposed. The technique assesses the joint probability that frequently reported behaviors are not reported and infrequently reported behaviors are reported. (Author)

A definition of test validity as the ratio of a covariance term to a variance term, analogous to the classical definition of test reliability, is proposed. When error scores on distinct tests are uncorrelated, the proposed definition coincides with the usual one, but it remains meaningful when error scores are correlated. (Author/BW)

Reliability maximizing weights are related to theoretically specified true score scaling weights to show a constant relationship that is invariant under separate linear tranformations on each variable in the system. Test theoretic relations should be derived for the most general model available and not for unnecessarily constrained models.…

A new measure of similarity between persons applicable in Q-analysis is proposed. It allows assumptions of non-orthogonality between the items, across which the similarity is computed. The similarity measure may also be applied in an R-analysis. (Author/JKS)

The correlation between bias components of test scores and unbiased observed scores is shown to be an effective predictor of changes in reliability and validity resulting from elimination of bias. Plausible assumptions about value of correlation and size of related variance components indicate that reducation in reliability and validity is a…

Responses of 903 Canadian college students to the Self-Monitoring Scale (M. Snyder, 1974) and artificial data are used to illustrate the importance of pairing test theoretical structure with quantitative characteristics, including conditional association (CA) and strong positive orthant dependence (SPOD). Tests for CA and SPOD are reviewed. (SLD)