The assumptions of the classical test-theory model are used to develop a theory of reliability for criterion-referenced measures which parallels that for norm-referenced measures. It is shown that the Spearman-Brown formula holds for criterion-referenced measures and that the criterion-referenced reliability coefficient can be used to correct…

Five-hundred dichotomously scored response patterns were generated with sequentially independent (SI) items and 500 with dependent (SD) items for each of thirty-six combinations of sampling parameters (i.e., three test lengths, three sample sizes, and four item difficulty distributions). KR-20, KR-21, and Split-Half (S-H) reliabilities were…

This paper, a revision of the original document, "Correcting Partial, Multiple, and Canonical Correlations for Attenuation" (see TM 000 535), presents the formula for correcting coefficients of partial correlation for attenuation due to errors of measurement. In addition, the correction for attenuation formulas for multiple and cannonical…

The correction for attenuation formulas for partial, multiple, and canonical correlation coefficients are discussed and the effects of measurement errors on these statistics are explored. The notation is standardized and the derivation extended where appropriate. It is shown that as the reliabilities of the predictors become more disparate, the…

The problem of the comparability of change scores is investigated. Change quotients and residual change scores are evaluated as alternative approaches and methods for estimating the true change and true score residual, the reliability of change scores and residuals, and procedures for constructing confidence intervals for residuals are explored.…

A number of models are proposed of the effects of demographic and environmental factors on IQ and its pattern of change over time. The proposed models are concerned with the determinants of an individual's true (but unobserved) IQ and the relationship between measured and true IQ's. (Author/RC)

Multidimensionality is known to affect the accuracy of item parameter and ability estimations, which subsequently influences the computation of item characteristic curves (ICCs) and true scores. By judiciously combining sections of a Law School Admission Test (LSAT), 11 sections of varying degrees of uni- and multidimensional structures are used…

A comparison of the relative ordering power of separate and grouped-items true-false tests indicated that neither type of test was uniformly superior to the other across all levels of knowledge of examinees. Grouped-item tests were found superior for examinees with low levels of knowledge. (Author/CTM)

Procedures are described for studying the reliability of decisions based on specific passing scores with tests made up of discrete items and designed to measure continuous rather than categorical traits. These procedures are based on the estimation of the joint distribution of true scores and observed scores. (CTM)

A test theory using only ordinal assumptions is presented, based on the idea that the test items are a sample from a universe of items. The sum across items of the ordinal relations for a pair of persons on the universe items is analogous to a true score. (SLD)

This commentary focuses on the application of D. Rogosa and G. Ghandour's work to observational research on classroom processes. Rogosa and Ghandour have shown that the short length of an observation is typically the dominant source of error. Investigators should conduct observations for as long as possible. (SLD)

This paper provides the basis for understanding score equating through item response theory (IRT). Theoretical justifications and practical advantages of IRT true-score test procedures are discussed. Three steps in the equating process are specified, and a self-test is included. (SLD)

A model-based approach to rater reliability for essays read by multiple readers is presented. Variation of rater severity (between-rater variation) and rater inconsistency (within-rater variation) is considered in the presence of between-examinee variation. An additive variance component model is posited and the method of moments for its…

The effects of luck and misinformation on ability of multiple-choice test scores to estimate examinee ability were investigated. Two measures of examinee ability were defined. Misinformation was shown to have little effect on ability of raw scores and a substantial effect on ability of corrected-for-guessing scores to estimate examinee ability.…

False-positive and false-negative dicisions are the fundamental errors committed with a mastery test; yet the estimation of the likelihood of committing these errors has not been investigated. Accordingly, two methods of estimating the likelihood of committing these errors are described and then investigated using Monte Carlo techniques.…