This paper discusses the benefits of using generalizabilty theory in lieu of classical test theory. Generalizability theory subsumes and extends the precepts of classical test theory by estimating the magnitude of multiple sources of measurement error and their interactions simultaneously in a single analysis. Since classical test theory examines…

Previous methods for estimating the conditional standard error of measurement (CSEM) at specific score or ability levels are critically discussed, and a brief summary of prior empirical results is given. A new method is developed which avoids theoretical problems inherent in some prior methods, is easy to implement, and estimates not only a…

This Monte Carlo investigation provides some possible solutions to problems related to choosing multiple-comparison methods that maximize true rejections and minimize false ones. It has been argued that the traditional Bonferroni approach to multiple comparisons, which satisfies the statistician's family-wise Type I error concerns, could be…

Linear and nonlinear methods for incorporating score precision information when the score scale is established for educational tests are compared. Examples illustrate the methods, which discourage overinterpretation of small score differences and enhance score interpretability by equalizing error variance along the score scale. Measurement error…

Results of correlation studies of 141 adult epileptics' scores on the Background Interference Procedure (BIP) indicated that the BIP often does not agree with abnormal neurological diagnoses but often does agree with psychiatric diagnoses of Organic Brain Syndrome (OBS). Suggests that future BIP validity studies include a behavioral measure of OBS…

The literature on measurement reliability shows the consensus that group heterogeneity with regard to the trait being measured is a factor that affects the sample measurement reliability, but the degree of such effect is not entirely clear. Sample performance also has the potential to affect measurement reliability because of its effect on the…

Where the accuracy of a measurement is important, whether for scientific or practical purposes, the investigator should evaluate how much random error affects the measurement. New research may not be necessary when a procedure has been studied enough to establish how much error it involves. But, with new measures, or measures being transferred…

Several robust absolute deviation statistics have been developed recently. Two such correlation coefficients are developed and discussed, one for ranked data and another for interval level data. The standard error and range of the coefficients are given. The algebraic relationship between the coefficients and three widely used correlation…

Simulated data based on five test score distributions indicate that a slight modification of the asymptotic normal theory for the estimation of the p and kappa indices in mastery testing will provide results which are in close agreement with those based on small samples from the beta-binomial distribution. (Author/BW)

Attempted to illuminate practical limitations on the Rasch model by focusing on the recovery of the density scale through five simulation trials. Results show that when error distributions are insufficient, the results may be ordinal at best, and when error distributions are nonsymmetrical, the positions of items may be biased with respect to the…

Generalizability Theory can be used to assess reliability in the presence of multiple sources and different types of error. It provides a flexible alternative to Classical Theory and can handle estimation of interrater reliability with any number of raters. (SK)

A Monte Carlo study compared the accuracy of different estimates of the standard error of correlations corrected for restriction in range. The procedure suggested by P. Bobko and A. Rieck (1980) generated the most accurate estimates of the standard error. Aspects of accuracy are discussed. (SLD)

Monte Carlo study results show that the runs test yields markedly asymmetrical error rates in the two tails and that neither directional nor nondirectional tests are satisfactory with respect to Type I errors. The test is not recommended for evaluating the independence of errors in time-series regression models. (SLD)