One of the major problems that a tree-approach to data analysis often encounters is the instability of tree-structures. The instability issue must be dealt with before data can be interpreted by this method. Examining instability at a node of a tree provides insight into the instability of the whole tree, because the same theory of instability…

Described herein are formulas and computational procedures for estimating the mean and second through fourth central moments of universe scores through multiple matrix sampling. Additionally, procedures are given for approximating the standard error associated with each estimate. All procedures are applicable when items are scored either…

Parallel merit reliability (PMR) indexes the same consistency of measurement that is reflected in a validity coefficient; it reflects the reliability of measurement across identical merit score cases. Research has identified the potential benefits of the PMR approach as providing item level and cut-score reliability indices without requiring…

Standard errors of pooled mean estimate in multiple matrix sampling were compared for two procedures. The data were from tests involving items with and without replacement. The two procedures involve the formulations of Madow and Lord, and Novick; the former permits sampling of item, with or without replacement, whereas the latter is to be used…

Data from 200 college-level tests were used to compare three reliability approximations (two of Saupe and one of Cureton) to Kuder-Richardson Formula 20 (KR20). While the approximations correlated highly (about .9) with the reliability estimate, they tended to be underapproximations. The explanation lies in an apparent bias of Lord's approximation…

Measurement error in dependent variables reduces the power of statistical tests to detect mean differences of specified magnitude. Procedures for determining power and sample size that consider the reliability of the dependent variable are discussed and illustrated. Methods for estimating reliability coefficients used in these procedures are…

Evaluates the projections of 1980 economic activity and industry output and employment. Discusses errors in employment projections (especially in underestimations of employment) and determines sources of errors. (SK)

Based on the test theory model for ordinal measurements proposed by Schulman and Haden, the present paper considers correlations between tests, attenuation, regressions involving true and observed scores, and predictions of test reliability. Comparisons are made to interval test theory. (Author/JKS)

This paper illustrates first how estimated Structural Equation Modeling (SEM) measurement error variances are actually estimates of score reliabilities. The major advantage of SEM over other analytic methods is that it accounts for measurement error. Score reliabilities are estimated as part of structural modeling, so that structural models test…

Given the potential value of reliability generalization (RG) studies in the development of cumulative psychometric knowledge, the purpose of this paper is to provide a tutorial on how to conduct such studies and to serve as a guide for researchers wishing to use this methodology. After some brief comments on classical test theory, the paper…

The purpose of this paper is to provide an in-depth critical analysis of the use and misuse of correlation coefficients. Various analytical and interpretational misconceptions are reviewed, beginning with the egregious assumption that correlational statistics may be useful in inferring causality. Additional misconceptions, stemming from…

D. Robinson and J. Levin (1997) proposed what they called a two-step procedure for analyzing statistical data in which researchers first evaluate the probability of an observed effect statistically (i.e., statistical significance), and, if and only if, it can be concluded that the underlying finding is too improbable to be due to chance, then they…

This paper focuses on handling measurement error in predictor variables using item response theory (IRT). Measurement error is of great important in assessment of theoretical constructs, such as intelligence or the school climate. Measurement error is modeled by treating the predictors as unobserved latent variables and using the normal ogive…

The Shine-Bower single subject ANOVA is extended to a multivariate case, with one example assuming between-variate dependencies among within-subject errors and the second assuming no between-variate dependencies among within-subject errors. Standard and simplified multivariate ANOVA procedures are used, respectively. (Author/CM)

This paper considers some appropriate and inappropriate uses of coefficient kappa and alternative kappa-like statistics. Discussion is restricted to the descriptive characteristics of these statistics for measuring agreement with categorical data in studies of reliability and validity. (Author)