The present study was conducted to characterize (a) the features of the thousands of primary reports synthesized in 47 reliability generalization (RG) measurement meta-analysis studies and (b) typical methodological practice within the RG literature to date. With respect to the treatment of score reliability in the literature, in an astounding…

Correlation matrices and standard deviations are the building blocks of many of the commonly conducted analyses in published research, and AERA and APA reporting standards recommend their inclusion when reporting research results. The authors argue that the inclusion of correlation/covariance matrices, standard deviations, and means can enhance…

Investigated how dissimilar in composition and variability samples inducting reliability coefficients from prior studies were from the cited prior samples from which coefficients were generalized. Results from 20 articles show that citing reliability coefficients from prior studies as the basis for concluding new scores are reliable is only…

Responds to criticisms of some "Educational and Psychological Measurement" policies and the reliability generalization meta-analytic methods of T. Vacha-Haase. Explores consequences of misunderstanding score reliability. (SLD)

Most training programs in education and psychology focus on classical test theory techniques for assessing score dependability. This paper discusses generalizability theory and explores its concepts using a small heuristic data set. Generalizability theory subsumes and extends classical test score theory. It is able to estimate the magnitude of…

The program evaluation standards approved by the American National Standards Institute (ANSI) in 1994 that deal with reliability and validity accurately represent contemporary views of the psychometric community with regard to reliability and validity. As such, these standards move the field forward. The ANSI standards recognize that reliability…

The reliability and construct validity of scores on the Personal Preferences Self-Description Questionnaire (PPSDQ) (B. Thompson) were studied. Classical psychometric analysis of data from 641 participants (item and alpha analyses and both oblique and orthogonal exploratory factor analyses) indicate that the PPSDQ has reasonable properties. (SLD)

Given the potential value of reliability generalization (RG) studies in the development of cumulative psychometric knowledge, the purpose of this paper is to provide a tutorial on how to conduct such studies and to serve as a guide for researchers wishing to use this methodology. After some brief comments on classical test theory, the paper…

Expectations for reporting factor analysis results as part of construct validation are explored in the context of emerging views of measurement validity. Desired practices are discussed for both exploratory and confirmatory factor analysis. A short computer program for conducting parallel analysis is included. (Author/SLD)

A general linear model (GLM) framework is used to suggest that structure coefficients ought to be interpreted in structural equation modeling confirmatory factor analysis (CFA) studies in which factors are correlated. The computation of structure coefficients in explanatory factor analysis and CFA is explained. Two heuristic data sets are used to…

Illustrates a number of ways in which confidence intervals for reliability coefficients can be estimated. Suggests that authors who submit articles to "Educational and Psychological Measurement" report confidence intervals for reliability estimates whenever they report score reliabilities and that they note the interval estimation…

After presenting a general linear model as a framework for discussion, this paper reviews five methodology errors that occur in educational research: (1) the use of stepwise methods; (2) the failure to consider in result interpretation the context specificity of analytic weights (e.g., regression beta weights, factor pattern coefficients,…

Research libraries are increasingly supplementing collection counts with perceptions of service quality as indices of status and productivity. The present study was undertaken to explore the reliability and validity of scores from the SERVQUAL measurement protocol (A. Parasuraman and others, 1991), which has previously been used in this type of…

The present study was a psychometric evaluation of two recently revised Jungian personality instruments, the Singer-Loomis Type Deployment Inventory (SL-TDI) and the Personal Preferences Self-Description Questionnaire (PPSDQ). The study also examined the relationship between Jungian personality dimensions and the Five-Factor Model of personality…

Instruments measuring Carl Jung's (1921/1971) theory of psychological types have been widely used in various counseling contexts. The most popular measure of types has been the Myers-Briggs Type Indicator (K. Briggs and I. Myers). This measure has been criticized for dichotomous scoring, forced-choice response formats, and differential gender…