The utility and interpretation of a coefficient derived by use of the generalizability theory is discussed in this article. Then it is employed to assess the reliability of several school-level variables from the High School and Beyond study. (JD)

This paper outlines the problems associated with the estimation of variance components in generalizability analyses using analysis of variance software, and discusses the most useful software currently available for this specialist application: the MIVQUE method of the Statistical Analysis System (SAS) procedure VARCOMP. (Author)

Simulations demonstrate the effects of correlated errors on the person-by-occasion design in which the confounding effect of equal time intervals results in correlated error terms in the linear model. Two specific error correlation structures were examined. Conditions under which underestimation and overestimation occur are discussed. (SLD)

This book presents a comprehensive overview of univariate and multivariate generalizability theory, a psychometric model that provides a powerful approach to the analysis of errors of measurement through the use of random-effects and mixed-model analysis of variance. (SLD)

The applicability is explored of the Bayesian random-effect analysis of variance (ANOVA) model developed by G. C. Tiao and W. Y. Tan (1966) and a method suggested by H. K. Suen and P. S. Lee (1987) for the generalizability analysis of autocorrelated data. According to Tiao and Tan, if time series data could be described as a first-order…

Khuri's and Satterthwaite's methods of obtaining confidence intervals of variance components are compared. The article discusses that Khuri's method may be applied to obtain confidence intervals for the variance components and other linear functions of the expected mean squares used in generalizability theory. (Author/JAZ)

Generalizability theory is based upon analysis of variance (ANOVA) and requires estimation of variance components for the ANOVA design under consideration in order to compute either G (Generalizability) or D (Decision) coefficients. Estimation of variance components has a number of alternative methods available using SAS, BMDP, and ad hoc…

Four approaches to estimating interrater reliability in early childhood special education research are illustrated and compared: correlation, comparison of means, percentage of agreement, and generalizability theory techniques. Generalizability theory techniques are proposed as a method for estimating the amount of variance attributable to…

In behavior research using cognitive and affective measures, there is often incongruity between the statistical analysis employed and the intended inference. This paper argues that incorporating items as levels of a random facet via generalizability theory allows the statistical examination of the inferential question in the desired universe of…

Most training programs in education and psychology focus on classical test theory techniques for assessing score dependability. This paper discusses generalizability theory and explores its concepts using a small heuristic data set. Generalizability theory subsumes and extends classical test score theory. It is able to estimate the magnitude of…

The framework and procedures of generalizability theory are introduced and illustrated in this instructional module that uses a hypothetical scenario involving writing proficiency. Generalizability analyses are useful for understanding the relative importance of various sources of error and for designing efficient measurement procedures. (SLD)

Generalizability theory is used to examine the sources of variability present in a teacher and course evaluation instrument. Two studies were conducted. In the first study, four different forms commonly used by one specific college of a large midwestern university were examined using responses of 915 students. The analysis of variance performed on…

This study is an attempt at a cohesive characterization of the concept of essay reliability. As such, it takes as a basic premise that previous and current practices in reporting reliability estimates for essay tests have certain shortcomings. The study provides an analysis of these shortcomings--partly to encourage a fuller understanding of the…

The purpose of this study was to compare the relative magnitude of the variance components and generalizability coefficients derived from the Teacher Performance Assessment Instruments (TPAI) data using two different methods of data collection: (1) occasions when observers were in the classroom for simultaneous observation and (2) occasions when…

Natural statistical models based on renewal processes for individuals' Behavior Stream are developed. For continuous observation, statistical and psychometric properties are obtained for empirical: (1) rates of behavior; (2) proportions of behavior type; (3) prevalence; and (4) event duration. Results provide a guide for design and analysis. (SLD)