The concept under investigation was the reliability of estimates of mean scores of groups under various assumptions of multiple-matrix sampling when reliabilities are computed according to procedures based on generalizability theory. Four different cases were compared with respect to the generalizability coefficients depending upon whether pupils…

Some sampling characteristics of three estimators of the intraclass correlation were investigated under a variety of conditions within the context of a one-way analysis of variance. The results promote caution in the use of all three estimators. The three estimators differed very little in their bias or in their standard errors. (Author/JKS)

Researchers often use the analysis of variance to test hypotheses about the means, followed by a multiple comparison technique when the F-test is significant. The technique is this study was developed by Newman (1939) and Keuls (1952). A flaw in the rationale underlying this technique was evaluated to determine whether the flaw is sufficiently…

The distributional characteristics of the Kaiser-Rice measure of sampling adequacy (MSA) were investigated with sample correlation matrices from multivariate normal populations where the level of correlation (LC) was systematically varied. Two additional variables were manipulated--sample size (SS) and number of variables (NV). Ten matrices were…

An experimental sampling procedure for communities on which coal had been surface-mined yielded missing cells and caused the number of degrees of freedom to be N instead of the usual N minus one. The apparent discrepancy is explained, and a solution to the problem is presented. (Author/JKS)

Principal component analysis, image component analysis, and maximum likelihood factor analysis were compared to assess the effects of variable sampling. Results with respect to degree of saturation and average number of variables per factor were clear and dramatic. Differential effects on boundary cases and nonconvergence problems were also found.…

It is argued in this paper that generalizability theory provides a uniquely useful framework for defining and quantifying the dependability of data for decision making. It does so by requiring careful specification of the conditions of measurement and the anticipated sources of variation in the results of the measurement procedure. A distinction…

Flexibility in evaluative research design does not necessitate the abandonment of randomly constructed comparison groups. Three designs are reviewed which provide at least the option of randomization while maintaining great flexibility. The strengths and weaknesses of the designs are discussed. (Author)

The amount of variance accounted for by treatment can be estimated with omega squared or with the squared multiple correlation coefficient. Monte Carlo methods were employed to compare omega squared, the squared multiple correlation coefficient, and the squared multiple correlation coefficient to which a shrinkage formula had been applied, in…

The U.S. Army Research Institute for the Behavioral and Social Sciences has developed a wide range of statistical models to test hypotheses generated in relation to an equally wide range of measurement and evaluation situations. The randomized block (RB) design has traditionally been a preferred model for much psychological research. The RB has…

A computer-simulated study was made of the sampling distribution of omega squared, a measure of strength of relationship in multivariate analysis of variance which had earlier been proposed by the author. It was found that this measure was highly positively biased when the number of variables is large and the sample size is small. A correction…

The paper describes the small sample stability of least square estimates of variance components within the context of generalizability theory. Monte Carlo methods are used to generate data conforming to some selected multifacet generalizability designs to illustrate the sampling behavior of variance component estimates. (Author/CTM)

A new approximation is proposed for testing the equality of "k" independent means in the face of heterogeneity of variance. Monte Carlo simulations show that the new procedure has nearly nominal Type I error rates and Type II error rates that are close to those produced by James's second-order approximation. (SLD)

A method of investigating for possible bias in test items is proposed that uses analysis of variance for item data based on groups that have been selected to have identical test score distributions. The item data used are arcsin transformations of item difficulties. The methodological procedure has the following advantages: (1) The arcsin…

Dunnett's procedure for comparing K-1 treatments with a control is discussed within the context of three nonparametric models: those of Kruskal-Wallis, Friedman, and Cochran. (Author/MH)