Most researchers using analysis of variance (ANOVA) use a fixed-effects model. However, a random- or mixed-effects model may be a more appropriate fit for many research designs. One benefit of the random- and mixed-effects models is that they yield more generalizable results. This paper focuses on the similarities and differences between the…

A brief, practical overview of "design effects" (DEFFs) is presented for users of the results of sample surveys. The overview is intended to help such users to determine how and when to use DEFFs and to compute them correctly. DEFFs are needed only for inferential statistics, not for descriptive statistics. When the selections for…

The stepwise regression method of selecting predictors for computer assisted multiple regression analysis was compared with forward, backward, and best subsets regression, using 16 data sets. The results indicated the stepwise method was preferred because of its practical nature, when the models chosen by different selection methods were similar…

The purpose of this paper is to demonstrate how multiple linear regression provides a viable statistical methodology for dealing with meta-analysis in general, and specifically with the issues of nonindependence and design complexity, such as multiple treatments. Since the F-test and t-test are special cases of the general linear model,…

A general model is described which can be used to represent the four common types of meta-analysis: (1) estimation of effect size by combining study outcomes; (2) estimation of effect size by contrasting study outcomes; (3) estimation of statistical significance by combining study outcomes; and (4) estimation of statistical significance by…

The accuracy of trait estimates obtained from three vertical equating methods was examined. The procedures studied included two anchor test designs and a single-group design. Data from two content areas and two grade combinations were studied. A three-parameter logistic model was used to perform the equatings. The results obtained were used to…

This study attempted to determine if there was a change in job attitude among employees of a state institution for the developmentally disabled after a move to new, superior facilities. An attitude scale was constructed and administered in December 1982, prior to the move. A second testing occurred two months later, after Experimental Group I had…

This study examines the application of the marginal maximum likelihood (MML) EM algorithm to the parameter estimation problem of the three-parameter normal ogive and logistic polychotomous item response models. A three-parameter normal ogive model, the Graded Response model, has been developed on the basis of Samejima's two-parameter graded…

Generalizability theory, which subsumes classical measurement theory as a special case, provides a general model for estimating the reliability of observational rating data by estimating the variance components of the measurement design. Research data from the "Heart Smart" health intervention program were analyzed as a heuristic tool.…

Hypothetical data sets are used to demonstrate how canonical correlation methods subsume other commonly utilized parametric methods. Analysis of variance, analysis of covariance, multiple analysis of variance, and multiple analysis of covariance are heavily used by educational researchers. It is concluded that researchers would do well to consider…

The central purpose of the present study was to evaluate several statistical techniques (analysis of variance (ANOVA) on raw gains, standardized ANOVA, standard analysis of covariance (ANCOVA), and z-score ANOVA) under various quasi-experimental conditions (three levels of reliability, three levels of sample size, three levels of gain, and the…

This paper reports the results of a Monte Carlo investigation of Type I errors in the single group repeated measures design where multiple measures are collected from each observational unit at each measurement occasion. The Type I error of three multivariate tests were examined. These were the doubly multivariate F test, the multivariate mixed…

A method for drawing inferences from complex samples is based on Rubin's approach to missing data in survey research. Standard procedures for drawing such inferences do not apply when the variables of interest are not observed directly, but must be inferred from secondary random variables which depend on the variables of interest stochastically.…

This report provides the statistical tools necessary to measure the extent of error that exists in organizational record data and group survey data. It is felt that traditional methods of measuring error are inappropriate or incomplete when applied to organizational groups, especially in studies of organizational change when the same variables are…

Statistical significance is misused in educational and psychological research when it is applied as a method to establish the reliability of research results. Other techniques have been developed which can be correctly utilized to establish the generalizability of findings. Methods that do provide such estimates are known as invariance or…