Type I error rates and statistical power for the univariate F test and the James second-order test were estimated for the two-factor fixed-effects completely randomized design. Results reveal that the F test Type I error rate can exceed the nominal significance level when cell variances differ. (SLD)

Some results on how the Alexander-Govern heteroscedastic analysis of variance (ANOVA) procedure (R. Alexander and D. Govern, 1994) performs under nonnormality are presented. This method can provide poor control of Type I errors in some cases, and in some situations power decreases as differences among the means get large. (SLD)

The use of multiple comparisons in analysis of variance (ANOVA) is discussed. It is argued that experimentwise Type I error rate inflation can be serious and that its influences are often unnoticed in ANOVA applications. Both classical balanced omnibus and orthogonal planned contrast tests inflate experimentwise error to an identifiable maximum.…

Various methods of estimating main effects from ordinal data are presented and contrasted. Problems discussed include: (1) at what level to accumulate ordinal data into linear measures; (2) how to maintain scaling across analyses; and (3) the inevitable confounding of within cell variance with measurement error. An example shows three methods of…

This article focuses on steps in conducting empirical-analytic research and the problems of controlling for or estimating three sources of error: the amount of measurement error, research design error, and the amount of statistical or sampling error. (Author/CT)

A method is presented for the simultaneous analysis of differential item functioning (DIF) in multi-factor situations. The method is unique in that it combines item response theory (IRT) and analysis of variance (ANOVA), takes a simultaneous approach to multifactor DIF analysis, and is capable of capturing interaction and controlling for possible…

For the conditions investigated in the study, the parametric ANCOVA was typically the procedure of choice both as a test of equality of conditional means and as a test of equality of conditional distributions. (Author/LMO)

Khuri's and Satterthwaite's methods of obtaining confidence intervals of variance components are compared. The article discusses that Khuri's method may be applied to obtain confidence intervals for the variance components and other linear functions of the expected mean squares used in generalizability theory. (Author/JAZ)

The Type I error and power properties of the 2x2x2 analysis of variance (ANOVA) and tests developed by McSweeney (1967), Bradley (1979), Harwell-Serlin (1989; Harwell, 1991), and Blair-Sawilowsky (1990) were compared using Monte Carlo methods. The ANOVA was superior under the Gaussian and uniform distributions. The Blair-Sawilowsky test was…

In educational research, nonessential factors are commonly ignored and when accounted for, they are often treated statistically as fixed effects. Yet many researchers in these situations generalize their findings beyond the specific levels selected; however, the analyses may require treating the factor as a random effect. Such inappropriate…

Most training programs in education and psychology focus on classical test theory techniques for assessing score dependability. This paper discusses generalizability theory and explores its concepts using a small heuristic data set. Generalizability theory subsumes and extends classical test score theory. It is able to estimate the magnitude of…

The current state of usage of regression models in analysis of variance (ANOVA) designs is empirically examined, and examples of several statistical errors made in usage are presented. The assumptions of the general linear model are that all predictors are known without error of measurement and are fixed with no replication or sample variation; in…

This report describes and evaluates the major computer software packages capable of computing standard errors for statistics estimated from complex samples. It first describes the problem and the proposed solutions. The two major programs presently available, SUPER CARP and OSIRIS, are described in general terms. The kinds of statistics available…

This study is an attempt at a cohesive characterization of the concept of essay reliability. As such, it takes as a basic premise that previous and current practices in reporting reliability estimates for essay tests have certain shortcomings. The study provides an analysis of these shortcomings--partly to encourage a fuller understanding of the…

Addresses three issues concerning the assessment of the overrating and underrating of academic competence: (1) the impossibility of separating effects of overrating and underrating from effects of perceived and actual competence; (2) the questionable validity of Connell and Ilardi's method; and (3) the proposal of a new method and its implications…