The distributional characteristics of the Kaiser-Rice measure of sampling adequacy (MSA) were investigated with sample correlation matrices from multivariate normal populations where the level of correlation (LC) was systematically varied. Two additional variables were manipulated--sample size (SS) and number of variables (NV). Ten matrices were…

This paper explains the underlying assumptions of the sampling distribution and its role in significance testing. To compute statistical significance, estimates of population parameters must be obtained so that only one sampling distribution is defined. A sampling distribution is the underlying distribution of a statistic. Sampling distributions…

Levin's (1975) sample-size determination procedure for completely randomized analysis of variance designs is extended to designs in which antecedent or blocking variables information is considered. In particular, a researcher's choice of designs is framed in terms of determining the respective sample sizes necessary to detect specified contrasts…

This study integrates into one comprehensive Monte Carlo simulation a vast array of previously defined and substantively interrelated research studies of the robustness of analysis of variance (ANOVA) and analysis of covariance (ANCOVA) statistical procedures. Three sets of balanced ANOVA and ANCOVA designs (group sizes of 15, 30, and 45) and one…

In true experiments in which sample material can be randomly assigned to treatment conditions, most researchers presume that the condition of equal sample sizes is statistically desirable. When one or more a priori contrasts can be identified which represent a few overriding experimental concerns, however, allocating sample material unequally will…

An influential statistics test recommends a Levene text for homogeneity of variance. A recent note suggests that Levene's test is upwardly biased for small samples. Another report shows inflated Alpha estimates and low power. Neither study utilized more than two sample sizes. This Monte Carlo study involved sampling from a normal population for…

In most behavioral science research very little attention is ever given to the probability of committing a Type II error, i.e., the probability of failing to reject a false null hypothesis. Recent publications by Cohen have led to insight on this topic for the fixed-effects analysis of variance and covariance. This paper provides social scientists…

The purpose of this study is to help define the precise nature and limits of the tolerable range in which a researcher may be relatively confident about the statistical validity of his or her research findings, focusing specifically on the statistical validity of results when violating the assumptions associated with the one-way, fixed-effects…

Accurate estimation of variance components used in generalizability theory is essential for the theory to be viewed as an efficacious mechanism for studying the reliability and validity of a measurement procedure. This paper explores two alternatives for dealing with the apparent instability of small sample size used in determining the accuracy of…

This paper describes an item response theory (IRT) based method of differential item functioning (DIF) detection that involves neither separate calibration nor ability grouping. IRT is used to generate residual scores, scores free of the effects of person or group ability and item difficulty. Analysis of variance is then used to test the group…

A framework for understanding methodological practices from the perspectives of internal validity, external validity, statistical control validity, and construct validity is presented. One hundred doctoral dissertations completed between 1980 and 1988 at a single urban public university were analyzed for various methodological practices and types…

Statistical significance is misused in educational and psychological research when it is applied as a method to establish the reliability of research results. Other techniques have been developed which can be correctly utilized to establish the generalizability of findings. Methods that do provide such estimates are known as invariance or…

The statistical properties of two methods of estimating gain scores for groups in quasi-experiments are compared: (1) gains in scores standardized separately for each group; and (2) analysis of covariance with estimated true pretest scores. The fan spread hypothesis is assumed for groups but not necessarily assumed for members of the groups.…