Conventional random-effects models in meta-analysis rely on large sample approximations instead of exact small sample results. While random-effects methods produce efficient estimates and confidence intervals for the summary effect have correct coverage when the number of studies is sufficiently large, we demonstrate that conventional methods…

In linear regression models estimated by ordinary least squares, it is often desirable to use hypothesis tests and confidence intervals that remain valid in the presence of heteroskedastic errors. Wald tests based on heteroskedasticity-consistent covariance matrix estimators (HCCMEs, also known as sandwich estimators or simply "robust"…

When undertaking quantitative hypothesis testing, social researchers need to decide whether the data with which they are working is suitable for parametric analyses to be used. When considering the relevant assumptions they can examine graphs and summary statistics but the decision making process is subjective and must also take into account the…

Recently a new mean scaled and skewness adjusted test statistic was developed for evaluating structural equation models in small samples and with potentially nonnormal data, but this statistic has received only limited evaluation. The performance of this statistic is compared to normal theory maximum likelihood and 2 well-known robust test…

This study proposes robust means modeling (RMM) approaches for hypothesis testing of mean differences for between-subjects designs in order to control the biasing effects of nonnormality and variance inequality. Drawing from structural equation modeling (SEM), the RMM approaches make no assumption of variance homogeneity and employ robust…

This study evaluated the robustness of two recent methods for analyzing multivariate repeated measures when the assumptions of covariance homogeneity and multivariate normality are violated. Specifically, the authors' work compares the performance of the modified Brown-Forsythe (MBF) procedure and the mixed-model procedure adjusted by the…

Methods for the analysis of within-subjects effects in multivariate groups by trials repeated measures designs are considered in the presence of heteroscedasticity of the group variance-covariance matrices and multivariate nonnormality. Under a doubly multivariate model approach to hypothesis testing, within-subjects main and interaction effect…

Kelley compared three methods for setting a confidence interval (CI) around Cohen's standardized mean difference statistic: the noncentral-"t"-based, percentile (PERC) bootstrap, and biased-corrected and accelerated (BCA) bootstrap methods under three conditions of nonnormality, eight cases of sample size, and six cases of population…

Effects of sample size and composition are systematically examined on the replicability of principal components, using observer ratings of personality from the California Adult Q-Set for 192 series of principal components analyses. Results indicate that dimensionality cannot be inferred from component robustness; they are empirically and logically…

Assessed the robustness of an estimation method for multilevel and path analysis with hierarchical data proposed by B. Muthen (1989) with unequal groups and small sample sizes and in the presence of a low or high intraclass correlation. Simulation results show the effects of varying these conditions on the within-group and between-groups part of…

The performance of analysis of covariance (ANCOVA) and six selected competitors was examined under varying experimental conditions through Monte Carlo simulations. The six alternatives were: (1) Quade's procedure (D. Quade, 1967); (2) Puri and Sen's solution (M. Puri and P. Sen, 1969); (3) Burnett and Barr's rank difference scores (T. Burnett and…

Type I error rates for the Johansen test were estimated using simulated data for a variety of conditions. Results indicate that Type I error rates for the Johansen test depend heavily on the number of groups and the ratio of the smallest sample size to the number of dependent variables. Sample size guidelines are presented. (SLD)

In this paper, six statistical tests of stochastic equality are compared with respect to Type I error and power through a Monte Carlo simulation. In the simulation, the skewness and kurtosis levels and the extent of variance heterogeneity of the two parent distributions were varied across a wide range. The sample sizes applied were either small or…

Examines the recommendations of H. Keselman, K. Carriere, and L. Lix (1993) regarding choice of sample size for obtaining robust tests of the repeated measures main and interaction hypotheses in a one Between-Subjects by one Within- Subjects design with a Welch-James type multivariate test when covariance matrices are heterogeneous. (SLD)

Five methods for testing the hypothesis of independence between two sets of variates were compared through simulation. Results indicate that two new methods, based on robust measures reflecting the linear association between two random variables, provide reasonably accurate control over Type I errors. Drawbacks to rank-based methods are discussed.…