I discuss three common practices that obfuscate or invalidate the statistical analysis of randomized controlled interventions in applied linguistics. These are (a) checking whether randomization produced groups that are balanced on a number of possibly relevant covariates, (b) using repeated measures ANOVA to analyze pretest-posttest designs, and…

Partially clustered designs, where clustering occurs in some conditions and not others, are common in psychology, particularly in prevention and intervention trials. This article reports results from a simulation comparing 5 approaches to analyzing partially clustered data, including Type I errors, parameter bias, efficiency, and power. Results…

This article uses meta-analyses published in "Psychological Bulletin" from 1995 to 2005 to describe meta-analyses in psychology, including examination of statistical power, Type I errors resulting from multiple comparisons, and model choice. Retrospective power estimates indicated that univariate categorical and continuous moderators, individual…

Cluster sampling results in response variable variation both among respondents (i.e., within-cluster or Level 1) and among clusters (i.e., between-cluster or Level 2). Properly modeling within- and between-cluster variation could be of substantive interest in numerous settings, but applied researchers typically test only within-cluster (i.e.,…

This study examines the effects of kindergarten retention on children's social-emotional development in the early, middle, and late elementary years. Previous studies have generated mixed results partly due to some major methodological challenges, including selection bias, measurement error, and divergent perceptions of multiple respondents in…

A probability distribution was developed for the Andrews-Pregibon (AP) statistic. The statistic, developed by D. F. Andrews and D. Pregibon (1978), identifies multivariate outliers. It is a ratio of the determinant of the data matrix with an observation deleted to the determinant of the entire data matrix. Although the AP statistic has been used…

It is demonstrated that, in the presence of population information, a linear combination of true scores can be estimated more efficiently than by the same linear combination of the observed scores. Three criteria for optimality are discussed, but they yield the same solution, described as a multivariate shrinkage estimator. (Author/SLD)

A methodology is presented for minimizing the mean error variance-covariance component in studies with resource constraints. The method is illustrated using a one-facet multivariate design. Extensions to other designs are discussed. (SLD)

This repeated measures factorial design study compared the results of two procedures for identifying multivariate outliers under varying conditions, the Mahalanobis distance and the Andrews-Pregibon statistic. Results were analyzed for the total number of outliers identified and number of false outliers identified. Simulated data were limited to…

Computer sampling from a multivariate normal spherical population was used to evaluate the type one error rates for a test of sphericity based on the distribution of the determinant of the sample correlation matrix. (Author/LMO)

Describes the appropriate control of experiment-wise error rates and the researcher's role in making decisions to control for Type 1 errors. Issues related to level of significance, one- versus two-tailed tests, and the use of multivariate statistics are included. (Author/CT)

Analytical solutions for point and variance estimators of the mediated effect, the ratio of mediated to direct effect, and the proportion of the total effect mediated were determined through simulation for different samples. The sample sizes needed for accuracy and stability are discussed with implications for mediated effects estimates. (SLD)

Ignoring a level can have a substantial impact on the conclusions of a multilevel analysis. For intercept-only models and for balanced data, we derive these effects analytically. For more complex random intercept models or for unbalanced data, a simulation study is performed. Most important effects concern estimates and corresponding standard…

The authors compare the operating characteristics of the bootstrap-F approach, a direct extension of the work of Berkovits, Hancock, and Nevitt, with Huynh's improved general approximation (IGA) and the Brown-Forsythe (BF) multivariate approach in a mixed repeated measures design when normality and multisample sphericity assumptions do not hold.…

The main purpose of this study is to examine the content structure of the Multistate Bar Examination (MBE) using the "table of specifications" model from the perspective of multivariate generalizability theory. Specifically, using MBE data collected over different years (six administrations: three from the February test and three from July test),…