Many experimental designs in educational and behavioral research involve at least one level of clustering. Clustering affects the precision of estimators and its impact on statistics in cross-sectional studies is well known. Clustering also occurs in longitudinal designs where students that are initially grouped may be regrouped in the following…

This article discusses estimation of average treatment effects for randomized controlled trials (RCTs) using grouped administrative data to help improve data access. The focus is on design-based estimators, derived using the building blocks of experiments, that are conducive to grouped data for a wide range of RCT designs, including clustered and…

Use of multivariate analysis (e.g., multivariate analysis of variance) is common when normally distributed outcomes are collected in intervention research. However, when mixed responses--a set of normal and binary outcomes--are collected, standard multivariate analyses are no longer suitable. While mixed responses are often obtained in…

This paper provides a guide to calculating statistical power for the complex multilevel designs that are used in most field studies in education research. For multilevel evaluation studies in the field of education, it is important to account for the impact of clustering on the standard errors of estimates of treatment effects. Using ideas from…

This report examines theoretical and empirical issues related to the statistical power of impact estimates under clustered regression discontinuity (RD) designs. The theory is grounded in the causal inference and HLM modeling literature, and the empirical work focuses on commonly-used designs in education research to test intervention effects on…

This paper presents an overview of repeated measures designs, designs that measure one group over time, also called within-subjects designs. Both univariate and multivariate designs are discussed, and advantages and disadvantages of each of these designs are considered. The sphericity assumption is examined, and methods are presented to correct…

The use of repeated measures research designs is explored. Repeated measures designs are often advantageous and can be implemented in a variety of research settings. One of the main advantages in repeated measures designs is the control of subject variability. Other advantages are the reduction of error variance and economy in subject recruitment.…

Two Monte Carlo studies were conducted to examine the sensitivity of goodness of fit indexes to lack of measurement invariance at 3 commonly tested levels: factor loadings, intercepts, and residual variances. Standardized root mean square residual (SRMR) appears to be more sensitive to lack of invariance in factor loadings than in intercepts or…

An analysis of repeated measures designs is proposed that uses an empirical Bayes estimator of the covariance matrix. The proposed analysis behaves like a univariate analysis when sample size is small or sphericity nearly satisfied, but behaves like multivariate analysis when sample size is large or sphericity is strongly violated. (SLD)

This paper presents a discussion of various statistical concepts and techniques in light of two propositions. The first is that researchers need to select analytical techniques that prevent them from committing Type VI errors, which are inconsistencies between the research question and the statistical analysis. The second is that many statistical…

Repeated measures experimental designs, often referred to as "within-subjects" designs, offer researchers opportunities to study research effects while "controlling" for subjects. These designs offer greater statistical power relative to sample size. However, threats to internal validity such as carryover or practice effects…

Examines the recommendations of H. Keselman, K. Carriere, and L. Lix (1993) regarding choice of sample size for obtaining robust tests of the repeated measures main and interaction hypotheses in a one Between-Subjects by one Within- Subjects design with a Welch-James type multivariate test when covariance matrices are heterogeneous. (SLD)

A methodology is presented for minimizing the mean error variance-covariance component in studies with resource constraints. The method is illustrated using a one-facet multivariate design. Extensions to other designs are discussed. (SLD)

Benefits and limitations of structural equation models for multivariate experiments with incomplete data are presented. Examples from studies of latent variable path models of cognitive performance illustrate analyses with latent variables, omitted variables, randomly missing data, and nonrandomly missing data. (SLD)

PhD and EdD dissertations were compared for research design, statistics, target populations, significance of results, age of subjects, and other characteristics. Analysis of 1,007 PhD and 960 EdD dissertations found PhD dissertations had more multivariate statistics and wider generalizability. EdD dissertations were more prevalent in educational…