This article discusses estimation of average treatment effects for randomized controlled trials (RCTs) using grouped administrative data to help improve data access. The focus is on design-based estimators, derived using the building blocks of experiments, that are conducive to grouped data for a wide range of RCT designs, including clustered and…

Estimating the effects of randomized experiments and, by extension, their mediating mechanisms, is often complicated by treatment noncompliance. Two estimation methods for causal mediation in the presence of noncompliance have recently been proposed, the instrumental variable method (IV-mediate) and maximum likelihood method (ML-mediate). However,…

This article describes the analysis of regression-discontinuity designs (RDDs) using the R packages rdd, rdrobust, and rddtools. We discuss similarities and differences between these packages and provide directions on how to use them effectively. We use real data from the Carolina Abecedarian Project to show how an analysis of an RDD can be…

When cluster randomized experiments are analyzed as if units were independent, test statistics for treatment effects can be anticonservative. Hedges proposed a correction for such tests by scaling them to control their Type I error rate. This article generalizes the Hedges correction from a posttest-only experimental design to more common designs…

Mediation is a causal process that evolves over time. Thus, a study of mediation requires data collected throughout the process. However, most applications of mediation analysis use cross-sectional rather than longitudinal data. Another implicit assumption commonly made in longitudinal designs for mediation analysis is that the same mediation…

This article examines theoretical and empirical issues related to the statistical power of impact estimates under clustered regression discontinuity (RD) designs. The theory is grounded in the causal inference and hierarchical linear modeling literature, and the empirical work focuses on common designs used in education research to test…

This article examines theoretical and empirical issues related to the statistical power of impact estimates for experimental evaluations of education programs. The author considers designs where random assignment is conducted at the school, classroom, or student level, and employs a unified analytic framework using statistical methods from the…

Power for the improved general approximation (IGA) and the Welch-James tests of the within-subjects (trials) main effects and the within-subjects x between-subjects (groups x trials) interaction was estimated for a design with one between- and one within-subjects factor. Results show that little if any power is sacrificed by using these methods.…

Evaluation of the binomial effect size display (BESD), proposed as a format for presenting effect sizes associated with certain research, suggests that its application is limited to presenting results of 2 X 2 tables in which the overall binomial success rate is 50%. Problems with BESD use are explored. (SLD)

Paired-samples experimental designs are appropriate and widely used when there is a natural correspondence or pairing of scores. However, researchers must not fail to consider the implications of undetected correlation between supposedly independent samples in the absence of explicit pairing. (SLD)

An analysis of repeated measures designs is proposed that uses an empirical Bayes estimator of the covariance matrix. The proposed analysis behaves like a univariate analysis when sample size is small or sphericity nearly satisfied, but behaves like multivariate analysis when sample size is large or sphericity is strongly violated. (SLD)

H. Levene (1960) proposed a heuristic test for heteroscedasticity in the case of a balanced two-way layout, based on analysis of variance of absolute residuals. Conditions under which design imbalance affects the test's characteristics are identified, and a simple correction involving leverage is proposed. (SLD)

Large-sample standard errors of linear equating for the counterbalanced design are derived using the general delta method. Computer simulations found that standard errors derived without the normality assumption were more accurate than those derived with the normality assumption in a large sample with moderately skewed score distributions. (SLD)

Describes and assesses missing data methods currently used to analyze data from matrix sampling designs implemented by the National Assessment of Educational Progress. Several improved methods are developed, and these models are evaluated using an EM algorithm to obtain maximum likelihood estimates followed by multiple imputation of complete data…