Multisite trials that randomize individuals (e.g., students) within sites (e.g., schools) or clusters (e.g., teachers/classrooms) within sites (e.g., schools) are commonly used for program evaluation because they provide opportunities to learn about treatment effects as well as their heterogeneity across sites and subgroups (defined by moderating…

Background: Randomized controlled trials (RCTs), or experiments, are the gold standard for intervention evaluation. However, the main appeal of RCTs--the clean identification of causal effects--can be compromised by interference, when one subject's actions can influence another subject's behavior or outcomes. In this paper, we formalize and study…

Randomized controlled trials have proliferated in education, in part because they provide an unbiased estimator for the causal impact of interventions. It is increasingly recognized that many such trials in education have low power to detect an effect if indeed there is one. However, it is less well known that low powered trials tend to…

In cluster randomized evaluations, a treatment or intervention is randomly assigned to a set of clusters each with constituent individual units of observations (e.g., student units that attend schools, which are assigned to treatment). One consideration of these designs is how many units are needed per cluster to achieve adequate statistical…

In cluster randomized evaluations, a treatment or intervention is randomly assigned to a set of clusters each with constituent individual units of observations (e.g., student units that attend schools, which are assigned to treatment). One consideration of these designs is how many units are needed per cluster to achieve adequate statistical…

As reporting of effect sizes in evaluation studies has proliferated, researchers and consumers of research need tools for interpreting or benchmarking the magnitude of those effect sizes that are relevant to the intervention, target population, and outcome measure being considered. Similarly, researchers planning education studies with social and…

Evidence-based education aims to support policy makers choosing between potential interventions. This rarely involves considering each in isolation; instead, sets of evidence regarding many potential policy interventions are considered. Filtering a set on any quantity measured with error risks the "winner's curse": conditional on…

Cost-effectiveness analysis is a widely used educational evaluation tool. The randomized controlled trials that aim to evaluate the cost-effectiveness of the treatment are commonly referred to as randomized cost-effectiveness trials (RCETs). This study provides methods of power analysis for two-level multisite RCETs. Power computations take…

We present an aggregation scheme that increases power in randomized controlled trials and quasi-experiments when the intervention possesses a robust and well-articulated theory of change. Intervention studies using longitudinal data often include multiple observations on individuals, some of which may be more likely to manifest a treatment effect…

The purpose of this paper is to present results of recent advances in power analyses to detect the moderator effects in Cluster Randomized Trials (CRTs). This paper focus on demonstration of the software PowerUp!-Moderator. This paper provides a resource for researchers seeking to design CRTs with adequate power to detect the moderator effects of…

Researchers are often interested in whether the effects of an intervention differ conditional on individual- or group-moderator variables such as children's characteristics (e.g., gender), teacher's background (e.g., years of teaching), and school's characteristics (e.g., urbanity); that is, the researchers seek to examine for whom and under what…

The purpose of this study is to propose a general framework for power analyses to detect the moderator effects in two- and three-level cluster randomized trials (CRTs). The study specifically aims to: (1) develop the statistical formulations for calculating statistical power, minimum detectable effect size (MDES) and its confidence interval to…

Purpose: This report investigates whether parent-reported stuttering severity ratings (SRs) provide similar estimates of effect size as percentage of syllables stuttered (%SS) for randomized trials of early stuttering treatment with preschool children. Method: Data sets from 3 randomized controlled trials of an early stuttering intervention were…

Current practice for conducting power analyses in hierarchical trials using survey based ICC and effect size estimates may be misestimating power because ICCs are not being adjusted to account for treatment effect heterogeneity. Results presented in Table 1 show that the necessary adjustments can be quite large or quite small. Furthermore, power…

Purpose: This study aimed to investigate the effects of water-based aerobic training on the lipid profile and lipoprotein lipase (LPL) levels in premenopausal women with dyslipidemia. Method: Forty women were randomly assigned to: aquatic training (WA; n = 20) or a control group (CG; n = 20). The WA group underwent 12 weeks of water-based interval…