Background/Context: In meta-analysis examining educational intervention, characterizing heterogeneity and exploring the sources of variation in synthesized effects have become increasingly prominent areas of interest. When combining results from a collection of studies, statistical dependency among their effects size estimates will arise when a…

Background: Education scientists are increasingly interested in constructing interventions that are adaptive over time to suit the evolving needs of students, classrooms, or schools. Such "adaptive interventions" (also referred to as dynamic treatment regimens or dynamic instructional regimes) determine which treatment should be offered…

Background: Multilevel randomized controlled trials (MRCTs) have been widely used to evaluate the causal effects of educational interventions. Traditionally, educational researchers and policymakers focused on the average treatment effects (ATE) of the intervention. Recently there has been an increasing interest in evaluating the heterogeneity of…

Research Context: In educational research, "context effects" are often of inferential interest to researchers as well as of evaluative interest to policymakers. While student education outcomes likely depend on individual-level influences like individual academic achievement, school contexts may also make a difference. Such questions are…

Background: Regression Discontinuous Design (RDD) is widely used in educational studies. Through RDD, researchers can obtain unbiased results when Randomized Experimental Design (RED) is inaccessible. Compared to RED, the RDD only requires a cut score variable (continuous) and a cutoff value to assign students to the treatment or control groups.…

Background: Counts are familiar outcomes in education research settings, including those involving tests of interventions. Clustered data commonly occur in education research studies, given that data are often collected from students within classrooms or schools. There is a wide array of distributions and models that can be used for clustered…

The purpose of this study is to disseminate the results of recent advances in statistical power analyses with regard to multilevel mediation and its implementation in the PowerUp!-Mediator software. The authors first focus on the conceptual and statistical differences among common asymptotic, component-wise, and resampling-based tests of mediation…

Background/Context: Chronic absenteeism is a serious problem that has been linked to lower academic achievement, diminished socioemotional skills, and an increased likelihood of high school dropout (Allensworth et al., 2021; Gottfried, 2014). As a result, many schools have begun to embrace early warning systems (EWS) as a tool to identify and flag…

Current practice for conducting power analyses in hierarchical trials using survey based ICC and effect size estimates may be misestimating power because ICCs are not being adjusted to account for treatment effect heterogeneity. Results presented in Table 1 show that the necessary adjustments can be quite large or quite small. Furthermore, power…

Like studies focused on the detection of a main or total effect, a principal consideration in the design of studies examining mediation is the power with which mediation effects can be detected if they exist (e.g., Raudenbush, 1997). However, unlike studies concerning the detection of main effects, literature has not established power formulas for…

Single-case designs are a class of research designs for evaluating intervention effects on individual cases. The designs are widely applied in certain fields, including special education, school psychology, clinical psychology, social work, and applied behavior analysis. The multiple baseline design (MBD) is the most frequently used single-case…

Randomized experiments are generally considered to provide the strongest basis for causal inferences about cause and effect. Consequently randomized field trials have been increasingly used to evaluate the effects of education interventions, products, and services. Populations of interest in education are often hierarchically structured (such as…

Modeling the distribution of site level effects is an important problem, but it is also an incredibly difficult one. Current methods rely on distributional assumptions in multilevel models for estimation. There it is hoped that the partial pooling of site level estimates with overall estimates, designed to take into account individual variation as…

Cluster randomized experiments are ubiquitous in modern education research. Although a variety of modeling approaches are used to analyze these data, perhaps the most common methodology is a normal mixed effects model where some effects, such as the treatment effect, are regarded as fixed, and others, such as the effect of group random assignment…

Decades of research evidence have consistently suggested teachers are the most important in-school factor related to student learning and achievement. Being taught by an effective teacher has important consequences for students' academic outcomes as well as longer-term impacts on postsecondary success and lifetime earnings. Yet how to measure…