Lord's Paradox occurs when a continuous covariate is statistically controlled for and the relationship between a continuous outcome and group status indicator changes in both magnitude and direction. This phenomenon poses a challenge to the notion of evidence-based policy, where data are supposed to be self-evident. We examined 50 effect size…

Cost-effectiveness analysis is a widely used educational evaluation tool. The randomized controlled trials that aim to evaluate the cost-effectiveness of the treatment are commonly referred to as randomized cost-effectiveness trials (RCETs). This study provides methods of power analysis for two-level multisite RCETs. Power computations take…

Professional development for teachers is regarded as one of the principal pathways through which we can understand and cultivate effective teaching and improve student outcomes. A critical component of studies that seek to improve teaching through professional development is the detailed assessment of the intermediate teacher development processes…

Given the natural hierarchical structure in school-setting data, multilevel modeling (MLM) has been widely employed in education research using a number of different statistical software packages. The purpose of this article is to review a recent feature of Stat-JR, the statistical analysis assistants (SAAs) embedded in Stat-JR (Version 1.0.5),…

The use of advanced quantitative methods within mixed methods research has been investigated in a limited capacity. In particular, hierarchical linear models are a popular approach to account for multilevel data, such as students within schools, but its use and value as the quantitative strand in a mixed methods study remains unknown. This article…

The purpose of this study is to disseminate the results of recent advances in statistical power analyses with regard to multilevel mediation and its implementation in the PowerUp!-Mediator software. The authors first focus on the conceptual and statistical differences among common asymptotic, component-wise, and resampling-based tests of mediation…

To plan cluster-randomized trials with sufficient statistical power to detect intervention effects on student achievement, researchers need multilevel design parameters, including measures of between-classroom and between-school differences and the amounts of variance explained by covariates at the student, classroom, and school level. Previous…

Variance heterogeneity is a common feature of educational data when treatment differences expressed through means are present, and often reflects a treatment by subject interaction with respect to an outcome variable. Identifying variables that account for this interaction can enhance understanding of whom a treatment does and does not benefit in…

Researchers designing multisite and cluster randomized trials of educational interventions will usually conduct a power analysis in the planning stage of the study. To conduct the power analysis, researchers often use estimates of intracluster correlation coefficients and effect sizes derived from an analysis of survey data. When there is…

Surveys are frequently used to inform consequential decisions about teachers, policies, and programs. Consequently, it is important to understand the validity of these instruments. This study assesses the validity of measures of instruction captured by an annual survey by comparing survey data with those of a validated daily log. The two…

Multiple imputation is widely accepted as the method of choice to address item-nonresponse in surveys. However, research on imputation strategies for the hierarchical structures that are typically found in the data in educational contexts is still limited. While a multilevel imputation model should be preferred from a theoretical point of view if…

When fitting hierarchical regression models, maximum likelihood (ML) estimation has computational (and, for some users, philosophical) advantages compared to full Bayesian inference, but when the number of groups is small, estimates of the covariance matrix (S) of group-level varying coefficients are often degenerate. One can do better, even from…

When fitting hierarchical regression models, maximum likelihood (ML) estimation has computational (and, for some users, philosophical) advantages compared to full Bayesian inference, but when the number of groups is small, estimates of the covariance matrix [sigma] of group-level varying coefficients are often degenerate. One can do better, even…

Measures of variability are successfully used in predictive modeling in research areas outside of education. This study examined how standard deviations can be used to address research questions not easily addressed using traditional measures such as group means based on index variables. Student survey data were obtained from the Organisation for…

Cluster randomized experiments are ubiquitous in modern education research. Although a variety of modeling approaches are used to analyze these data, perhaps the most common methodology is a normal mixed effects model where some effects, such as the treatment effect, are regarded as fixed, and others, such as the effect of group random assignment…