The present simulation study examined the statistical properties (namely, Type I error and statistical power) of various novel randomized single-case multiple-baseline designs and associated randomized-test analyses for comparing the A- to B-phase immediate abrupt outcome changes in two independent intervention conditions. It was found that with…

Multiple imputation (MI) is a popular method for handling missing data. In education research, it can be challenging to use MI because the data often have a clustered structure that need to be accommodated during MI. Although much research has considered applications of MI in hierarchical data, little is known about its use in cross-classified…

Analyses that reveal how treatment effects vary allow researchers, practitioners, and policymakers to better understand the efficacy of educational interventions. In practice, however, standard statistical methods for addressing heterogeneous treatment effects (HTE) fail to address the HTE that may exist "within" outcome measures. In…

Background: Efficacy trials in education often possess a motivating theory of change: how and why should the desired improvement in outcomes occur as a consequence of the intervention? In scenarios with repeated measurements, certain subgroups may be more or less likely to manifest a treatment effect; the theory of change (TOC) provides guidance…

Intervention researchers often use curriculum-based measurement of reading fluency (CBM-R) with a brief experimental analysis (BEA) to identify an effective intervention for individual students. The current study synthesized data from 22 studies that used CBM-R data within a BEA by computing the standard error of measure (SEM) for the median data…

The "2016 Knowledge for Teaching Early Elementary Mathematics" (2016 K-TEEM) test measures teachers' mathematical knowledge for teaching early elementary mathematics. The 2016 K-TEEM is the third version of the K-TEEM (Schoen, Bray, Wolfe, Tazaz, & Nielsen, 2017). In this report, we present results of the first large-scale field test…

There has been a growing need felt by practitioners, researchers, and evaluators to obtain a common measure of science engagement that can be used in different out-of-school time (OST) science learning settings. We report on the development and validation of a novel 10-item self-report instrument designed to measure, communicate, and ultimately…

Program evaluation often suffers due to time constraints, imperfect instruments, incomplete data, and the need to report standardized metrics. This article about the evaluation process for the Wi$eUp financial education program showcases the difficulties inherent in evaluation and suggests best practices for assessing program effectiveness. We…

We study a popular dual enrollment program in Washington State, "Running Start" using a new administrative database that links high school and postsecondary data. Conditional on prior high school performance, we find that students participating in Running Start are more likely to attend any college but less likely to attend four-year…

We study a popular dual enrollment program in Washington State, "Running Start" using a new administrative database that links high school and postsecondary data. Conditional on prior high school performance, we find that students participating in Running Start are more likely to attend any college but less likely to attend four-year…

In a pretest-posttest cluster-randomized trial, one of the methods commonly used to detect an intervention effect involves controlling pre-test scores and other related covariates while estimating an intervention effect at post-test. In many applications in education, the total post-test and pre-test scores that ignores measurement error in the…

Multilevel modeling (MLM) is frequently used to detect group differences, such as an intervention effect in a pre-test--post-test cluster-randomized design. Group differences on the post-test scores are detected by controlling for pre-test scores as a proxy variable for unobserved factors that predict future attributes. The pre-test and post-test…

Equivocal evidence exists regarding the degree of cognitive stability and prevalence of cognitive impairment in very late life. The objective of the current study was to examine mental status performance and change over time within a sample of Iowa centenarians. The baseline sample consisted of 152 community-dwelling and institutionalized…

Randomized controlled trials (RCTs) are considered the "gold standard" for evaluating an intervention's effectiveness. Recently, the federal government has placed increased emphasis on the use of opportunistic experiments. A key criterion for conducting opportunistic experiments, however, is that there is relatively easy access to data…

Researchers have become increasingly interested in programs' main and interaction effects of two variables (A and B, e.g., two treatment variables or one treatment variable and one moderator) on outcomes. A challenge for estimating main and interaction effects is to eliminate selection bias across A-by-B groups. I introduce Rubin's causal model to…