Randomized controlled trials have proliferated in education, in part because they provide an unbiased estimator for the causal impact of interventions. It is increasingly recognized that many such trials in education have low power to detect an effect if indeed there is one. However, it is less well known that low powered trials tend to…

Despite recent emphasis on the use of randomized control trials (RCTs) for evaluating education interventions, in most areas of education research, observational methods remain the dominant approach for assessing program effects. Over the last three decades, the within-study comparison (WSC) design has emerged as a method for evaluating the…

This multiyear scale-up effectiveness study of Open Court Reading (OCR) involved approximately 4,500 students and more than 1,000 teachers per year in Grades K-5 from 49 elementary schools in seven districts across the country. Using a school-level cluster randomized trial design, we assessed the implementation and effectiveness of Open Court…

In this longitudinal study, the cross-language transfer from second language (L2) to first language (L1) was examined among Spanish-speaking English-language learners in an English intervention (Grades 1-3) in the southwest United States. Path analysis revealed statistically significant transfers (ps < 0.05) for the treatment group from English…

The Louisiana Scholarship Program (LSP) offers publicly funded vouchers to students in low-performing schools with family income no greater than 250% of the poverty line, allowing them to enroll in participating private schools. Initially established in 2008 as a pilot program in New Orleans, the LSP was expanded statewide in 2012. This article…

I discuss three common practices that obfuscate or invalidate the statistical analysis of randomized controlled interventions in applied linguistics. These are (a) checking whether randomization produced groups that are balanced on a number of possibly relevant covariates, (b) using repeated measures ANOVA to analyze pretest-posttest designs, and…

This study evaluated two 20-week reading interventions for pupils entering secondary school with reading difficulties. The interventions were delivered by trained teaching assistants (three 35-min sessions per week). 287 pupils (ages 11-13) from 27 schools were randomly allocated to three groups: reading intervention (targeting word recognition…

Reading comprehension in the content areas is a challenge for many middle grade students. Text structure-based instruction has yielded positive outcomes in reading comprehension at all grade levels in small and large studies. The text structure strategy delivered via the web, called Intelligent Tutoring System for the Text Structure Strategy…

Reading and comprehending content area texts require learners to effectively select and encode with hierarchically strategic memory structures in order to combine new information with prior knowledge. Unfortunately, evidence from state and national tests shows that children fail to successfully navigate the reading comprehension challenges they…

This study evaluated the ability for two rhythmic rhyming programs to raise phonological awareness in the early literacy classroom. Year 1 (5-6-year-olds) from low socioeconomic status schools in Bedfordshire, learned a program of sung or spoken rhythmic rhymes, or acted as controls. The project ran with two independent cohorts (Cohort 1 N = 98,…

Early math skills are a strong predictor of later achievement for young children, not only in math, but in other domains as well. Exhibiting strong math skills in elementary school is predictive of later high school completion and college attendance. To that end, the Making Pre-K Count and High 5s studies set out to rigorously assess whether…

To date, evaluation research and policy analysis have focused mainly on average program impacts and paid little systematic attention to their variation. Recently, the growing number of multi-site randomized trials that are being planned and conducted make it increasingly feasible to study "cross-site" variation in impacts. Important…

We surveyed researchers in the health and exercise sciences to explore different areas and magnitudes of bias in researchers' decision making. Participants were presented with scenarios (testing a central hypothesis with p = 0.06 or p = 0.04) in a random order and surveyed about what they would do in each scenario. Participants showed significant…

This paper capitalizes on the addition of geocodes for Head Start centers in which children were randomly assigned to address questions about the role of neighborhood characteristics in moderating impacts of assignment to the Head Start program. Researchers explore the extent to which impacts of assignment to Head Start on outcomes for children…

Functional behavioral assessment (FBA) is an empirically supported intervention associated with decreasing problem behavior and increasing appropriate behavior. To date, few studies have examined multimedia approaches to FBA training. This paper provides the outcomes of a randomized controlled trial across three university sites and evaluates…