We present an aggregation scheme that increases power in randomized controlled trials and quasi-experiments when the intervention possesses a robust and well-articulated theory of change. Intervention studies using longitudinal data often include multiple observations on individuals, some of which may be more likely to manifest a treatment effect…

Background: Efficacy trials in education often possess a motivating theory of change: how and why should the desired improvement in outcomes occur as a consequence of the intervention? In scenarios with repeated measurements, certain subgroups may be more or less likely to manifest a treatment effect; the theory of change (TOC) provides guidance…

This paper describes the context, processes and issues experienced over 5 years in which a RCT was carried out to evaluate a programme for children aged 7-8 who were struggling with their reading. Its specific aim is to illuminate questions about the design of complex teaching approaches and their evaluation using an RCT. This covers the early…

This paper presents evidence confirming the validity of the RD design in the Reading Recovery study by examining the ability of the RD design to replicate the 1st grade results observed in the original i3 RCT focused on short-term impacts. Over 1,800 schools participated in the RD study over all four cohort years. The RD design used cutoff-based…

Educational stakeholders are keen to know the magnitude and importance of different interventions. However, the way evidence is communicated to support understanding of the effectiveness of an intervention is controversial. Typically studies in education have used the standardised mean difference as a measure of the impact of interventions. This…

RAND Europe evaluated the implementation of a technology-based intervention that made use of the Microsoft Immersive Reader (IR) in UK classrooms for eight- and nine-year-old pupils. This is the first evaluation of this intervention that uses a randomised control trial (RCT) design, following promising findings related to the use of IR from a…

This report summarizes the results of a cluster-randomized field trial that estimated the effect of "BURST[R]: Reading" on primary grades students' early literacy achievement. BURST is a widely adopted supplemental reading program designed for use with students struggling to acquire early literacy skills and is meant to provide…

School districts are increasingly adopting technology-based resources in an attempt to improve student achievement. This paper reports the two-year results from randomized control trial of Achieve3000 in the Wake County Public School System (WCPSS) in Raleigh, North Carolina. Achieve3000 is an early literacy program that differentiates non-fiction…

This article illustrates the value of large-scale impact evaluations with counterfactual components. It begins by exploring the limitations of small-scale impact studies, which do not allow reliable inference to a wider population or which do not use valid comparison groups. The paper then describes the design features of a recent large-scale…

In a common approach for scaling up effective educational practice, schools adopt evidence-based programs to be implemented with fidelity. An alternative approach assumes that programs should be adapted to local contexts. In this randomized trial of a reading intervention, we study a scaffolded sequence of implementation in which schools first…

This multiyear scale-up effectiveness study of Open Court Reading (OCR) involved approximately 4,500 students and more than 1,000 teachers per year in Grades K-5 from 49 elementary schools in seven districts across the country. Using a school-level cluster randomized trial design, we assessed the implementation and effectiveness of Open Court…

Background: Many school-based interventions are being delivered in the absence of evidence of effectiveness (Snowling & Hulme, 2011, "Br. J. Educ. Psychol., 81," 1). Aims: This study sought to address this oversight by evaluating the effectiveness of the commonly used the Lexia Reading Core5 intervention, with 4- to 6-year-old pupils…

This mixed-method study examined the implementation and the challenges to implementation for participants in randomized controlled trials of two school-based programs for early adolescents: the Tier One Word Generation (WG) program, and the Tier Two Strategic Adolescent Reading Intervention (STARI). Levels of implementation for WG and STARI varied…

The purpose of this report is to present the results of a two-year randomized control trial evaluation of the SPARK literacy program. SPARK is an early grade literacy program developed by Boys & Girls Clubs of Greater Milwaukee. In 2010, SPARK was awarded an Investing in Innovation (i3) Department of Education grant to further develop the…

Accelerated Reader™ is a computerized supplementary reading program that provides guided reading instruction to students in grades K-12. It aims to improve students' reading skills through reading practice and by providing frequent feedback on students' progress to teachers. The Accelerated Reader™ program requires students to select and read a…