Background: Efficacy trials in education often possess a motivating theory of change: how and why should the desired improvement in outcomes occur as a consequence of the intervention? In scenarios with repeated measurements, certain subgroups may be more or less likely to manifest a treatment effect; the theory of change (TOC) provides guidance…

School districts are increasingly adopting technology-based resources in an attempt to improve student achievement. This paper reports the two-year results from randomized control trial of Achieve3000 in the Wake County Public School System (WCPSS) in Raleigh, North Carolina. Achieve3000 is an early literacy program that differentiates non-fiction…

Nearly one-quarter of U.S. eighth graders score below basic on national assessments of reading (NCES, 2013) and are poorly equipped for the reading demands of secondary school. Struggling adolescent readers cannot summarize a simple passage, use context to determine word meanings, and have difficulties making text-based inferences. In addition,…

The purpose of this report is to present the results of a two-year randomized control trial evaluation of the SPARK literacy program. SPARK is an early grade literacy program developed by Boys & Girls Clubs of Greater Milwaukee. In 2010, SPARK was awarded an Investing in Innovation (i3) Department of Education grant to further develop the…

Recognizing that standardized instructional treatments may not be equally effective across contexts, a growing number of scholars are arguing that the strategy of adopting packaged programs and striving to implement them faithfully is neither realistic nor desirable. An alternative view is that scaling up educational treatments requires balancing…

The Fidelity of Implementation (FOI) study that is the focus of this report was conducted as a component of a scale-up effectiveness trial of the SRA/McGraw-Hill Open Court Reading program. The overall purpose of the FOI study was to support and provide context for findings from the larger experimental impact study of Open Court Reading (OCR). To…

As part of the 2010 economic stimulus, a $55 million "Investing in Innovation" (i3) grant from the US Department of Education was awarded to scale up Reading Recovery across the nation. This paper presents the final round of results from the large-scale, mixed methods randomized evaluation of the implementation and impacts of Reading…

This study addresses the effectiveness of a widely used core reading program that reflects the research-based practices recommended by the National Reading Panel. This and other similar programs are increasingly used to prevent reading difficulties and ensure that all children are reading at or above grade level by the end of third grade.…

Nationally, two-thirds of high school students are unable to read and comprehend complex academic materials, think critically about texts, and synthesize information from multiple sources, or communicate what they have learned. Without a substantial change in their academic literacy, U.S. high school students face continued academic problems in…

Interest in variation in program impacts--How big is it? What might explain it?--has inspired recent work on the analysis of data from multi-site experiments. One critical aspect of this problem involves the use of random or fixed effect estimates to visualize the distribution of impact estimates across a sample of sites. Unfortunately, unless the…

The tutoring component of the Investing in Innovations (i3) funded Milwaukee Community Literacy Project (MCLP) is based on the Reading Recovery program. Reading Recovery focuses on in-school tutoring with lesson plans written, and assessments analyzed, by the tutors themselves. The objective of this study is to measure the impact of the MCLP on…

This paper considers one challenge from recent emphasis on rigorous research for decision-making in education: the interpretation of non-replicated findings. Under Clinton and Bush, the federal government called for and funded highly rigorous research. The "gold-standard" for research was randomized-control trials (RCTs), which…

As evidence-based reform becomes increasingly important in educational policy, it is becoming essential to understand how research design might contribute to reported effect sizes in experiments evaluating educational programs. The purpose of this study was to examine how methodological features such as types of publication, sample sizes, and…