Evaluators of education interventions increasingly need to design studies to detect impacts much smaller than the 0.20 standard deviations that Cohen (1988) characterized as "small." For example, an evaluation of Response to Intervention from the Institute of Education Sciences (IES) detected impacts ranging from 0.13 to 0.17 standard…

As evidence-based reform becomes increasingly important in educational policy, it is becoming essential to understand how research design might contribute to reported effect sizes in experiments evaluating educational programs. The purpose of this study was to examine how methodological features such as types of publication, sample sizes, and…

With research indicating substantial differences among teachers in terms of their effectiveness (Nye, Konstantopoulous, & Hedges, 2004), a major focus of recent research in education has been on improving teacher quality through professional development (Desimone, 2009; Institute of Educations Sciences [IES], 2012; Measures of Effective…

Improving the quality of teacher preparation is an important national issue, because the quality of teaching plays such a large role in students' learning. One key indicator of students' academic success is the competence and capability of their teachers. However, the availability of well-prepared and effective teachers varies widely across the…

One purpose of education research is to develop and rigorously evaluate the effectiveness of programs for supporting students' learning and achievement. The Institute of Education Sciences has amplified that purpose (Shadish & Cook, 2009) and attempted to improve the methodological standards for conducting such work--primarily through the What…

To date, evaluation research and policy analysis have focused mainly on average program impacts and paid little systematic attention to their variation. Recently, the growing number of multi-site randomized trials that are being planned and conducted make it increasingly feasible to study "cross-site" variation in impacts. Important…

Accurate indicators of educational effectiveness are needed to advance national policy goals of raising student achievement and closing social/cultural based achievement gaps. If constructed and used appropriately, such indicators for both program evaluation and the evaluation of teacher and school performance could have a transformative effect on…

Access to Head Start has been shown to improve children's preschool experiences and school readiness on selected factors through the end of 1st grade. Two more years of follow-up, through the end of 3rd grade, can now be examined to determine whether these effects continue into the middle elementary grades. The statistical design and impact…

This presentation focuses on the quasi-experimental methods used to select comparison schools for an evaluation of a federal investing in innovation (i3) validation grant. The Northeast Tennessee College and Career Ready Consortium (NETCO) consists of 29 high schools participating in a five-year program to expand students' access to rigorous…

In this paper the authors extend recent work on mediation in a multilevel setting and on causal inference under interference among units to develop a template for the mediation analysis of group randomized educational interventions. The present work will contribute to the literature on interference, in particular on interference in the context of…

The purpose of this study is to extend the work of Spybrook and Raudenbush (2009) and examine how the research designs and sample sizes changed from the planning phase to the implementation phase in the first wave of studies funded by IES. The authors examine the impact of the changes in terms of the changes in the precision of the study from the…

The concept of "implementation fidelity" is broadly used to capture the extent to which an intervention is executed as intended by the designers of the intervention (Century, Rudnick, & Freeman, 2010; Huntley, 2005, McNaught, Tarr, & Sears, 2010, Munter, 2010). Though implementation fidelity instruments are often used to assess…

To provide greater support for students as they transition to ninth grade, the Texas Legislature funded the Texas Ninth Grade Transition and Intervention (TNGTI) grant program. TNGTI grantees implemented a variety of supports to at-risk students transitioning to ninth grade, including a summer transition program, an early warning data system, and…

The purpose of this study is to compare the efficacy of three different approaches to professional development in Earth science education: preparing teachers to "adopt" expert-"design" curricula; preparing teachers to use a principled approach to design curricula; and preparing teachers to use a principled approach to "adapt" curricula. The claims…

The current study will examine possible contextual effects relative to differences in reading comprehension performance in the state of Florida. While the Reading First (RF) Impact study examined such difference using a regression discontinuity design, the authors are primarily interested in other analytic methods that might answer different…