Background: As the body of scientific evidence about what works in education grows, so does the need to effectively communicate that evidence to policy-makers and practitioners. Websites and clearinghouses such as the "What Works Clearinghouse" (WWC), "Evidence for ESSA," "Blueprints for Healthy Youth Development,"…

Background: Traditional survey efforts to gather outcome data at scale have significant limitations, including cost, time, and respondent burden. This pilot study explored new and innovative large-scale methods of collecting and validating data from publicly available sources. Taking advantage of emerging data science techniques, we leverage…

To date, evaluation research and policy analysis have focused mainly on average program impacts and paid little systematic attention to their variation. Recently, the growing number of multi-site randomized trials that are being planned and conducted make it increasingly feasible to study "cross-site" variation in impacts. Important…

The authors present work towards automatically assessing data collection behaviors as middle school students engage in inquiry within a physics microworld. In this study, the authors used machine learned models that can detect when students test their articulated hypotheses, design controlled experiments, and engage in planning behaviors using…

In this paper, the authors explore the use of learning analytics as a method for easing the cognitive demands on teachers implementing the HALF instructional model. Learning analytics has been defined as "the measurement, collection, analysis and reporting of data about learners and their contexts for the purposes of understanding and…

The concept of "implementation fidelity" is broadly used to capture the extent to which an intervention is executed as intended by the designers of the intervention (Century, Rudnick, & Freeman, 2010; Huntley, 2005, McNaught, Tarr, & Sears, 2010, Munter, 2010). Though implementation fidelity instruments are often used to assess…

Educators have long used test scores to make educational decisions, but only within the last decade has the availability of data been systematic (Abelman, Elmore, Even, Kenyon, & Marshall, 1999). In recent years, interest has spiked in data-driven decision making in education (Marsh, Pane, & Hamilton, 2006). With technological advances and…

This paper illustrates how information collected through interviews can develop a richer understanding of the assignment mechanism, which can result in more plausible causal effect estimates from observational studies and provides a roadmap for sensitivity analysis. Focusing on the issue of assignment to algebra in 8th grade, the author shows how…

This paper outlines a method of determining the achieved relative strength (ARS) of an intervention by calculating the extent to which core intervention components are present in both the treatment and control condition. Using data from several examples the authors demonstrate how to calculate ARS indices, and subsequently how to incorporate them…