Educational researchers advocate the use of an effect size and its confidence interval to assess the effectiveness of interventions instead of relying on a p-value, which has been blamed for lack of reproducibility of research findings and the misuse of statistics. The aim of this study is to provide a framework, which can provide direct evidence…

This article reviews important statistical methods for estimating the impact of interventions on outcomes in education settings, particularly programs that are implemented in field, rather than laboratory, settings. We begin by describing the causal inference challenge for evaluating program effects. Then four research designs are discussed that…

In the coming years, public programs will capture even more and richer data than they do now, including data from web-based tools used by participants in employment services, from tablet-based educational curricula, and from electronic health records for Medicaid beneficiaries. Program evaluators seeking to take full advantage of these data…

Educational stakeholders are keen to know the magnitude and importance of different interventions. However, the way evidence is communicated to support understanding of the effectiveness of an intervention is controversial. Typically studies in education have used the standardised mean difference as a measure of the impact of interventions. This…

The purpose of this study was to examine the effects of proactive student-success coaching, informed by predictive analytics, on student academic performance and persistence. Specifically, semester GPA and semester-to-semester student persistence were the investigated outcomes. Uniquely, the community college focused the intervention on only…

Objective: This study reevaluates the effects of a randomized controlled trial (RCT) of job search interventions on labor force participation. In the trial, various methods of job search interventions were included, of which about half were research-based JOBS (Job Opportunity and Basic Skills) interventions and the other half were practice-based,…

Background: This study investigated the long-term effectiveness of Preventure, a selective personality-targeted prevention program, in reducing the uptake of alcohol, harmful use of alcohol, and alcohol-related harms over a 3-year period. Methods: A cluster randomized controlled trial was conducted to assess the effectiveness of Preventure.…

Building on common assumptions in theories of interest and mathematics education, this experimental study examined the effect of context personalization based on individual preferences, group personalization, and example choice with preselected popular examples on middle school students' situational interest and performance in mathematics.…

Questions regarding the generalizability of results from educational experiments have been at the forefront of methods development over the past five years. This work has focused on methods for estimating the effect of an intervention in a well-defined inference population (e.g., Tipton, 2013; O'Muircheartaigh and Hedges, 2014); methods for…

Despite consensus that school absenteeism has negative consequences for children's life outcomes, until recently, little was known about the prevalence of absenteeism or its potential to moderate the impacts of school-based interventions. This study provides evidence from a randomized experiment of a preschool intervention involving 1,876 children…

Randomized experiments are commonly used to evaluate if particular interventions improve student achievement. While these experiments can establish that a treatment actually "causes" changes, typically the participants are not randomly selected from a well-defined population and therefore the results do not readily generalize. Three…

A randomized control trial was conducted to study whether providing 10th grade students with information about the returns to upper secondary and tertiary education, and a source of financial aid for tertiary education, can contribute to improve student performance. The study finds that the intervention had no effects on the probability of taking…

U.S. social programs, set up to address important problems, often fall short by funding specific models/strategies ("interventions") that are not effective. When evaluated in scientifically-rigorous studies, social interventions in K-12 education, job training, crime prevention, and other areas are frequently found ineffective or…