Data extraction is a crucial, yet labor-intensive and error-prone part of evidence synthesis. To date, efforts to harness machine learning for enhancing efficiency of the data extraction process have fallen short of achieving sufficient accuracy and usability. With the release of large language models (LLMs), new possibilities have emerged to…

When randomized control trials are not possible, quasi-experimental methods often represent the gold standard. One quasi-experimental method is difference-in-difference (DiD), which compares changes in outcomes before and after treatment across groups to estimate a causal effect. DiD researchers often use fairly exhaustive robustness checks to…

A linear model is often used to find the effect of a binary treatment D on a noncontinuous outcome Y with covariates X. Particularly, a binary Y gives the popular "linear probability model (LPM)," but the linear model is untenable if X contains a continuous regressor. This raises the question: what kind of treatment effect does the…

Multi-site randomized controlled trials (RCTs) provide unbiased estimates of the average impact in the study sample. However, their ability to accurately predict the impact for individual sites outside the study sample, to inform local policy decisions, is largely unknown. To extend prior research on this question, we analyzed six multi-site RCTs…

Plausible competing developmental models show similar or identical structural equation modeling model fit indices, despite making very different causal predictions. One way to help address this problem is incorporating outside information into selecting among models. This study attempted to select among developmental models of children's early…

Background: Network meta-analysis (NMA) of survival data with a multidimensional treatment effect has been introduced as an alternative to NMA based on the proportional hazards assumption. However, these flexible models have some limitations, such as the use of an approximate likelihood based on discrete hazards, rather than a likelihood for…

The present study reports findings from a school-level randomized controlled trial (RCT) of the Double Check cultural responsivity and student engagement model. Consistent with the focus of this special issue on the conceptual and methodological advances for understanding contextual, identity, and cultural effects in intervention research, we…

We provide evidence about college financial aid from an eight-year randomized trial where high school ninth graders received a $12,000 merit-based grant offer. The program was designed to be free of tuition/fees at community colleges and substantially lower the cost of four-year colleges. During high school, it increased students' college…

Despite the increase in federal tiered evidence initiatives that require the use of rigorous evaluation designs, such as randomized experiments, there has been limited guidance in the evaluation literature on practical strategies to implement such studies successfully. This paper provides lessons learned in executing experiments in applied…

The gold-standard for evaluating the effect of an educational intervention on student outcomes is running a randomized controlled trial (RCT). However, RCTs may often be small due to logistical considerations, and resulting treatment effect estimates may lack precision. Recent methods improve experimental precision by incorporating information…

Time-course model-based network meta-analysis (MBNMA) has been proposed as a framework to combine treatment comparisons from a network of randomized controlled trials reporting outcomes at multiple time-points. This can explain heterogeneity/inconsistency that arises by pooling studies with different follow-up times and allow inclusion of studies…

There has been increased interest in recent years regarding the utility of imported universal prevention and promotion (P&P) programmes in UK schools, many of which have a coaching model attached. However, there have been relatively few studies exploring the cultural transferability and social validity of these models, even though evidence…

Network meta-analysis (NMA) combines direct and indirect evidence from trials to calculate and rank treatment estimates. While modelling approaches for continuous and binary outcomes are relatively well developed, less work has been done with time-to-event outcomes. Such outcomes are usually analysed using Cox proportional hazard (PH) models.…

We address the task of determining, from statistical averages alone, whether a population under study consists of several subpopulations, unknown to the investigator, each responding to a given treatment markedly differently. We show that such determination is feasible in three cases: (1) randomized trials with binary treatments, (2) models where…

Low- and middle-income countries have made significant progress getting children into school, but student learning and achievement are often dreadfully low (Berry, Barnett, & Hinton, 2015; Pritchett, 2013). Approximately 250 million children across the world are not acquiring basic reading and math skills, even though about half have spent at…