Design-based methods have recently been developed as a way to analyze data from impact evaluations of interventions, programs, and policies (Freedman, 2008; Lin, 2013; Imbens and Rubin, 2015; Schochet, 2013, 2016; Yang and Tsiatis, 2001). The non-parametric estimators are derived using the building blocks of experimental designs with minimal…

Recent methods have sought to improve precision in randomized controlled trials (RCTs) by utilizing data from large observational datasets for covariate adjustment. For example, consider an RCT aimed at evaluating a new algebra curriculum, in which a few dozen schools are randomly assigned to treatment (new curriculum) or control (standard…

Objective: To evaluate different recruitment methods to enroll participants into a mHealth pilot RCT: banner ads on Facebook and OkCupid, and targeted electronic outreach (e.g., emails to community-based organizations and to professors at local colleges). Participants: Between October 2015 and May 2016, 114 college-aged Black and Latina women 18…

Background/Context: Early childhood education research has consistently found evidence of immediate, positive effects on developmental outcomes. Despite these strong initial benefits, follow-up analyses often reveal that these effects fadeout in the years following participation (e.g., Bailey et al., 2017, Clements et al., 2013; Lipsey et al.…

It is generally agreed that learning about the Holocaust has some impact on students. However, discussions about the nature and magnitude of impact tend to be intuition-based rather than evidence-based. This is exacerbated by studies giving insight into how Holocaust education is related to salient variables rather than studies which demonstrate…

This paper describes the context, processes and issues experienced over 5 years in which a RCT was carried out to evaluate a programme for children aged 7-8 who were struggling with their reading. Its specific aim is to illuminate questions about the design of complex teaching approaches and their evaluation using an RCT. This covers the early…

Randomized clinical trials underpin evidence-based clinical practice, but flaws in their conduct may lead to biased estimates of intervention effects and hence invalid treatment recommendations. The main approach to the empirical study of bias is to collate a number of meta-analyses and, within each, compare the results of trials with and without…

Teacher professional development (TPD) programs are increasingly using video recordings of teaching practice to develop teacher capacity and foster student learning. However, consensus has yet to be reached about how to utilize video recordings in TPD for physical education (PE) teachers. We used semi-structured interviews and evaluations of PE…

Background/Context: When a researcher tests an educational program, product, or policy in a randomized controlled trial (RCT) and detects a significant effect on an outcome, the intervention is usually classified as something that "works." When the expected effects are not found, however, there is seldom an orderly and transparent…

Across the evidence-based policy and practice (EBPP) community, including education, randomised controlled trials (RCTS) rank as the most "rigorous" evidence for causal conclusions. This paper argues that that is misleading. Only narrow conclusions about study populations can be warranted with the kind of "rigour" that RCTs…

Purpose: The aim of this study was to investigate secondary voice outcomes and detraining effects of 2 head and neck strengthening exercises, which have been used in swallowing rehabilitation, that is, the head-lift exercise (HLE) and the recline exercise (RE), in healthy older adults. Method: Twenty-seven healthy older adults (between 60 and 85…

In this essay, I suggest that a slightly modified version of Freedman's formulation of the clinical equipoise requirement is justified. I begin this essay with a brief discussion of the equipoise requirement. In the second and third sections, I discuss several objections to the clinical equipoise requirement as well as two attempts to justify the…

Randomized controlled trials in education are necessary to keep pace with the evidence-based practices demanded by schools and the nation. In this study, kindergarten students in a school district in Indiana were randomized on the class level: The experimental condition consisted of students utilizing a computer-adaptive reading program for 15…

Cluster Randomized Trial (CRT) designs are inherently multilevel and reflect the hierarchical structure of schools and the wider education system. To capture this multilevel nature, CRTs are commonly analysed using multilevel (or hierarchical) linear models. It is fairly common for CRT designs and analyses to include school and individual/pupil…

This paper presents evidence confirming the validity of the RD design in the Reading Recovery study by examining the ability of the RD design to replicate the 1st grade results observed in the original i3 RCT focused on short-term impacts. Over 1,800 schools participated in the RD study over all four cohort years. The RD design used cutoff-based…