A randomized experiment that measures the impact of a social policy in a sample of the population reveals whether the policy will work on average with universal application. An experiment that includes only the subset of the population that volunteers for the intervention generates narrower "proof-of-concept" evidence of whether the…

Background/Context: When a researcher tests an educational program, product, or policy in a randomized controlled trial (RCT) and detects a significant effect on an outcome, the intervention is usually classified as something that "works." When the expected effects are not found, however, there is seldom an orderly and transparent…

The organisers of the 2012 London Olympics have endeavoured explicitly to use the Games to inspire a generation. This is nothing short of putting the main claim of Olympism to the test, but surprisingly the Inspire project has received virtually no scholarly scrutiny. Using an educationally-informed view of inspiration, this paper interrogates the…

Respondent incentives are widely used to increase response rates, but their effect on nonresponse bias has not been researched as much. To contribute to the research, we analyze an incentive experiment embedded within the third wave of the German household panel survey "Panel Labor Market and Social Security" conducted by the German…

Though retrieving information typically results in improved memory on a subsequent test (the testing effect), Peterson and Mulligan (2013) outlined the conditions under which retrieval practice results in poorer recall relative to restudy, a phenomenon dubbed the "negative testing effect." The item-specific-relational account proposes…

This study was designed to assess the impact of a decision-making curriculum (PEER-DM) on the social peer relationship knowledge and self-protective decision-making skills of adolescents with disabilities in hypothetical situations involving negative peer pressure. A randomized design was used to assign students with disabilities from…

Multisite trials, in which individuals are randomly assigned to alternative treatment arms within sites, offer an excellent opportunity to estimate the cross-site average effect of treatment assignment (intent to treat or ITT) "and" the amount by which this impact varies across sites. Although both of these statistics are substantively…

The quality of mixed methods systematic reviews relies on the quality of primary-level studies. The synthesis of qualitative evidence and the recent development of synthesizing mixed methods studies hold promise, but also pose challenges to evidence synthesis.

Optimal levels of treatment fidelity, a critical moderator of intervention effectiveness, are often difficult to sustain in applied settings. It is unknown whether performance feedback, a widely researched method for increasing educators' treatment fidelity, is an evidence-based practice. The purpose of this review was to evaluate the current…

Despite guidelines and repeated calls from the literature, statistical mediation analysis in youth treatment outcome research is rare. Even more concerning is that many studies that "have" reported mediation analyses do not fulfill basic requirements for mediation analysis, providing inconclusive data and clinical implications. As a result, after…

The purposes of this paper are to (a) propose an operational standard for defining a "practice," (b) encourage development of professional standards for visual and statistical analysis of single-case research, and (c) propose a standard for using single-case research results to identify practices that are "evidence-based." These topics are not new…

In order to better assist practitioners and better serve persons with autism spectrum disorders (ASD) and their families, it is vital for professionals to systematically evaluate the existing body of literature and synthesize its scientific evidence, so that the efficacy of research can be translated to evidence-based practices (EBPs) (Wheeler,…

In partnership with Joint School District 2 in Meridian, Idaho, this theory-driven study assessed the impact of Keeping Learning on Track® (KLT™), a professional development program pioneered by Dylan Wiliam and his colleagues at the Educational Testing Service (ETS). A team of Northwest Evaluation Association™ (NWEA™) researchers surveyed…

Purpose: Single-subject experimental designs (SSEDs) represent an important tool in the development and implementation of evidence-based practice in communication sciences and disorders. The purpose of this article is to review the strategies and tactics of SSEDs and their application in speech-language pathology research. Method: The authors…

This article examines changes in the research design, sample size, and precision between the planning phase and implementation phase of group randomized trials (GRTs) funded by the Institute of Education Sciences. Thirty-eight GRTs funded between 2002 and 2006 were examined. Three studies revealed changes in the experimental design. Ten studies…