Purpose: This article provides benchmark data on within-group effect sizes from published randomized clinical trials supporting the efficacy of dialectical behavioral therapy (DBT) for borderline personality disorder (BPD) in adults aged 18-65 years. Method: Within-group effect sizes were calculated via the Glass approach and adjusted for sample…

This article provides benchmark data on within-group effect sizes from published randomized clinical trials that supported the efficacy of problem-solving therapy (PST) for depression among adults. Benchmarks are broken down by type of depression (major or minor), type of outcome measure (interview or self-report scale), whether PST was provided…

Cluster randomized experiments are now widely used to examine intervention effects in prevention science. It is meaningful to use empirical benchmarks for interpreting effect size in prevention science. The effect size (i.e., the standardized mean difference, calculated by the difference of the means between the treatment and control groups,…

Randomized experiments have long been established as the gold standard for addressing causal questions. However, experiments are not always feasible or desired, so observational methods are also needed. When multiple observations on the same variable are available, a repeated measures design may be used to assess whether a treatment administered…

The purpose of this study is to study the performance of different methods (inverse probability weighting and estimation of informative bounds) to control for differential attrition by comparing the results of different methods using two datasets: an original dataset from Portland Public Schools (PPS) subject to high rates of differential…

Researchers are increasingly using comparative interrupted time series (CITS) designs to estimate the effects of programs and policies when randomized controlled trials are not feasible. In a simple interrupted time series design, researchers compare the pre-treatment values of a treatment group time series to post-treatment values in order to…

Purpose: This article provides benchmark data on within-group effect sizes from published randomized clinical trials (RCTs) supporting the efficacy of trauma-focused cognitive behavioral therapy (TF-CBT) for traumatized children. Methods: Within-group effect-size benchmarks for symptoms of trauma, anxiety, and depression were calculated via the…

Policy-makers are frequently interested in understanding how effective a particular intervention may be for a specific (and often broad) population. In many fields, particularly education and social welfare, the ideal form of these evaluations is a large-scale randomized experiment. Recent research has highlighted that sites in these large-scale…

In 2013-14, the Wake County Public School System (WCPSS) launched Achieve3000 as a randomized controlled trial in 16 elementary schools. Achieve3000 is an early literacy program that differentiates non-fiction reading passages based on individual students' Lexile scores. Twoyear results show that Achieve3000 did not have a significant impact on…

The goal of this paper is to provide guidance for applied education researchers in using multi-level data to study the effects of interventions implemented at the school level. Two primary approaches are currently employed in observational studies of the effect of school-level interventions. One approach employs intact school matching: matching…

If children do not learn how to read in the first few years of primary school, they will struggle to complete the cycle and are at greater risk of dropping out. It is therefore crucial to identify and test interventions that have the potential of making a large impact, can be implemented quickly, and are affordable to be taken to scale by the…

The nested data structure inherent in education (i.e. students nested in schools nested in districts) makes intact school matching an appealing approach in observational studies of educational interventions and policies for both theoretical and practical purposes. This paper provides guidance to applied education researchers who are employing…

Performance-based scholarships have two main goals: (1) to give students more money for college; and (2) to provide incentives for academic progress. MDRC launched the Performance-Based Scholarship (PBS) Demonstration in 2008 to evaluate the effectiveness of these scholarships in a diverse set of states, institutions, and low-income student…

The sharp regression discontinuity design (RDD) has three key weaknesses compared to the randomized clinical trial (RCT). It has lower statistical power, it is more dependent on statistical modeling assumptions, and its treatment effect estimates are limited to the narrow subpopulation of cases immediately around the cutoff, which is rarely of…

Growing interest in teaching quality and accountability has focused attention on the need for rigorous studies and evaluations of professional development (PD) programs. However, the study of PD has been hampered by a lack of suitable instruments. The authors present data from the Teacher Knowledge Assessment System (TKAS), which was designed to…