Observational studies are common in educational research, where subjects self-select or are otherwise non-randomly assigned to different interventions (e.g., educational programs, grade retention, special education). Unbiased estimation of a causal effect with observational data depends crucially on the assumption of ignorability, which specifies…

Despite the emphasis on data-based decision making (DBDM) in educational policy in several countries, evidence regarding the intended effect of improved student achievement is still scarce. The purpose of this study is to examine the effects of a DBDM intervention on teacher efficacy. This study incorporates an experimental longitudinal research…

Educare is a network of enhanced Early Head Start (EHS)/Head Start (HS) (birth to age 5) programs that implement innovative Research-Program Partnerships (RPPs) to engage researchers, program leaders, staff, and at times, other stakeholders in a collaborative approach to supporting data use practices for decision-making and continuous quality…

Single case designs (SCDs) are short time series that assess intervention effects by measuring units repeatedly over time both in the presence and absence of treatment. For a variety of reasons, interest in the statistical analysis and meta-analysis of these designs has been growing in recent years. This paper proposes modeling SCD data with…

Single case designs (SCDs) generally consist of a small number of short time series in two or more phases. The analysis of SCDs statistically fits in the framework of a multilevel model, or hierarchical model. The usual analysis does not take into account the uncertainty in the estimation of the random effects. This not only has an effect on the…

Single-case or single-subject experimental designs (SSED) are used to evaluate the effect of one or more treatments on a single case. Although SSED studies are growing in popularity, the results are in theory case-specific. One systematic and statistical approach for combining single-case data within and across studies is multilevel modeling. The…

Launched in 2008, the Strategic Data Project, housed at the Center for Education Policy Research at Harvard University, seeks to bridge the divide between educational research and practice in order to transform the use of data in education to improve student achievement. Through the project, the authors build robust research partnerships with…

The purpose of this project is to demonstrate the practical methods developed to utilize a dataset consisting of both multivariate and multilevel effect size data. The context for this project is a large-scale meta-analytic review of the predictors of academic achievement. This project is guided by three primary research questions: (1) How do we…

Randomized controlled trials (RCTs) are considered the "gold standard" for evaluating an intervention's effectiveness. Recently, the federal government has placed increased emphasis on the use of opportunistic experiments. A key criterion for conducting opportunistic experiments, however, is that there is relatively easy access to data…

Recent guidance on the treatment of missing data in experiments advocates the use of sensitivity analysis and worst-case bounds analysis for addressing non-ignorable missing data mechanisms; moreover, plans for the analysis of missing data should be specified prior to data collection (Puma et al., 2009). While these authors recommend only that…

The TALPS study aims to build on the existing research base to develop promising methodologies to identify chronically low-performing and turnaround schools, as well as to identify promising strategies for turning around chronically low-performing schools. By looking specifically at schools identified as turnaround, in comparison to nonturnaround…

Students beginning high school commonly experience increased stress and behavior problems alongside declines in grades, attendance, interest in school, and perceptions of academic competence and self-esteem (Alvidrez & Weinstein, 1993; Reyes et al., 2000). Moreover, research indicates that, relative to students who graduate from high school,…

In this paper, the authors examine some of the ways that different types of non-equivalent comparison groups can be used to strengthen causal inferences based on regression discontinuity design (RDD). First, they consider a design that incorporates pre-test data on assignment scores and outcomes that were collected either before the treatment…

Given the different possibilities of matching in the context of multilevel data and the lack of research on corresponding matching strategies, the author investigates two main research questions. The first research question investigates the advantages and disadvantages of different matching strategies that can be pursued with multilevel data…

The primary aim of the Middle School Intervention Project (MSIP) is to evaluate the impact of a multi-component intervention for struggling adolescent readers on reading outcomes. The intervention consists of: (1) targeted, Tier 2 reading and (2) school engagement interventions, and (3) data-based-decision-making (DBDM) teams to review and act on…