Background/Context: Methodological background: Meta-analysis typically depends on the assumption that true effects follow the normal distribution. While assuming normality of effect "estimates" is often supported by a central limit theorem, normality for the distribution of interventions' "true" effects is a computational…

Despite recent emphasis on the use of randomized control trials (RCTs) for evaluating education interventions, in most areas of education research, observational methods remain the dominant approach for assessing program effects. Over the last three decades, the within-study comparison (WSC) design has emerged as a method for evaluating the…

Questions regarding the generalizability of results from educational experiments have been at the forefront of methods development over the past five years. This work has focused on methods for estimating the effect of an intervention in a well-defined inference population (e.g., Tipton, 2013; O'Muircheartaigh and Hedges, 2014); methods for…

Randomized experiments are commonly used to evaluate if particular interventions improve student achievement. While these experiments can establish that a treatment actually "causes" changes, typically the participants are not randomly selected from a well-defined population and therefore the results do not readily generalize. Three…

Propensity score analysis (PSA) is a methodological technique which may correct for selection bias in a quasi-experiment by modeling the selection process using observed covariates. Because logistic regression is well understood by researchers in a variety of fields and easy to implement in a number of popular software packages, it has…

The goal of this research is to make sensitivity analysis accessible not only to empirical researchers but also to the various stakeholders for whom educational evaluations are conducted. To do this it derives anchors for the omitted variable (OV)-program participation association intrinsically, using the Love plot to present a wide range of…

In some instances, intentionally or not, study designs are such that there is clustering in one group but not in the other. This paper describes methods for computing effect size estimates and their variances when there is clustering in only one group and the analysis has not taken that clustering into account. The authors provide the effect size…

Access to Head Start has been shown to improve children's preschool experiences and school readiness on selected factors through the end of 1st grade. Two more years of follow-up, through the end of 3rd grade, can now be examined to determine whether these effects continue into the middle elementary grades. The statistical design and impact…

The What Works Clearinghouse (WWC) seeks to provide educators, policymakers, researchers, and the public with a central and trusted source of scientific evidence for what works in education. The WWC was established in 2002 by the U.S. Department of Education's Institute of Education Sciences (IES). It serves as a decision-making resource by…

The purpose of this research to investigate the heterogeneity of per-pupil expenditure (PPE) slope estimates in predicting student achievement. The research question guiding this project is: how does the measured relationship between per-pupil expenditure vary across studies that use different models? In concert with SREE's 2012 conference mission…

The authors' primary aim was to investigate a potential causal relationship between spatial ability and math ability. To do so, they used a pretest-training-posttest experimental design in which children received short-term spatial training and were tested on problem solving in math. They focused on first and second graders because earlier studies…

Instrumental variable estimators hold the promise of enabling researchers to estimate the effects of educational treatments that are not (or cannot be) randomly assigned but that may be affected by randomly assigned interventions. Examples of the use of instrumental variables in such cases are increasingly common in educational and social science…

In this paper, the authors summarize a quantitative analysis demonstrating that the CyGaMEs toolset for embedded assessment of learning within instructional games measures growth in conceptual knowledge by quantifying player behavior. CyGaMEs stands for Cyberlearning through GaME-based, Metaphor Enhanced Learning Objects. Some scientists of…

In 2005, the U.S. Department of Education awarded eight Striving Readers grants to projects around the country. The goal of Striving Readers is to improve literacy skills and achievement for struggling readers in middle and high school and to increase the research base using randomized clinical research to address improvement in adolescent…

The current study will examine possible contextual effects relative to differences in reading comprehension performance in the state of Florida. While the Reading First (RF) Impact study examined such difference using a regression discontinuity design, the authors are primarily interested in other analytic methods that might answer different…