Fixed effects (FE) panel models have been used extensively in the past, as those models control for all stable heterogeneity between units. Still, the conventional FE estimator relies on the assumption of parallel trends between treated and untreated groups. It returns biased results in the presence of heterogeneous slopes or growth curves that…

Students often believe that statistical significance is the only determinant of whether a quantitative result is "important." In this paper, I review traditional null hypothesis statistical testing to identify what questions inferential statistics can and cannot answer, including statistical significance, effect size and direction,…

Analyzing heterogeneous treatment effects (HTEs) plays a crucial role in understanding the impacts of educational interventions. A standard practice for HTE analysis is to examine interactions between treatment status and preintervention participant characteristics, such as pretest scores, to identify how different groups respond to treatment.…

In educational contexts, students often self-select into specific interventions (e.g., courses, majors, extracurricular programming). When students self-select into an intervention, systematic group differences may impact the validity of inferences made regarding the effect of the intervention. Propensity score methods are commonly used to reduce…

Analyzing heterogeneous treatment effects (HTE) plays a crucial role in understanding the impacts of educational interventions. A standard practice for HTE analysis is to examine interactions between treatment status and pre-intervention participant characteristics, such as pretest scores, to identify how different groups respond to treatment.…

Researchers commonly interpret effect sizes by applying benchmarks proposed by Jacob Cohen over a half century ago. However, effects that are small by Cohen's standards are large relative to the impacts of most field-based interventions. These benchmarks also fail to consider important differences in study features, program costs, and scalability.…

Researchers commonly interpret effect sizes by applying benchmarks proposed by Cohen over a half century ago. However, effects that are small by Cohen's standards are large relative to the impacts of most field-based interventions. These benchmarks also fail to consider important differences in study features, program costs, and scalability. In…

Teacher coaching has emerged as a promising alternative to traditional models of professional development. We review the empirical literature on teacher coaching and conduct meta-analyses to estimate the mean effect of coaching programs on teachers' instructional practice and students' academic achievement. Combining results across 60 studies that…

In this article, school sector differences in tertiary entrance performance were examined using longitudinal data from the state of Victoria in Australia for 2011. Analysis of students' Tertiary Entrance Aggregate, from which the Australian Tertiary Admission Rank is derived, revealed non-trivial effect sizes of sector on performance. Results…

Propensity score analysis has been used to minimize the selection bias in observational studies to identify causal relationships. A propensity score is an estimate of an individual's probability of being placed in a treatment group given a set of covariates. Propensity score analysis aims to use the estimate to create balanced groups, akin to a…

The purpose of this paper is to use both conceptual and statistical approaches to explore publication bias in recent causal effects studies in science education, and to draw from this exploration implications for researchers, journal reviewers, and journal editors. This paper fills a void in the "science education" literature as no…

Children from low-income families are at risk of learning outcome difficulties, particularly in literacy. Various studies link poor literacy results with performance later in primary and secondary school, and suggest that poverty, literacy skills and weak instructional methods combine to drastically limit the educational opportunities for many…

Donald Campbell's approach to causal inference (D. T. Campbell, 1957; W. R. Shadish, T. D. Cook, & D. T. Campbell, 2002) is widely used in psychology and education, whereas Donald Rubin's causal model (P. W. Holland, 1986; D. B. Rubin, 1974, 2005) is widely used in economics, statistics, medicine, and public health. Campbell's approach focuses on…

Some form of a short interrupted time series (ITS) is often used to evaluate state and national programs. An ITS design with a single treatment group assumes that the pretest functional form can be validly estimated and extrapolated into the postintervention period where it provides a valid counterfactual. This assumption is problematic. Ambiguous…

In an effort to expand the pool of scientific evidence available for review, the What Works Clearinghouse (WWC) assembled a panel of national experts in single-case design (SCD) and analysis to draft SCD Standards. SCDs are adaptations of interrupted time-series designs and can provide a rigorous experimental evaluation of intervention effects.…