Much of the evidential basis for recent policy decisions is grounded in effect size: the standardised mean difference in outcome scores between a study's intervention and comparison groups. This is interpreted as measuring educational influence, importance or effectiveness of the intervention. This article shows this is a category error at two…

This case study considers how a non-equivalent control group design was used in a PhD study examining the impact of a multiple intelligence teaching approach on student engagement and learning. Two groups of second year undergraduate nursing students undertook the study: a treatment group (n = 26) and a control group (n = 18). The intervention for…

This Methodological Brief introduces the reader to the regression discontinuity design (RDD), which is a method that when used correctly can yield estimates of research treatment effects that are equivalent to those obtained through randomized control trials and can therefore be used to infer causality. However, RDD does not require the random…

The purpose of this study is to examine the effects of Head Start on early literacy skills relevant to school readiness of English language learners compared to their peers. The comparisons of literacy outcomes were conducted between English language learners and non-English language learners when both groups participated and were not in Head…

In this paper, the authors examine some of the ways that different types of non-equivalent comparison groups can be used to strengthen causal inferences based on regression discontinuity design (RDD). First, they consider a design that incorporates pre-test data on assignment scores and outcomes that were collected either before the treatment…

In this age of data based decision making and accountability, parent involvement and data collection are paramount. This study represents a significant contribution to educational research by extending the understanding of home-school communication media with specific regard to daily progress reports. The purpose of this study was to compare…

When lotteries are infeasible, researchers must rely on observational methods to estimate charter effectiveness at raising student test scores. Considerable attention has been paid to observational studies by the Stanford Center for Research on Education Outcomes (CREDO), which have analyzed charter performance in 27 states. However, the…

This article presents a method to evaluate pretest effects on posttest scores in the absence of an un-pretested control group using published results of pretesting effects due to Willson and Putnam. Confidence intervals around the expected theoretical gain due to pretesting are computed, and observed gains or differential gains are compared with…

KIPP (Knowledge is Power Program) is a national network of public charter schools whose stated mission is to help underserved students enroll in and graduate from college. Prior studies (see Tuttle et al. 2013) have consistently found that attending a KIPP middle school positively affects student achievement, but few have addressed longer-term…

KIPP (Knowledge is Power Program) is a national network of public charter schools whose stated mission is to help underserved students enroll in and graduate from college. Prior studies (see Tuttle et al. 2013) have consistently found that attending a KIPP middle school positively affects student achievement, but few have addressed longer-term…

KIPP (Knowledge is Power Program) is a national network of public charter schools whose stated mission is to help underserved students enroll in and graduate from college. Prior studies (see Tuttle et al. 2013) have consistently found that attending a KIPP middle school positively affects student achievement, but few have addressed longer-term…

Due to their motivational nature, there has been growing interest in the potential of games to help teach academic content and skills. This report examines how different levels of detail about a game's scoring rules affect math learning and performance. Data were collected from 164 students in the fourth to sixth grades at five after-school…

This report summarizes the complex research literature on quantitative methods for assessing how impacts of educational interventions on instructional practices and student learning differ across students, educators, and schools. It also provides technical guidance about the use and interpretation of these methods. The research topics addressed…

Educational researchers frequently study the impact of treatments or interventions on educational outcomes. However, when observational or quasiexperimental data are used for such investigations, selection bias can adversely impact researchers' abilities to make causal inferences about treatment effects. One way to deal with selection bias is to…

This paper provides practical guidance for researchers who are designing and analyzing studies that randomize schools--which comprise three levels of clustering (students in classrooms in schools)--to measure intervention effects on student academic outcomes when information on the middle level (classrooms) is missing. This situation arises…