Given decision-makers often prioritize causal research that identifies the impact of treatments on the people they serve, a key question in education research is, "Does it work?". Today, however, researchers are paying increasing attention to successive questions that are equally important from a practical standpoint--not only does it…

This paper scrutinizes the increasing trend of using international large-scale assessment (ILSA) data for causal inferences in educational research, arguing that such inferences are often tenuous. We explore the complexities of causality within ILSAs, highlighting the methodological constraints that challenge the validity of causal claims derived…

Background: For decades, education researchers have relied on the work of Campbell, Cook, and Shadish to help guide their thinking about valid impact estimates in the social sciences (Campbell & Stanley, 1963; Shadish et al., 2002). The foundation of this work is the "validity typology" and its associated "threats to…

Within evidence-based education, results from randomised controlled trials (RCTs), and meta-analyses of them, are taken as reliable evidence for effectiveness -- they speak to "what works". Extending RCT results requires establishing that study samples and settings are representative of the intended target. Although widely recognised as…

Ethics is a complicated field and much has been written about its application to educational research. In this paper we introduce a way of planning for and dealing with situations that arise in the course of research that promotes detailed ethical analysis. We present a framework based on the work of Seedhouse and Flinders and describe a…

This report summarizes the complex research literature on quantitative methods for assessing how impacts of educational interventions on instructional practices and student learning differ across students, educators, and schools. It also provides technical guidance about the use and interpretation of these methods. The research topics addressed…

Experimental designs are distinguished as the best method to respond to questions involving causality. The purpose of the present paper is to explicate the logic of experimental design and why it is so vital to questions that demand causal conclusions. In addition, types of internal and external validity threats are discussed. To emphasize the…

This paper contains a consideration of the nature and role of warrants for research conclusions in educational research. The paper argues the need for an explicit warrant in the form of a logical and persuasive link between the evidence produced and the conclusions drawn (with appropriate qualifications and caveats). It describes social scientific…

Educational researchers are not engaged in discovering mechanistic universal truths sought by positivists or tidy prescriptions about what works. Instead, they provide guidelines and interpretive material to liberate the teacher's intelligence so that the teacher can use whatever ability he or she can achieve. (SLD)