This guide will help researchers design and implement impact studies in education so that the findings are more generalizable to the study's target population. Guidance is provided on key steps that researchers can take, including defining the target population, selecting a sample of schools--and replacement schools, when needed--managing school…

This paper discusses some of the recurrent issues which the authors have noticed in educational research. These might be concerned with the nature of social science research, such as its representation of the reality of student learning, the role for replication studies, and the tolerance for disagreements. There are also issues related to the…

The purpose of this study is to disseminate the results of recent advances in statistical power analyses with regard to multilevel mediation and its implementation in the PowerUp!-Mediator software. The authors first focus on the conceptual and statistical differences among common asymptotic, component-wise, and resampling-based tests of mediation…

Digital learning platforms (DLPs) are beginning to become open to research. Specifically, in our work in SEERNet, developers are extending five platforms, each used in either K-12 or higher education by more than 100,000 users, to enable third-party researchers to explore, develop, and test improvements. SEERNet seeks to enable equity-relevant…

Data analysis is one of the most important, yet least understood, stages of the qualitative research process. Through rigorous analysis, data can illuminate the complexity of human behavior, inform interventions, and give voice to people's lived experiences. While significant progress has been made in advancing the rigor of qualitative analysis,…

This report presents the statistical theory underlying the "RCT-YES" software that estimates and reports impacts for RCTs for a wide range of designs used in social policy research. The report discusses a unified, non-parametric design-based approach for impact estimation using the building blocks of the Neyman-Rubin-Holland causal…

As evidence-based reform becomes increasingly important in educational policy, it is becoming essential to understand how research design might contribute to reported effect sizes in experiments evaluating educational programs. The purpose of this study was to examine how methodological features such as types of publication, sample sizes, and…

This paper provides a guide to calculating statistical power for the complex multilevel designs that are used in most field studies in education research. For multilevel evaluation studies in the field of education, it is important to account for the impact of clustering on the standard errors of estimates of treatment effects. Using ideas from…

Scientifically based research used to inform evidence based school reform efforts has been required by the federal government in order to receive grant funding since the reenactment of No Child Left Behind (2002). Educational evaluators are thus faced with the challenge to use rigorous research designs to establish causal relationships. However,…

Mathematics education researchers conducting instruction experiments using a design research methodology are challenged with the analysis of often complex and large amounts of qualitative data. In this paper, we present two case studies that show how multimedia analysis software can greatly support video data analysis and theory development in…

The Programme for International Assessment of Adult Competencies (PIAAC) will establish technical standards and guidelines to ensure that the survey design and implementation processes of PIAAC yield high-quality and internationally comparable data. This document provides a revised version of the technical standards and guidelines originally…

One purpose of educational research is to provide information about the likely impact of interventions or treatments on policy-relevant populations of students. Randomized experiments are useful for estimating the causal effects of interventions on the students in schools that participate in the experiments. Unfortunately, the samples of schools…

CASCADE-SEA stands for computer supported curriculum analysis, design and evaluation for science education in Africa. It is the name of a computer program designed to help secondary level science teachers in southern Africa create exemplary paper-based lesson materials. Research conducted alongside the design and development of the CASCADE-SEA…

To appreciate the true impact of scientifically based research on education, it may be useful first to understand the meaning of the various terms that the law employs as part of its definition of SBR (Scientifcally Based Research). The This article presents an explication from a Software & Information Industry Association publication titled…

Present research connected with computer-based instruction indicates that many investigators rely on conventional experimental design and data analysis to address the research question. Although this is appropriate in many cases, the limits of hard and software resources and sample size in other cases have made conventional approaches a poor…