This article develops new closed-form variance expressions for power analyses for commonly used difference-in-differences (DID) and comparative interrupted time series (CITS) panel data estimators. The main contribution is to incorporate variation in treatment timing into the analysis. The power formulas also account for other key design features…

Underlying all What Works Clearinghouse (WWC) products are WWC Study Review Guides, which are intended for use by WWC certified reviewers to assess studies against the WWC evidence standards. As part of an ongoing effort to increase transparency, promote collaboration, and encourage widespread use of the WWC standards, the Institute of Education…

This document provides step-by-step instructions on how to complete the Study Review Guide (SRG, Version S3, V2) for single-case designs (SCDs). Reviewers will complete an SRG for every What Works Clearinghouse (WWC) review. A completed SRG should be a reviewer's independent assessment of the study, relative to the criteria specified in the review…

Traditionally, evaluation has primarily tried to answer the question "Does a program, service, or policy work?" Recently, more attention is given to questions about variation in program effects and the mechanisms through which program effects occur. Addressing these kinds of questions requires moving beyond assessing average program…

In this commentary, we offer a set of visual tools that can assist education researchers, especially those in the field of mathematics, in developing cohesiveness from a mixed methods perspective, commencing at a study's research questions and literature review, through its data collection and analysis, and finally to its results. This expounds…

This article presents 3 standardized effect size measures to use when sharing results of an analysis of mediation of treatment effects for cluster-randomized trials. The authors discuss 3 examples of mediation analysis (upper-level mediation, cross-level mediation, and cross-level mediation with a contextual effect) with demonstration of the…

Covariate adjustment can increase the precision of estimates by removing unexplained variance from the error in randomized experiments, although chance covariate imbalance tends to counteract the improvement in precision. The author develops an easy measure to examine chance covariate imbalance in randomization by standardizing the average…

This paper focuses on the trade-offs that lie at the intersection of methodological requirements for causal effect studies and policies that affect how and to what extent schools engage in such studies. More specifically, current federal funding priorities encourage large-scale randomized studies of interventions in authentic settings. At the same…

The development of a usability evaluation method for educational systems or applications, called the self-report-based sequential analysis, is described herein. The method aims to extend the current practice by proposing self-report-based sequential analysis as a new usability method, which integrates the advantages of self-report in survey…

Hierarchical Linear Modeling (HLM) sample size recommendations are mostly made with traditional group-design research in mind, as HLM as been used almost exclusively in group-design studies. Single-case research can benefit from utilizing hierarchical linear growth modeling, but sample size recommendations for growth modeling with HLM are scarce…

How doctoral programs train future researchers in quantitative methods has important implications for the quality of scientifically based research in education. The purpose of this article, therefore, is to examine how quantitative methods are used in the literature and taught in doctoral programs. Evidence points to deficiencies in quantitative…

A guide to survey research is presented for both novice and experienced researchers. Steps of the survey research process are covered: (1) planning the survey to determine the purpose of the study, collecting background information, designing the sample, and making a time line for completing the project; (2) questionnaire design, including the…

Discusses the effects of sampling scheme selection on the admissibility of log-linear models for multinomial and product multinomial sampling schemes for prospective and retrospective sampling. Notes that in multinomial sampling, marginal frequencies are not fixed, whereas for product multinomial sampling, uni- or multidimensional frequencies are…

THE SELECTION AND USE OF RESPONSE VARIABLES IN EDUCATIONAL EXPERIMENTS USING MULTIVARIATE ANALYSIS WERE CONSIDERED AND ASSESSED. RESPONSE VARIABLES WERE DEEMED CENTRAL CONCERNS FOR EITHER THEORETICAL OR PRACTICALLY ORIENTED RESEARCH, AND THEIR COMPLEXITY WAS DEALT WITH UNDER THE HEADINGS OF APTITUDE INPUT MEASURES, REPEATED LEARNING MEASURES,…

The authors of this article contend that the Guinan and Foulds study was inadequately designed and executed, and the results indicate little of the "usefulness" of the test, much less illuminate the important hypothesis central to the investigation. Specific suggestions for further research in marathon group evaluation are made. (Author)