This article develops new closed-form variance expressions for power analyses for commonly used difference-in-differences (DID) and comparative interrupted time series (CITS) panel data estimators. The main contribution is to incorporate variation in treatment timing into the analysis. The power formulas also account for other key design features…

As a research field geared toward understanding and improving learning, Learning Analytics (LA) must be able to provide empirical support for causal claims. However, as a highly applied field, tightly controlled randomized experiments are not always feasible nor desirable. Instead, researchers often rely on observational data, based on which they…

Randomization of large number of participants to different treatment groups is often not a feasible or preferable way to answer questions of immediate interest to professional practice. Single case designs (SCDs) are a class of research designs that are experimental in nature but require only a few participants, all of whom receive the…

Experimental studies are often employed to test the effectiveness of teaching innovations such as new pedagogy, curriculum, or learning resources. This article offers guidance on good practice in developing research designs, and in drawing conclusions from published reports. Random control trials potentially support the use of statistical…

The aim of this article is to provide practitioners and other nonresearchers a basic understanding of research designs to aid in appropriately interpreting and applying research findings in special education. Research design provides the blueprint for conducting a research study and shapes what kind of knowledge is generated by the study. We…

Treatment effect estimates from a "regression discontinuity design" (RDD) have high internal validity. However, the arguments that support the design apply to a subpopulation that is narrower and usually different from the population of substantive interest in evaluation research. The disconnect between RDD population and the…

This paper presents best practices for conducting survey research using Amazon Mechanical Turk (MTurk). Readers will learn the benefits, limitations, and trade-offs of using MTurk as compared to other recruitment services, including SurveyMonkey and Qualtrics. A synthesis of survey design guidelines along with a sample survey are presented to help…

This paper discusses the contrasting perspectives of academic prose versus grant writing, and lists strategies grant specialists can use to help researchers break old habits and replace them with techniques better suited to the world of competitive grant proposals. [This article is a reprint of "Why Academics Have a Hard Time Writing Good…

This review protocol from the What Works Clearinghouse (WWC) covers the following issues relevant to screening criteria and full review for the Access to Higher Education Practice Guide: (1) time frame; (2) study design; (3) age and/or grade range; (4) brief description of intervention; (5) key subgroups; (6) location and language; (7) outcomes…

E-learning research is plenty of difficulties, as also research in education is. Usually, the high number of features involved in e-learning processes complicates and masks the identification and isolation of the factors which cause the expected benefits, when they exist. At the same time, a bunch of threats are ready to weaken the validity of the…

The self-modeling intervention has been studied for more than 40 years, most often through single-subject research design. This article evaluates the use of the intervention with behaviors associated with both externalizing and internalizing disorders. It compares and summarizes the use of the self-modeling intervention with behaviors such as:…

Researchers may be interested in examining the impact of programs that prepare youth and adults for successful careers but unable to implement experimental designs with true randomization of participants. As a result, these studies can be compromised by underlying factors that impact group selection and thus lead to potentially biased results.…

In this article, we outline a course wherein the instructors teach students how to conduct rigorous qualitative research. We discuss the four major distinct, but overlapping, phases of the course: conceptual/theoretical, technical, applied, and emergent scholar. Students write several qualitative reports, called qualitative notebooks, which…

Experimental designs that randomly assign entire clusters of individuals (e.g., schools and classrooms) to treatments are frequently advocated as a way of guarding against contamination of the estimated average causal effect of treatment. However, in the absence of contamination, experimental designs that randomly assign intact clusters to…

The reaction that occurs when Mentos are added to bottled soft drinks has become a staple demonstration in earth science courses to explain how volcanoes erupt. This paper presents how this engaging exercise can be used in a marketing research course to provide hands-on experience with problem formation, hypothesis testing, and causal research. A…