As a research field geared toward understanding and improving learning, Learning Analytics (LA) must be able to provide empirical support for causal claims. However, as a highly applied field, tightly controlled randomized experiments are not always feasible nor desirable. Instead, researchers often rely on observational data, based on which they…

Comparison studies conducted to determine which instructional interventions are most efficient for teaching discrete behaviors to individuals with disabilities are potentially valuable, although some threats to internal validity may be more likely in these studies. Studies included in this review typically met common internal validity standards,…

The Benefit Offset National Demonstration (BOND) tested a $1 reduction in benefits per $2 earnings increase above the level at which Social Security Disability Insurance benefits drop from full to zero under current law. BOND included a rare and large "population-representative" experiment: It applied the rule to a nationwide, random…

Experimental studies are often employed to test the effectiveness of teaching innovations such as new pedagogy, curriculum, or learning resources. This article offers guidance on good practice in developing research designs, and in drawing conclusions from published reports. Random control trials potentially support the use of statistical…

Treatment effect estimates from a "regression discontinuity design" (RDD) have high internal validity. However, the arguments that support the design apply to a subpopulation that is narrower and usually different from the population of substantive interest in evaluation research. The disconnect between RDD population and the…

This paper presents best practices for conducting survey research using Amazon Mechanical Turk (MTurk). Readers will learn the benefits, limitations, and trade-offs of using MTurk as compared to other recruitment services, including SurveyMonkey and Qualtrics. A synthesis of survey design guidelines along with a sample survey are presented to help…

Survey research is prevalent among many professional fields. Both cost effective and time efficient, this method of research is commonly used for the purposes of gaining insight into the attitudes, thoughts, and opinions of populations. Additionally, because there are several types of survey research designs and data collection instruments, the…

Regression discontinuity design (RD) has been widely used to produce reliable causal estimates. Researchers have validated the accuracy of RD design using within study comparisons (Cook, Shadish & Wong, 2008; Cook & Steiner, 2010; Shadish et al, 2011). Within study comparisons examines the validity of a quasi-experiment by comparing its…

The purpose of this study was to analyze and compare recent research papers on foreign language education in Turkish context with those published in international context to shed light on researchers and policy makers for future studies. This study filled a gap in this field and also aimed to increase the rate of acceptance of research papers…

Background: The efficacy of three dietary treatments for ADHD has been repeatedly tested in randomized controlled trials (RCTs). These interventions are restricted elimination diets (RED), artificial food colour elimination (AFCE) and supplementation with free fatty acids (SFFA). There have been three systematic reviews and associated…

Randomized controlled trials (RCTs) and regression discontinuity (RD) studies both provide estimates of causal effects. A major difference between the two is that RD only estimates local average treatment effects (LATE) near the cutoff point of the forcing variable. This has been cited as a drawback to RD designs (Cook & Wong, 2008).…

The sharp regression discontinuity design (RDD) has three key weaknesses compared to the randomized clinical trial (RCT). It has lower statistical power, it is more dependent on statistical modeling assumptions, and its treatment effect estimates are limited to the narrow subpopulation of cases immediately around the cutoff, which is rarely of…

This review examines a research study that compares high school graduation rates of students who used vouchers to attend private high schools in Milwaukee and students who attended public high schools in that same city. The study reports that for the most recent year of data, a sample of voucher students had estimated graduation rates 12…

Formative and design experiments represent a methodology uniquely suited for educational research. Providing a practical overview of this emerging approach, the authors address the following questions: What is the origin of formative and design experiments and how do they compare to other approaches to investigating interventions in classrooms?…