Single-case experimental designs (SCEDs) have a long history in clinical and educational disciplines. One underdeveloped area in advancing SCED design and analysis is understanding the process of how internal validity threats and operational concerns are avoided or mitigated. Two strategies to ameliorate such issues in SCED involve replication and…

Scholars who conduct process tracing often face the problem of missing data. The inability to document key steps in their causal chains makes it difficult to validate theoretical models. In this article, we conceptualize "missingness" as it relates to process tracing, describe different scenarios in which it is pervasive, and present…

As a research field geared toward understanding and improving learning, Learning Analytics (LA) must be able to provide empirical support for causal claims. However, as a highly applied field, tightly controlled randomized experiments are not always feasible nor desirable. Instead, researchers often rely on observational data, based on which they…

Causal effects are commonly defined as comparisons of the potential outcomes under treatment and control, but this definition is threatened by the possibility that either the treatment or the control condition is not well defined, existing instead in more than one version. This is often a real possibility in nonexperimental or observational…

Treatment effect estimates from a "regression discontinuity design" (RDD) have high internal validity. However, the arguments that support the design apply to a subpopulation that is narrower and usually different from the population of substantive interest in evaluation research. The disconnect between RDD population and the…

Organizational and applied sciences have long struggled with improving causal inference in quasi-experiments. We introduce organizational researchers to propensity scoring, a statistical technique that has become popular in other applied sciences as a means for improving internal validity. Propensity scoring statistically models how individuals in…

Selection bias is a problem for mathematics education researchers interested in using observational rather than experimental data to make causal inferences about the effects of different instructional methods in mathematics on student outcomes. Propensity score methods represent 1 approach to dealing with such selection bias. This article…

Counterfactuals are routinely used in physical and biological sciences to develop and evaluate sophisticated, non-linear models. They have been used with telling effect in the study of economic history and American politics. For some historians, counterfactual arguments have no scholarly standing. They consider them flights of fancy, fun over a…