Three quantitative methods are compared for their ability to understand different COVID-19 fatality ratios in 33 OECD countries. Linear regression provides a limited overview without sensitivity to the diversity of cases. Cluster Analysis and Dynamic Patterns Synthesis (DPS) gives scrutiny to the granularity of case similarities and differences,…

Background: Structural Equation Modeling (SEM) is a powerful and broadly utilized statistical framework. Researchers employ these models to dissect relationships into direct, indirect, and total effects (Bollen, 1989). These models unpack the "black box" issues within cause-and-effect studies by examining the underlying theoretical…

Qualitative comparative analysis (QCA) is an empirical research method that has gained some popularity in the social sciences. At the same time, the literature has long been convinced that QCA is prone to committing causal fallacies when confronted with non-causal data. More specifically, beyond a certain case-to-factor ratio, the method is…

Regression discontinuity (RD) designs have gained significant popularity as a quasi-experimental device for evaluating education programs and policies. In this paper, we present a comprehensive review of RD designs, focusing on the continuity-based framework, the most widely adopted RD framework. We first review the fundamental aspects of RD…

Physical activity (PA) promotion is an ideal intervention target for public health because it has the potential to help individuals feel better, sleep better, and perform daily tasks more easily, in addition to providing disease prevention benefits. There is strong evidence that individual-level theory-based behavioral interventions are effective…

Randomized A/B tests allow causal estimation without confounding but are often under-powered. This paper uses a new dataset, including over 250 randomized comparisons conducted in an online learning platform, to illustrate a method combining data from A/B tests with log data from users who were not in the experiment. Inference remains exact and…

Quasi-experimental methods are a cornerstone of applied social science, providing critical answers to causal questions that inform policy and practice. Although open science principles have influenced experimental research norms across the social sciences, these practices are rarely implemented in quasi-experimental research. In this paper, we…

By design, randomized experiments (XPs) rule out bias from confounded selection of participants into conditions. Quasi-experiments (QEs) are often considered second-best because they do not share this benefit. However, when results from XPs are used to generalize causal impacts, the benefit from unconfounded selection into conditions may be offset…

Researchers in behavioral sciences are often interested in longitudinal behavior change outcomes and the mechanisms that influence changes in these outcomes over time. The statistical models that are typically implemented to address these research questions do not allow for investigation of mechanisms of dynamic change over time. However, latent…

In science in general and in the context of single-case experimental designs, replication of the effects of the intervention within and/or across participants or experiments is crucial for establishing causality and for assessing the generality of the intervention effect. Specific developments and proposals for assessing whether an effect has been…

As part of a growing emphasis on "STEM," engineering has gained prominence in precollege education. In response to that trend, an emerging area of educational research focuses on the "Nature of Engineering" (NOE), a collection of ideas about what engineering is, what engineers do, and how engineering is related to science and…

In the complex world of educational policy, causal inferences will be debated. As we review non-experimental designs in educational policy, we focus on how to clarify and focus the terms of debate. We begin by presenting the potential outcomes/counterfactual framework and then describe approximations to the counterfactual generated from the…

Conventional advice discourages controlling for postoutcome variables in regression analysis. By contrast, we show that controlling for commonly available postoutcome (i.e., future) values of the treatment variable can help detect, reduce, and even remove omitted variable bias (unobserved confounding). The premise is that the same unobserved…

Background: For decades, education researchers have relied on the work of Campbell, Cook, and Shadish to help guide their thinking about valid impact estimates in the social sciences (Campbell & Stanley, 1963; Shadish et al., 2002). The foundation of this work is the "validity typology" and its associated "threats to…

Identifying causal mechanisms is an essential aspect of disparities research. Rigorous investigations of mechanisms can open the black box and clarify linkages in the causal chain, indicating how effects occur. Studies of mechanisms can also illuminate novel ways to reduce inequality when intervention at the root cause of unequal outcomes is not…