Sensitivity analysis, a statistical method crucial for validating inferences across disciplines, quantifies the conditions that could alter conclusions (Razavi et al., 2021). One line of work is rooted in linear models and foregrounds the sensitivity of inferences to the strength of omitted variables (Cinelli & Hazlett, 2019; Frank, 2000). A…

Causal inference is a central topic in education research, although oftentimes it relies on observational studies, which makes causal identification methodologically challenging. This manuscript introduces causal graphs as a powerful language for elucidating causal theories and an effective tool for causal identification analysis. It discusses…

This article uses the interventionist theory of causation, a counterfactual theory taken from philosophy of science, to strengthen causal analysis in process tracing research. Causal claims from process tracing are re-expressed in terms of so-called hypothetical interventions, and concrete evidential tests are proposed which are shown to…

As evidence from evaluation and experimental studies continue to influence decision and policymaking, applied researchers and practitioners require tools to derive valid and credible inferences. Over the past several decades, research in causal inference has progressed with the development and application of propensity scores. Since their…

Directed acyclic graphs (DAGs) are now a popular tool to inform causal inferences. We discuss how DAGs can also be used to encode theoretical assumptions about nonprobability samples and survey nonresponse and to determine whether population quantities including conditional distributions and regressions can be identified. We describe sources of…

This article introduces a collection of four datasets, similar to Anscombe's quartet, that aim to highlight the challenges involved when estimating causal effects. Each of the four datasets is generated based on a distinct causal mechanism: the first involves a collider, the second involves a confounder, the third involves a mediator, and the…

Qualitative comparative analysis (QCA) is an empirical research method that has gained some popularity in the social sciences. At the same time, the literature has long been convinced that QCA is prone to committing causal fallacies when confronted with non-causal data. More specifically, beyond a certain case-to-factor ratio, the method is…

Background: Clustered randomized controlled trials are commonly used to evaluate the effectiveness of treatments. Frequently, stratified or paired designs are adopted in practice. Fogarty (2018) studied variance estimators for stratified and not clustered experiments and Schochet et. al. (2022) studied that for stratified, clustered RCTs with…

Background: The emergence of causal research in education was almost strictly quantitative twenty years ago, however, that landscape has changed considerably. The number of intervention studies fielded and completed annually has increased substantially, and the quality of the evaluations is much more robust, including paying much greater attention…

In causal mediation studies that decompose an average treatment effect into indirect and direct effects, examples of post-treatment confounding are abundant. In the presence of treatment-by-mediator interactions, past research has generally considered it infeasible to adjust for a post-treatment confounder of the mediator-outcome relationship due…

Combining observational and experimental data for causal inference can improve treatment effect estimation. However, many observational data sets cannot be released due to data privacy considerations, so one researcher may not have access to both experimental and observational data. Nonetheless, a small amount of risk of disclosing sensitive…

Post-instrument covariates are often included as controls in instrumental variable (IV) analyses to address a violation of the exclusion restriction. However, we show that such analyses are subject to biases unless strong assumptions hold. Using linear constant-effects models, we present asymptotic bias formulas for three estimators (with and…

Researchers tasked with understanding the effects of educational technology innovations face the challenge of providing evidence of causality. Given the complexities of studying learning in authentic contexts interwoven with technological affordances, conducting tightly-controlled randomized experiments is not always feasible nor desirable. Today,…

When planning a cluster randomized trial, evaluators often have access to an enumerated cohort representing the target population of clusters. Practicalities of conducting the trial, such as the need to oversample clusters with certain characteristics in order to improve trial economy or support inferences about subgroups of clusters, may preclude…

The clustered observational study (COS) design is the observational study counterpart to the clustered randomized trial. In a COS, a treatment is assigned to intact groups, and all units within the group are exposed to the treatment. However, the treatment is non-randomly assigned. COSs are common in both education and health services research. In…