Three quantitative methods are compared for their ability to understand different COVID-19 fatality ratios in 33 OECD countries. Linear regression provides a limited overview without sensitivity to the diversity of cases. Cluster Analysis and Dynamic Patterns Synthesis (DPS) gives scrutiny to the granularity of case similarities and differences,…

Background: Structural Equation Modeling (SEM) is a powerful and broadly utilized statistical framework. Researchers employ these models to dissect relationships into direct, indirect, and total effects (Bollen, 1989). These models unpack the "black box" issues within cause-and-effect studies by examining the underlying theoretical…

Quasi-experimental methods are a cornerstone of applied social science, providing critical answers to causal questions that inform policy and practice. Although open science principles have influenced experimental research norms across the social sciences, these practices are rarely implemented in quasi-experimental research. In this paper, we…

Randomized A/B tests allow causal estimation without confounding but are often under-powered. This paper uses a new dataset, including over 250 randomized comparisons conducted in an online learning platform, to illustrate a method combining data from A/B tests with log data from users who were not in the experiment. Inference remains exact and…

Background: For decades, education researchers have relied on the work of Campbell, Cook, and Shadish to help guide their thinking about valid impact estimates in the social sciences (Campbell & Stanley, 1963; Shadish et al., 2002). The foundation of this work is the "validity typology" and its associated "threats to…

Replication has long been a cornerstone for establishing trustworthy scientific results, but there remains considerable disagreement about what constitutes as a replication, how results from these studies should be interpreted, and whether direct replication of results is even possible. This article addresses these concerns by presenting the…

Evaluating blocked randomized experiments from a potential outcomes perspective has two primary branches of work. The first focuses on larger blocks, with multiple treatment and control units in each block. The second focuses on matched pairs, with a single treatment and control unit in each block. These literatures not only provide different…

Qualitative comparative analysis (QCA) is a set-theoretic configurational approach that uses the logic of Boolean algebra to conceptualize and empirically examine potentially complex causal relations. The potential of this methodological innovation to draw innovative insights toward answering enduring questions and to foster novel research has…

The combination of Qualitative Comparative Analysis (QCA) with process tracing, which we call set-theoretic multimethod research (MMR), is steadily becoming more popular in empirical research. Despite the fact that both methods have an elected affinity based on set theory, it is not obvious how a within-case method operating in a single case and a…

Statisticians have made great progress in creating methods that reduce our reliance on parametric assumptions. However this explosion in research has resulted in a breadth of inferential strategies that both create opportunities for more reliable inference as well as complicate the choices that an applied researcher has to make and defend.…

In the causal inference literature, evaluating blocking from a potential outcomes perspective has two main branches of work. The first focuses on larger blocks, with multiple treatment and control units in each block. The second focuses on matched pairs, with a single treatment and control unit in each block. These literatures not only provide…

After explicit instruction on how to give possible bidirectional (two-way) causality explanations for a correlation, 240 students from eight sections of social psychology and research methods courses wrote "reverse causality" explanations on various test questions, creating a total of 882 answers. Averaging across multiple graded…

Methods of causal inference are not widely used by education researchers, even though they can be extremely useful tools for eliminating selection bias and confounding factors in empirical studies. For example, researchers have established that taking additional math classes in high school is strongly correlated with success in college and higher…

We delineate the underlying homogeneity assumption, procedural variants, and implications of the comparative method and distinguish this from Mill's method of difference. We demonstrate that additional units can provide "placebo" tests for the comparative method even if the scope of inference is limited to the two units under comparison.…

Over the past two decades, there have been calls by scholars for a more concerted effort to develop empirically grounded research studies on educational leadership management and administration in an indigenous context rather than hegemonic western contexts. This paper presents a review of contemporary literature on educational leadership from…