We provide a novel characterization of augmented balancing weights, also known as automatic debiased machine learning (AutoDML). These popular "doubly robust" or "de-biased machine learning estimators" combine outcome modeling with balancing weights -- weights that achieve covariate balance directly in lieu of estimating and…

Missing data are a pervasive problem in data analysis. Three common methods for addressing the problem are (a) complete-case analysis, where only units that are complete on the variables in an analysis are included; (b) weighting, where the complete cases are weighted by the inverse of an estimate of the probability of being complete; and (c)…

Providing a rich context has become a sine qua non of principled teaching of applied statistical thinking. With increasing opportunities to access secondary data, there should be increasing opportunity to work with rich context. We review the contextual information provided in 41 data sets suitable for introductory tertiary statistics teaching,…

Background: The generalizability of a study's results continues to be at the forefront of concerns in evaluation research in education (Tipton & Olsen, 2018). Over the past decade, statisticians have developed methods, mainly based on propensity scores, to improve generalizations in the absence of random sampling (Stuart et al., 2011; Tipton,…

We consider the statistical inference for noisy incomplete binary (or 1-bit) matrix. Despite the importance of uncertainty quantification to matrix completion, most of the categorical matrix completion literature focuses on point estimation and prediction. This paper moves one step further toward the statistical inference for binary matrix…

The limitations of Cohen's ? are reviewed and an alternative G-index is recommended for assessing nominal-scale agreement. Maximum likelihood estimates, standard errors, and confidence intervals for a two-rater G-index are derived for one-group and two-group designs. A new G-index of agreement for multirater designs is proposed. Statistical…

Qualitative comparative analysis (QCA) is an empirical research method that has gained some popularity in the social sciences. At the same time, the literature has long been convinced that QCA is prone to committing causal fallacies when confronted with non-causal data. More specifically, beyond a certain case-to-factor ratio, the method is…

Latent class analysis (LCA) is a widely used technique for detecting unobserved population heterogeneity in cross-sectional data. Despite its popularity, the performance of LCA is not well understood. In this study, we evaluate the performance of LCA with binary data by examining classification accuracy, parameter estimation accuracy, and coverage…

Causal inference is a fundamental goal of many research endeavors, including scholarship in the field of language education and learning. Randomized controlled trials are considered an ideal design to test causal claims, but not all claims can be subjected to experimental treatment due to ethical and practical constraints. In this article, we…

Background: Clustered randomized controlled trials are commonly used to evaluate the effectiveness of treatments. Frequently, stratified or paired designs are adopted in practice. Fogarty (2018) studied variance estimators for stratified and not clustered experiments and Schochet et. al. (2022) studied that for stratified, clustered RCTs with…

Background: The emergence of causal research in education was almost strictly quantitative twenty years ago, however, that landscape has changed considerably. The number of intervention studies fielded and completed annually has increased substantially, and the quality of the evaluations is much more robust, including paying much greater attention…

In causal mediation studies that decompose an average treatment effect into indirect and direct effects, examples of post-treatment confounding are abundant. In the presence of treatment-by-mediator interactions, past research has generally considered it infeasible to adjust for a post-treatment confounder of the mediator-outcome relationship due…

Beginning with debates about the effects of smoking on lung cancer, sensitivity analyses characterizing the hypothetical unobserved conditions that can alter statistical inferences have had profound impacts on public policy. One of the most ascendant techniques for sensitivity analysis is Oster's (2019) coefficient of proportionality, which…

Meta-analysis of proportions is conceptually simple: Faced with a binary outcome in multiple studies, we seek inference on some overall proportion of successes/failures. Under common effect models, exact inference has long been available, but is not when we more realistically allow for heterogeneity of the proportions. Instead a wide range of…

This paper informs users of data collected in international large-scale assessments (ILSA), by presenting argumentsunderlining the importance of considering two design features employed in these studies. We examine a commonmisconception stating that the uncertainty arising from the assessment design is negligible compared with that arisingfrom the…