In qualitative comparative analysis, as with all methods, there is a question about how many cases are needed to make an analysis robust. In deciding on the number of cases, a key consideration is the number of conditions to be analyzed. I suggest that adding cases is preferable to dropping conditions if there are too many conditions relative to…

In sparse data meta-analyses (with few trials or zero events), conventional methods may distort results. Although better-performing one-stage methods have become available in recent years, their implementation remains limited in practice. This study examines the impact of using conventional methods compared to one-stage models by re-analysing…

As traditionally conceived, publication bias arises from selection operating on a collection of individually unbiased estimates. A canonical form of such selection across studies (SAS) is the preferential publication of affirmative studies (i.e., those with significant, positive estimates) versus nonaffirmative studies (i.e., those with…

The authors of this article present arguments for why and how qualitative research should be used in economic education. These arguments recognize the nature of economics as a discipline and economics educators' current expertise and preferences. The authors have five goals: (i) clarifying how and why the use of qualitative research in economic…

Meta-analytic models for dependent effect sizes have grown increasingly sophisticated over the last few decades, which has created challenges for a priori power calculations. We introduce power approximations for tests of average effect sizes based upon several common approaches for handling dependent effect sizes. In a Monte Carlo simulation, we…

Social scientists seeking to inform policy or public action must carefully consider how to identify effects and express inferences because actions based on invalid inferences will not yield the intended results. Recognizing the complexities and uncertainties of social science, we seek to inform inevitable debates about causal inferences by…

Alongside the rise in online qualitative research, fraudulent representation of research participants is increasing, and current strategies to address this issue within studies of non-neurodiverse samples may exclude autistic voices. Seeking to reduce fraudulent or "scammer" participation in our focus group study, and increase data…

Quantitative education research is often perceived to be "objective" or "neutral." However, quantitative research has been and continues to be used to perpetuate inequities; these inequities arise as both intended effects and unintended side effects of traditional quantitative research. In this review of the literature, we…

Purpose: The purpose of this dissertation is to investigate the feasibility of using tests of robustness in peer review. This study involved selecting three high-impact papers which featured open data and utilized bioinformatic analyses but provided no source code and refactoring these to allow external survey participants to swap tools,…

Young and Holsteen (YH) introduce a number of tools for evaluating model uncertainty. In so doing, they are careful to differentiate their method from existing forms of model averaging. The fundamental difference lies in the way in which the underlying estimates are weighted. Whereas standard approaches to model averaging assign higher weight to…

This study aims to investigate the trustworthiness of empirical research published in distance education (DE) journals, an area that has yet to be systematically explored. The review covers 238 empirical studies which were based on primary data and published in 2021 in eight DE journals listed in Social Science Citation Index (SSCI) or Emerging…

Meta-analyses based on systematic literature reviews are commonly used to obtain a quantitative summary of the available evidence on a given topic. However, the reliability of any meta-analysis is constrained by that of its constituent studies. One major limitation is the possibility of small-study effects, when estimates from smaller and larger…

The commenter's proposal may be a reasonable method for addressing uncertainty in predictive modeling, where the goal is to predict "y." In a treatment effects framework, where the goal is causal inference by conditioning-on-observables, the commenter's proposal is deeply flawed. The proposal (1) ignores the definition of…

There is a need for effect sizes that are readily interpretable by a broad audience. One index that might fill this need is [pi], which represents the proportion of scores in one group that exceed the mean of another group. The robustness of estimates of [pi] to violations of normality had not been explored. Using simulated data, three estimates…

In this study, we re-analyze recent empirical research on replication from a meta-analytic perspective. We argue that there are different ways to define "replication failure," and that analyses can focus on exploring variation among replication studies or assess whether their results contradict the findings of the original study. We…