Most researchers receive formal training in frequentist statistics during their undergraduate studies. In particular, hypothesis testing is usually rooted on the null hypothesis significance testing paradigm and its p-value. Null hypothesis Bayesian testing and its so-called Bayes factor are now becoming increasingly popular. Although the Bayes…

This article uses the interventionist theory of causation, a counterfactual theory taken from philosophy of science, to strengthen causal analysis in process tracing research. Causal claims from process tracing are re-expressed in terms of so-called hypothetical interventions, and concrete evidential tests are proposed which are shown to…

The product Bayes factor (PBF) synthesizes evidence for an informative hypothesis across heterogeneous replication studies. It can be used when fixed- or random effects meta-analysis fall short. For example, when effect sizes are incomparable and cannot be pooled, or when studies diverge significantly in the populations, study designs, and…

Researchers often have hypotheses concerning the state of affairs in the population from which they sampled their data to compare group means. The classical frequentist approach provides one way of carrying out hypothesis testing using ANOVA to state the null hypothesis that there is no difference in the means and proceed with multiple comparisons…

We first review existing literature on cumulative logit models along with various ways to test the parallel lines assumption. Building on the traditional frequentist framework, we introduce a method of Bayesian assessment of null values to provide an alternative way to examine the parallel lines assumption using highest density intervals and…

The internal validity of observational study is often subject to debate. In this study, we define the counterfactuals as the unobserved sample and intend to quantify its relationship with the null hypothesis statistical testing (NHST). We propose the probability of a robust inference for internal validity, that is, the PIV, as a robustness index…

In late November 2020, there was a flurry of media coverage of two companies' claims of 95% efficacy rates of newly developed COVID-19 vaccines, but information about the confidence interval was not reported. This paper presents a way of teaching the concept of hypothesis testing and the construction of confidence intervals using numbers announced…

This article presents two alternative methods to null hypothesis significance testing (NHST) for improving inferences from underpowered research designs. Post hoc design analysis (PHDA) assesses whether an NHST analysis generating null findings might otherwise have had sufficient power to detect effects of plausible magnitudes. Bayesian analysis…

Developmental research, like many fields, is plagued by low sample sizes and inconclusive findings. The problem is amplified by the difficulties associated with recruiting infant participants for research as well as the increased variability in infant responses. With sequential testing designs providing a viable alternative to paradigms facing…

Generalization is a fundamental problem solved by every cognitive system in essentially every domain. Although it is known that how people generalize varies in complex ways depending on the context or domain, it is an open question how people "learn" the appropriate way to generalize for a new context. To understand this capability, we…

Purpose: Null hypothesis significance testing is commonly used in audiology research to determine the presence of an effect. Knowledge of study outcomes, including nonsignificant findings, is important for evidence-based practice. Nonsignificant "p" values obtained from null hypothesis significance testing cannot differentiate between…

It is now widely accepted that the standard inferential toolkit used by the scientific research community--null-hypothesis significance testing (NHST)--is not fit for purpose. Yet despite the threat posed to the scientific enterprise, there is no agreement concerning alternative approaches for evidence assessment. This lack of consensus reflects…

This article develops a set-theoretic approach to Bayes's theorem and Bayesian process tracing. In the approach, hypothesis testing is the procedure whereby one updates beliefs by narrowing the range of states of the world that are regarded as possible, thus diminishing the domain in which the actual world can reside. By explicitly connecting…

Students studying statistics often misunderstand what statistics represent. Some of the most well-known misunderstandings of statistics revolve around null hypothesis significance testing. One pervasive misunderstanding is that the calculated p-value represents the probability that the null hypothesis is true, and that if p < 0.05, there is…

In this response, we first show that Simpson's proposed analysis answers a different and less interesting question than ours. We then justify the choice of prior for our Bayes factors calculations, but we also demonstrate that the substantive conclusions of our article are not substantially affected by varying this choice.