This article uses the interventionist theory of causation, a counterfactual theory taken from philosophy of science, to strengthen causal analysis in process tracing research. Causal claims from process tracing are re-expressed in terms of so-called hypothetical interventions, and concrete evidential tests are proposed which are shown to…

"Integrative evaluation" is an approach with two main phases: identification of plausible rival hypotheses and integration of rival hypotheses. The first phase may correspond to traditional adversary evaluation, whereas the second phase, that is not included in adversary evaluation, requires integrative thinking which can be applied when…

Previous simulation studies of randomization tests applied in single-case educational intervention research contexts have typically focused on A-to-B phase changes in means/levels. In the present simulation study, we report the results of two multiple-baseline investigations, one targeting between-phase changes in slopes/trends and the other…

This study presents a novel approach contributing to our understanding of the design, development, and implementation AI-based systems for conducting double-blind online randomized controlled trials (RCTs) for higher education research. The process of the entire interaction with the participants (n = 1193) and their allocation to test and control…

In narrative synthesis of evidence, it can be the case that the only quantitative measures available concerning the efficacy of an intervention is the direction of the effect, that is, whether it is positive or negative. In such situations, the sign test has been proposed in the literature and in recent Cochrane guidelines as a way to test whether…

Understanding that hypothesis testing is aimed at seeking information rather than producing desirable outcomes is indispensable for designing informative experiments. This study investigated the developmental course of information seeking compared to producing an effect in young children. In a between-subjects design, 4- to 6-year-olds (N = 109)…

BASIE is a framework for interpreting impact estimates from evaluations. It is an alternative to null hypothesis significance testing. This guide walks researchers through the key steps of applying BASIE, including selecting prior evidence, reporting impact estimates, interpreting impact estimates, and conducting sensitivity analyses. The guide…

Single-Case Experimental Designs (SCEDs) have lately been recognized as a valuable alternative tolarge group studies. SCEDs form a great tool for the evaluation of treatment effectiveness in heterogeneous and low-incidence conditions, which are common in the field of communication disorders. Mediation analysis is indispensable in treatment…

A central aim of physics education research is to understand the processes of learning and use that understanding to inform instruction. To this end, researchers often conduct studies to measure the effect of classroom interventions on student outcomes. Many of these intervention studies have provided an empirical foundation of reformed teaching…

In the L2 Motivational Self System (L2MSS) motivation is conceptualized to be generated by discomfort associated with the learner's experience of a discrepancy between their current L2 self, and their ideal L2 self. However, in the L2MSS model, this discrepancy is not operationalized. A questionnaire containing measures of current L2 selves was…

Bed bugs are on the rise and are increasingly perceived as harmful parasites. Because individuals affected by bed bugs often feel disgust and shame and are stigmatized, bed bugs are an important public health and environmental justice concern and therefore a health education issue as well. In this quasi-experimental study, we examine how different…

Researchers are often interested in testing the effectiveness of an intervention on multiple outcomes, for multiple subgroups, at multiple points in time, or across multiple treatment groups. The resulting multiplicity of statistical hypothesis tests can lead to spurious findings of effects. Multiple testing procedures (MTPs) are statistical…

Recent research findings (DeRosa, Fisher, & Steege, 2015) suggest that minimizing exposure to the establishing operation (EO) for destructive behavior when differential reinforcement interventions like functional communication training (FCT) are introduced may produce more immediate reductions in destructive behavior and prevent or mitigate…

In recent years, there has been increasing focus on the issue of multiple hypotheses testing in education evaluation studies. In these studies, researchers are typically interested in testing the effectiveness of an intervention on multiple outcomes, for multiple subgroups, at multiple points in time or across multiple treatment groups. When…

This experimental study examined the validity of 3 underlying mechanisms that have been proposed in the research literature to explain the anxiety-reducing effect of coloring mandalas: distraction, structure, and centering. Four conditions were tested: (a) freely coloring a mandala, (b) coloring a mandala with preset colors, (c) freely drawing a…