I. Purpose: There are instances where equivalences rather than differences are of interest (e.g., Lakens, 2017). For instance, researchers may be interested in testing whether two versions of interventions are equivalent or whether an alternative but more cost-efficient program can provide similar outcomes (Lakens et al., 2018; Schuirmann, 1987).…

The purpose of this paper is to elaborate and comment on some themes related to construct validity, construct choice, and construct respecification in the Campbellian validity system. It is argued that Reichardt's [(2008, November). An alternative to the Campbellian conceptualization of validity. (Paper presentation). Evaluation 2008 Conference,…

The randomized control trial (RCT) is the primary experimental design in education research due to its strong internal validity for causal inference. However, in situations where RCTs are not feasible or ethical, quasi-experiments are alternatives to establish causal inference. This paper serves as an introduction to several quasi-experimental…

Background/Context: Single-case experimental designs (SCEDs) play an important role in evaluating interventions in psychological, educational, and behavioral research. Unlike between-subjects designs, SCEDs involve a small number of cases whose responses to controlled experimental conditions are measured repeatedly over time. The evaluation of…

The difference in differences design is widely used to assess treatment effects in natural experiments or other situations where random assignment cannot, or is not, used (see, e.g., Angrist & Pischke, 2009). The researcher must make important decisions about which comparisons to make, the measurements to make, and perhaps the number of…

Background/Context: Single-case experiment designs (SCEDs) are experimental designs in which a small number of cases are repeatedly measured over time, with manipulation of baseline and intervention phases. Because SCEDs often rely on direct behavioral observations, count data are common. To account for both the clustering and the non-normal…

Background: Multisite designs, also known as blocked designs, are experimental designs in which the random assignment of treatment and control conditions is within each site (or block) after the random selection of sites (or blocks). Multisite designs exhibit remarkable adaptability and, statistically, it can maintain a rigorous basis for…

To design effective instruction, educators need to know "what" design strategies are generally effective and why these strategies work, based on the mechanisms through which they operate. Experimental comparison studies, which compare one instructional design against another, can generate much needed evidence in support of effective…

Background: Educational programs often use student test scores to determine access to some treatment, such as remedial support or graduation (Jacob & Lefgren, 2004; Martorell, 2004; Matsudaira, 2008; Papay et al., 2011, 2014). In these cases, treatment assignment is based on the student's score from one or more subjects. For example, students…

Prior research has suggested that clustered regression discontinuity designs are a formidable alternative to cluster randomized designs because they provide targeted treatment assignment while maintaining a high-quality basis for inferences on local treatment effects. However, methods for the design and analysis of clustered regression…

Mediation analysis in Single Case Experimental Designs (SCEDs) evaluates intervention mechanisms for individuals. Despite recent methodological developments, no clear guidelines exist for maximizing power to detect the indirect effect in SCEDs. This study compares frequentist and Bayesian methods, determining (1) minimum required sample size to…

In recent years, the growing incidence of blended and online learning has highlighted instructional design concerns, especially STEM instructional design. Existing studies have often adopted observations, questionnaires, or interviews to evaluate STEM instructional design plans. However, there is still a lack of quantitative, measurable, and…

Qualitative thematic analysis is a commonly used and widely applicable form of qualitative analysis, though it can be challenging to implement. Due to its use across research questions, qualitative traditions, and fields, thematic analysis is also prevalent in mixed methods studies. Despite its widespread use, the term "thematic…

We consider a class of multiple-group individually-randomized group trials (IRGTs) that introduces a (partially) cross-classified structure in the treatment condition (only). The novel feature of this design is that the nature of the treatment induces a clustering structure that involves two or more non-nested groups among individuals in the…

Large language models (LLMs) provide cost-effective but possibly inaccurate predictions of human behavior. Despite growing evidence that predicted and observed behavior are often not "interchangeable," there is limited guidance on using LLMs to obtain valid estimates of causal effects and other parameters. We argue that LLM predictions…