A recent issue of the Australian Journal of Language and Literacy included an article reporting on a systematic narrative review of the research literature that indicated that there was insufficient evidence to conclude whether genre theory and systemic functional linguistics either 'worked' or 'did not work'. The criteria used to evaluate these…

I. Purpose of the Study: Detecting whether interventions work or not (through main effect analysis) can provide empirical evidence regarding the causal linkage between malleable factors (e.g., interventions) and learner outcomes. In complement, moderation analyses help delineate for whom and under what conditions intervention effects are most…

Meta-analytic studies of second language (L2) learning typically employ a classic approach to meta-analysis. Although the classic approach can clarify findings, a multivariate, multilevel meta-analysis (3M) approach increases transparency by accounting for (a) dependencies in the evidence presented by primary studies, (b) methodological…

In 2020, the What Works Clearinghouse (WWC) for single-case experimental design (SCED) studies released an updated version of their standards and procedures handbook (Version 4.1). Because of these updates, there is a need to understand the implications for the field in terms of quality rating of the design, and subsequent synthesis of evidence at…

In fields (such as population health) where randomised trials are often lacking, systematic reviews (SRs) can harness diversity in study design, settings and populations to assess the evidence for a putative causal relationship. SRs may incorporate causal assessment approaches (CAAs), sometimes called 'causal reviews', but there is currently no…

When higher education students are assessed multiple times, teachers need to consider how these assessments can be combined into a single pass or fail decision. A common question that arises is whether students should be allowed to take a resit. Previous research has found little to no clear learning benefits of resits and therefore suggested they…

Background/Context: Methodological background: Meta-analysis typically depends on the assumption that true effects follow the normal distribution. While assuming normality of effect "estimates" is often supported by a central limit theorem, normality for the distribution of interventions' "true" effects is a computational…

Background/Context: Some observational studies involve multiple layers of treatment selection, specifically in the context of the extended time accommodation (ETA) for English language learners (ELLs). In ETA settings, the first selection occurs due to the eligibility rule, where students whose ELL English proficiency is below a certain threshold…

This article, part of the "Learning Disability Quarterly" special series dedicated to single-case research design (SCRD), summarizes the review process and outlines how to provide high-quality SCRD study manuscript reviews. We offer recommendations for the entirety of the review process, including accepting the role of reviewer,…

Multilevel meta-analysis is an innovative synthesis technique used for the quantitative integration of effect size estimates across participants and across studies. The quantitative summary allows for objective, evidence-based, and informed decisions in research, practice, and policy. Based on previous methodological work, the technique results in…

Background/Context: When a researcher tests an educational program, product, or policy in a randomized controlled trial (RCT) and detects a significant effect on an outcome, the intervention is usually classified as something that "works." When the expected effects are not found, however, there is seldom an orderly and transparent…

Understanding the impact of undergraduate research experiences (UREs) and course-based undergraduate research experiences (CUREs) is crucial as universities debate the value of allocating scarce resources to these activities. We report on the Berkeley Undergraduate Research Evaluation Tools (BURET), designed to assess the learning outcomes of UREs…

Recent studies have suggested single case design (SCD) intervention research is subject to publication bias, wherein studies are more likely to be published if they possess large or statistically significant effects and use rigorous experimental methods. The nature of SCD and the purposes for which it might be used could suggest that large effects…

Background: The use of randomised controlled trials (RCTs) in education has increased significantly over the last 15 years. However, their use has also been subject to sustained and rather trenchant criticism from significant sections of the education research community. Key criticisms have included the claims that: it is not possible to undertake…

Graduate teaching assistants (GTAs) are responsible for direct instruction of geoscience undergraduate students at an array of universities and have a major effect on the knowledge, beliefs, and practices of their students. GTAs benefit from in-department training in both beliefs and practices that align with the existing literature on teaching…