Using a sample of 70,399 published p-values from 192 meta-analyses, we empirically estimate the counterfactual distribution of p-values in the absence of any biases. Comparing observed p-values with counterfactually expected p-values allows us to estimate how many p-values are published as being statistically significant when they should have been…

Fetters et al.'s (2013) mixed methods integration framework uses construction metaphors: building, connecting, merging, and embedding. In a similar vein, this article uses an architectural metaphor and introduces design patterns as building blocks for mixed methods research design. A design pattern embeds one specific design decision into its…

This paper argues for three aspects of tolerance with respect to QCA research: tolerance with respect to different approaches to QCA; producing QCA research with tolerance (work that is resistant to criticism); and for QCA researchers to be clear about the tolerance of the solutions they present -- especially in terms of calibration and truth…

Researchers increasingly take advantage of the comparative case design to build theory, but the degree of case dependence is occasionally discussed and theorized. We suggest that the comparative case study design might be subject to an often underappreciated threat--dependence across cases--under certain conditions. Using research on innovation…

This study aims to introduce network meta-analysis (NMA) to provide educational researchers with an extended view of the reviewing educational research. Meta-analytic methods have been widely used in educational research reviews. However, weaknesses have emerged in the multi-group comparison analysis of educational studies where different…

This article develops new closed-form variance expressions for power analyses for commonly used difference-in-differences (DID) and comparative interrupted time series (CITS) panel data estimators. The main contribution is to incorporate variation in treatment timing into the analysis. The power formulas also account for other key design features…

As a research field geared toward understanding and improving learning, Learning Analytics (LA) must be able to provide empirical support for causal claims. However, as a highly applied field, tightly controlled randomized experiments are not always feasible nor desirable. Instead, researchers often rely on observational data, based on which they…

Designing and interpreting controlled experiments are important inquiry skills addressed in many current science curricula. The relevant skills associated with the design and interpretation of controlled experiments are summarised under the term control-of-variables strategy (CVS). Research on elementary school students' CVS skills shows that they…

Inquiry-based laboratories were implemented into a General Chemistry Laboratory sequence, and the impact of these exercises on students' experimental design skills was assessed using a four-part assessment developed for this study. This assessment contained a multiple-choice section, a section asking students to explain their reasoning behind a…

Comparison studies conducted to determine which instructional interventions are most efficient for teaching discrete behaviors to individuals with disabilities are potentially valuable, although some threats to internal validity may be more likely in these studies. Studies included in this review typically met common internal validity standards,…

Background: Over the past 20 years, much has been written about the merits and limitations of matching methods, particularly propensity score matching (see, e.g., Stuart, 2010 for a review of matching approaches), and the myriad ways matching can be implemented to identify a comparison group that is similar to the treatment group on observable…

We conducted a systematic literature review of 111 empirical studies published between 1969 and 2018. We found that research does not exponentially grow in comparison to research on higher education or leadership; empirical research emerging in the 1980s does not completely replace non-empirical inquiry. Studies from English-speaking authors…

Background: Special education services are individualized supports provided to students with disabilities (SWD) that should help them make appropriate educational progress given their abilities, educational needs, and goals (Endrew F. v. Douglas County School District, 2017). Concerns about the benefits of these services have emerged given the…

Communicating with peers often poses challenges for individuals with autism spectrum disorder (ASD); these social interactions, however, can be especially difficult during the teenage years. For many adolescents with ASD, peer interaction is often limited or nonexistent, even for those taught in general education settings. The purpose of this…

Purpose: To assess strengths and weaknesses of the reading level (RL) match approach and its potential to generate insights regarding the cognitive foundations of reading ability and disability. Method: We applied RL-match design to a sample of 2nd-6th graders reading a consistent orthography, Russian, using an "extreme phenotype"…