Practitioners and policymakers rely on meta-analyses to inform decision making around the allocation of resources to individuals and organizations. It is therefore paramount to consider the validity of these results. A well-documented threat to the validity of research synthesis results is the presence of publication bias, a phenomenon where…

This meta-analysis provides an extensive and organized summary of intervention studies in education that are grounded in motivation theory. We identified 74 published and unpublished papers that experimentally manipulated an independent variable and measured an authentic educational outcome within an ecologically valid educational context. Our…

In this meta-analysis, we systematically reviewed research on digital games and learning for K-16 students. We synthesized comparisons of game versus nongame conditions (i.e., media comparisons) and comparisons of augmented games versus standard game designs (i.e., value-added comparisons). We used random-effects meta-regression models with robust…

Peer review is valued in higher education, but also widely criticized in terms of potential biases, particularly gender. We evaluate gender differences in peer reviews of grant applications, extending Bornmann, Mutz, and Daniel's meta-analyses that reported small gender differences in favor of men (d = 0.04), but a substantial heterogeneity in…

Meta-analytic techniques were used to summarize the statistical robustness literature on Type I error properties of alternatives to the one-way analysis of variance "F" test. The James (1951) and Welch (1951) tests performed best under violations of the variance homogeneity assumption, although their use is not always appropriate. (SLD)

Multivocal literatures are comprised of all accessible writings on a common topic and embody the views of diverse authors. The exploratory case study method is proposed as a way to ensure rigor in reviews of such literatures because it addresses problems associated with the selective use of data. (SLD)

Research over the past 15 years concerning parametric and nonparametric statistics and 10 recently developed nonparametric techniques for testing interactions in experimental design are reviewed. The new nonparametric techniques are robust, powerful, versatile, and easy to compute. A fabricated dataset is used to illustrate the technique. (TJH)

The use of analysis of variance (ANOVA), multivariate analysis of variance (MANOVA), and analysis of covariance (ANCOVA) by educational researchers was studied through review of several prominent research journals. The analyses suggest that researchers rarely verify that validity assumptions are satisfied, and typically use analyses that are…