Randomized controlled trials have proliferated in education, in part because they provide an unbiased estimator for the causal impact of interventions. It is increasingly recognized that many such trials in education have low power to detect an effect if indeed there is one. However, it is less well known that low powered trials tend to…

The purposes of this manuscript are to identify common statistical mistakes in sport management, and to provide scholars with suggestions on how to develop and improve the quality of quantitative research. We have reviewed articles published from 2001 to 2017 in the "Journal of Sport Management," "Sport Management Review,"…

This letter documents some problems in Ancaiani et al. (2015). Namely the evaluation of concordance, based on Cohen's kappa, reported by Ancaiani et al. was not computed on the whole random sample of 9,199 articles, but on a subset of 7,597 articles. The kappas relative to the whole random sample were in the range 0.07-0.15, indicating an…

Traditions of statistical significance testing in second language (L2) quantitative research are strongly entrenched in how researchers design studies, select analyses, and interpret results. However, statistical significance tests using "p" values are commonly misinterpreted by researchers, reviewers, readers, and others, leading to…

The educational literature, the popular press, and educated laypeople have all echoed a conclusion from the book "Academically Adrift" by Richard Arum and Josipa Roksa (which has now become received wisdom), namely, that 45% of college students showed no significant gains in critical thinking skills. Similar results were reported by…

Recently, Stephen Gorard has outlined strong objections to the use of significance testing in social research. He has argued, first, that as the samples used in social research are almost always non-random it is not possible to use inferential statistical techniques and, second, that even if a truly random sample were achieved, the logic behind…

Given the American Psychological Association's strong recommendation to always report effect sizes in research, scholars have a responsibility to provide complete information regarding their findings. The purposes of this study were to (a) determine the frequencies with which different effect sizes were reported in published, peer-reviewed…

Just as PK-12 teachers are taught how to teach, college and university professors should also receive instruction in how to teach. They should acquire pedagogical skills and understand methods for planning and content delivery prior to entering the classroom. The knowledge base of the discipline and a focus on research are emphasized in the…

This "What Works Clearinghouse Procedures and Standards Handbook (Version 3.0)" provides a detailed description of the standards and procedures of the What Works Clearinghouse (WWC). The remaining chapters of this Handbook are organized to take the reader through the basic steps that the WWC uses to develop a review protocol, identify…

This paper challenges Hanushek and Woessmann's (2008) contention that the quality and not the quantity of schooling determines a nation's rate of economic growth. I first show that their statistical analysis is flawed. I then show that when a nation's average test scores and average schooling attainment are included in a national income model,…

Classic parametric statistical significance tests, such as analysis of variance and least squares regression, are widely used by researchers in many disciplines, including psychology. For classic parametric tests to produce accurate results, the assumptions underlying them (e.g., normality and homoscedasticity) must be satisfied. These assumptions…

It is suggested that the binomial theorem be used to compute the probability that a given number of Type I errors would occur when a group of null hypotheses are true and that this result be used as the level of significance for the test of an overall hypothesis. (Author/BW)

Too often researchers rely upon the classical normal theory parametric tests to analyze non-normal data, even though the tests may not be robust to violations of that assumption. Fligner's class of two-sample tests for scale is an important development because the test is distribution-free and has desirable properties. This paper outlines the…

Communication researchers have recently expressed concern with the lack of statistical power in their literature. Authors propose a method for increasing statistical power: the partitioning of the decision region in three parts. This procedure results in an unambiguous interpretation of nonsignificant results and leads to increased power. (PD)

A dialog between two fictional teachers provides some basic examples of how research that uses approved methodology may provide results that are significant statistically but not significant practically. (PGD)