In this paper, we address the following questions: What misconceptions do high school students exhibit in their first encounter with significance test problems through a repeated sampling approach? Which theory or framework could explain the presence and features of such patterns? With brief prior instruction on the use of Fathom software to…

There is a growing concern that much of the published research literature is distorted by the pursuit of statistically significant results. In a seminal article, Ioannidis and Trikalinos (2007, "Clinical Trials") proposed an omnibus (I&T) test for significance chasing (SC) biases. This test compares the observed number of studies…

This article advocates that introductory statistics be taught by basing all calculations on a single simple margin-of-error formula and deriving all of the standard introductory statistical concepts (confidence intervals, significance tests, comparisons of means and proportions, etc) from that one formula. It is argued that this approach will…

When selecting a multilevel model to fit to a dataset, it is important to choose both a model that best matches characteristics of the data's structure, but also to include the appropriate fixed and random effects parameters. For example, when researchers analyze clustered data (e.g., students nested within schools), the multilevel model can be…

This comment discusses some issues about the use and reporting of experimental studies in education, illustrated by a recently published study that claimed (i) that an educational innovation was effective despite outcomes not reaching statistical significance, and (ii) that this refuted the findings of an earlier study. The two key issues raised…

In this study, we re-analyze recent empirical research on replication from a meta-analytic perspective. We argue that there are different ways to define "replication failure," and that analyses can focus on exploring variation among replication studies or assess whether their results contradict the findings of the original study. We…

For the two-way factorial design in analysis of variance, the current article explicates and compares three methods for controlling the Type I error rate for all possible simple interaction contrasts following a statistically significant interaction, including a proposed modification to the Bonferroni procedure that increases the power of…

This article considers the nature and place of tests of statistical significance (ToSS) in science, with particular reference to psychology. Despite the enormous amount of attention given to this topic, psychology's understanding of ToSS remains deficient. The major problem stems from a widespread and uncritical acceptance of null hypothesis…

Purpose: Evidence-based data analysis methods are important in clinical research fields, including speech-language pathology and audiology. Although commonly used, null hypothesis significance testing (NHST) has several limitations with regard to the conclusions that can be drawn from results, particularly nonsignificant findings. Bayes factors…

This paper compares the use of confidence intervals (CIs) and a sensitivity analysis called the number needed to disturb (NNTD), in the analysis of research findings expressed as 'effect' sizes. Using 1,000 simulations of randomised trials with up to 1,000 cases in each, the paper shows that both approaches are very similar in outcomes, and each…

Early intervention efforts have been effective in reducing disruptive behaviors and the probability of poor developmental outcomes. Early interventions include common practice elements to improve social functioning and decrease problem behaviors that disrupt the teaching-learning process. The First Step Next intervention has been well validated…

The purposes of this manuscript are to identify common statistical mistakes in sport management, and to provide scholars with suggestions on how to develop and improve the quality of quantitative research. We have reviewed articles published from 2001 to 2017 in the "Journal of Sport Management," "Sport Management Review,"…

A key job of education policymakers is to make decisions under uncertainty. They must weigh the risks, rewards, and costs of different interventions, policies, and mixes of resources, and make decisions even when the likely outcome is uncertain. How policymakers think about and deal with uncertainty has important implications for resource…

This study investigated the relationships between perceived teacher empowerment, job satisfaction, and organizational climate among 206 randomly sampled kindergarten teachers in Shanghai, China. Chinese versions of the School Participant Empowerment Scale, the Teacher Work Satisfaction Survey, and the Organizational Climate Scale were validated…

University and college institutional advertisements, which typically are broadcast as public service announcements during the halftime of football games, were the subject of a quantitative analysis focused on commonality in messaging and employment of the semiotic theory of brand advertising. Findings indicate advertisements focus on students'…