Students often believe that statistical significance is the only determinant of whether a quantitative result is "important." In this paper, I review traditional null hypothesis statistical testing to identify what questions inferential statistics can and cannot answer, including statistical significance, effect size and direction,…

In their response to our paper, Nicholson and Ridgway agree with the majority of what we wrote. They echo our concerns about the misuse of inferential statistics and NHST in particular. Very little of their response explicitly challenges the points we made but where it does their defence of the use of inferential techniques does not stand up to…

A classic TV commercial once asked, "How many licks does it take to get to the center of a Tootsie Roll Tootsie Pop?" The narrator claims, "The world may never know" (Tootsie Roll 2012), but an Internet search returns a multitude of answers, some of which include rigorous systematic approaches by academics to address the…

A multiple testing method for examining factorial invariance for latent constructs evaluated by multiple indicators in distinct populations is outlined. The procedure is based on the false discovery rate concept and multiple individual restriction tests and resolves general limitations of a popular factorial invariance testing approach. The…

The self-correcting nature of psychological and educational science has been seriously questioned. Recent special issues of "Perspectives on Psychological Science" and "Psychology of Aesthetics, Creativity, and the Arts" have roundly condemned current organizational models of research and dissemination and have criticized the…

Asterisks should not be used to indicate if the result of a hypothesis test is significant.

A quiet methodological revolution, a modeling revolution, has occurred over the past several decades, almost without discussion. In contrast, the 20th century ended with contentious argument over the utility of null hypothesis significance testing (NHST). The NHST controversy may have been at least partially irrelevant, because in certain ways the…

Null distributions of permutation tests for two-sample, paired, and block designs are simulated using the R statistical programming language. For each design and type of data, permutation tests are compared with standard normal-theory and nonparametric tests. These examples (often using real data) provide for classroom discussion use of metrics…

This paper proposes an argument framework for the teaching of null hypothesis statistical testing and its application in support of research. Elements of the Toulmin (1958) model of argument are used to illustrate the use of p values and Type I and Type II error rates in support of claims about statistical parameters and subject matter research…

Very little explanatory power is required in order for regressions to exhibit statistical significance. This article discusses some of the causes and implications. (Contains 2 tables.)

Hypothesis testing is one of the more difficult concepts for students to master in a basic, undergraduate statistics course. Students often are puzzled as to why statisticians simply don't calculate the probability that a hypothesis is true. This article presents an exercise that forces students to lay out on their own a procedure for testing a…

A procedure is discussed for testing the significance of the difference in two correlated correlation coefficients, using Fisher's Z-Transformation. The procedure is applicable to a wide range of problems involving tests between dependent correlations and has documented mathematical support when its power curves are examined. (MH)

Although statistical significance testing as the sole basis for result interpretation is a flawed practice, significance tests can be useful as one of three criteria that must be demonstrated to establish a position empirically. Statistical significance testing provides evidence that an event did not happen by chance but gives no evidence of the…

Details are given for the use of the mixed effects multivariate analysis of variance table provided by the BMD12V computer program to compute raw generalized variances and hence the U and F statistics for the mixed effects model. (Author/JKS)

This article describes an example which is useful when teaching hypothesis testing in order to highlight the interrelationships that exist among the level of significance, the sample size and the statistical power of a test. The example also allows students to see how what they learn in the classroom directly affects the content of some of the…