Context: A perennial problem in applied statistics is the inability to justify strong claims about cause-and-effect relationships without full knowledge of the mechanism determining selection into treatment. Few research designs other than the well-implemented random assignment study meet this requirement. Researchers have proposed partial…

There has been much controversy over the null hypothesis significance testing procedure, with much of the criticism centered on the problem of inverse inference. Specifically, p gives the probability of the finding (or one more extreme) given the null hypothesis, whereas the null hypothesis significance testing procedure involves drawing a…

Null hypothesis significance testing (NHST) has been the subject of debate for decades and alternative approaches to data analysis have been proposed. This article addresses this debate from the perspective of scientific inquiry and inference. Inference is an inverse problem and application of statistical methods cannot reveal whether effects…

Multilevel models are often used to evaluate hypotheses about relations among constructs when data are nested within clusters (Raudenbush & Bryk, 2002), although alternative approaches are available when analyzing nested data (Binder & Roberts, 2003; Sterba, 2009). The overarching goal of this article is to suggest when it is appropriate…

Many methods exist for imputing missing data but fewer methods have been proposed to test the missing data mechanism. Little (1988) introduced a multivariate chi-square test for the missing completely at random data mechanism (MCAR) that compares observed means for each pattern with expectation-maximization (EM) estimated means. As an alternative,…

Examines potential problems in the use of inferential statistics for single population proportions, differences between two population proportions, and quotients of two population proportions. Discusses hypothesis testing versus interval estimation. Emphasizes the importance of selecting the appropriate formula for the standard error and…

Suggests that testing developmental ordering hypotheses is difficult because rare use of ratio scales prevents direct comparison of measures. Demonstrates that the observed data pattern is constrained by the underlying relationship--although observed data pattern may not reflect the exact relationship, it limits possible relationships. Shows the…

The central thesis of this article is that the process of eliminating validity threats depends fundamentally on no-difference findings, a fact that has not been made explicit by researchers. The implications of this neglect are explored using examples from a number of different substantive areas such as psychology, health, and medicine.…

Discusses five common myths about statistical significance testing (SST), the possible erroneous and harmful contributions of SST to educational research, and suggested alternatives to SST for mathematics education research. (Contains 61 references.) (MKR)

This paper evaluates the logic underlying various criticisms of statistical significance testing and makes specific recommendations for scientific and editorial practice that might better increase the knowledge base. Reliance on the traditional hypothesis testing model has led to a major bias against nonsignificant results and to misinterpretation…