A guide for professional counselors and counseling researchers for calculating and interpreting Percent Improvement as an indicator of clinical significance is provided. Strategies for reporting findings are described and illustrated. Guidelines for contextualizing discussions of clinical significance within the boundaries of psychometric evidence…

This paper is directed to researchers who conduct and report education intervention studies. Its purpose is to stimulate and guide them to go a step beyond reporting the statistics that emerge from their analysis of the differences between experimental groups on the respective outcome variables. With what is often very minimal additional effort,…

This paper includes a nontechnical description of methods for calculating effect sizes in intellectual and developmental disability studies. Different hypothetical studies are used to illustrate how null hypothesis significance testing (NHST) and effect size findings can result in quite different outcomes and therefore conflicting results. Whereas…

This article reports results of a corpus-based evaluation of 14 syntactic complexity measures as objective indices of college-level English as a second language (ESL) writers' language development. I analyzed large-scale ESL writing data from the Written English Corpus of Chinese Learners (Wen, Wang, & Liang, 2005) using a computational system…

The National Assessment of Educational Progress (NAEP) is a continuing and nationally representative assessment of what this nation's students know and can do. NAEP has often been called the "gold standard" of assessments because it is developed using the best thinking of assessment and content specialists, education experts, and…

The concept of statistical significance is explained, with specific numerical illustrations. (MNS)

Over the years, methodologists have been recommending that researchers use magnitude of effect estimates in result interpretation to highlight the distinction between statistical and practical significance (cf. R. Kirk, 1996). A magnitude of effect statistic (i.e., effect size) tells to what degree the dependent variable can be controlled,…

In his early exposition of path analysis, Duncan (1966) noted that the method "provides a calculus for indirect effects." Despite the interest in indirect causal effects, most users treat them as if they are population parameters and do not test whether they are statistically significant. Sobel (1982) has recently derived the asymptotic…

A dialog between two fictional teachers provides some basic examples of how research that uses approved methodology may provide results that are significant statistically but not significant practically. (PGD)

The second half of a dialogue between two fictional teachers examines the significance of statistical significance in research and considers the factors affecting the extent to which research results provide important or useful information. (PGD)

Although the primary responsibility rests with the authors of articles reporting nonsignificant results to demonstrate the worth of the results by discussing the power of the tests, consumers should be prepared to conduct their own power analyses. This article demonstrates the use of power analysis for the interpretation of nonsignificant…

An explanation of the misuse of statistical significance testing and the true meaning of "significance" is offered. Literature about the criticism of current practices of researchers and publications is reviewed in the context of tests of significance. The problem under consideration occurs when researchers attempt to do more than just establish…

The random variable p and its functions figure in several "tests of combined significance," meta-analysis summaries based on sample significance values, and ps have been used singly, as well as in other tests for evaluating the outcomes of individual research studies. In this work, asymptotic distributions of the sample one-sided…

Designed to measure end-users' efficacy in online searching based on their presearch experience with printed literature retrieval systems for the search topic, this experiment examined differences between manual searches preceding electronic searches and vice versa. The results were not statistically significant due to the small sample size.…

The original nine programs for semantic differential analysis have been condensed into three programs which have been further refined and augmented. They yield: (1) means, standard deviations, and standard errors for each subscale on each concept; (2) Evaluation, Potency, and Activity (EPA) means, standard deviations, and standard errors; (3)…