This comment discusses some issues about the use and reporting of experimental studies in education, illustrated by a recently published study that claimed (i) that an educational innovation was effective despite outcomes not reaching statistical significance, and (ii) that this refuted the findings of an earlier study. The two key issues raised…

Recently, Stephen Gorard has outlined strong objections to the use of significance testing in social research. He has argued, first, that as the samples used in social research are almost always non-random it is not possible to use inferential statistical techniques and, second, that even if a truly random sample were achieved, the logic behind…

This "What Works Clearinghouse Procedures and Standards Handbook (Version 3.0)" provides a detailed description of the standards and procedures of the What Works Clearinghouse (WWC). The remaining chapters of this Handbook are organized to take the reader through the basic steps that the WWC uses to develop a review protocol, identify…

Null hypothesis significance testing has dominated quantitative research in education and psychology. However, the statistical significance of a test as indicated by a p-value does not speak to the practical significance of the study. Thus, reporting effect size to supplement p-value is highly recommended by scholars, journal editors, and academic…

Very little explanatory power is required in order for regressions to exhibit statistical significance. This article discusses some of the causes and implications. (Contains 2 tables.)

Statistical significance reporting and use in educational and psychological research is reviewed. An assessment of the use of statistical significance in articles published in MECD from 1990-1996 is presented. The elements of statistical significance (including sample size, effect size, and power), interpretation of results, common erroneous…

Despite the increasing criticism of statistical significance testing by researchers, particularly in the publication of the 1994 American Psychological Association's style manual, statistical significance test results are still popular in journal articles. For this reason, it remains important to understand the logic of inferential statistics. A…

This working paper contains the overheads used in a seminar designed to introduce some basic concepts of statistics to nonstatisticians. The seminar has been presented on several occasions. The first part of the seminar, and the first set of overheads, deals with the essentials of statistics, including: (1) population, sample, and inference; (2)…

Presents an opinion on the appropriate use of significance tests, especially in the context of regression analysis, the most commonly encountered statistical technique in education and related disciplines. Briefly discusses the appropriate use of power analysis. Contains 47 references. (Author/SV)

Discusses five common myths about statistical significance testing (SST), the possible erroneous and harmful contributions of SST to educational research, and suggested alternatives to SST for mathematics education research. (Contains 61 references.) (MKR)

The use of tests of statistical significance was explored, first by reviewing some criticisms of contemporary practice in the use of statistical tests as reflected in a series of articles in the "American Psychologist" and in the appointment of a "Task Force on Statistical Inference" by the American Psychological Association…

An empirical investigation of methodological issues associated with evaluating treatment effect in single-subject research (SSR) designs is presented. This investigation: (1) conducted a generalizability (G) study to identify the sources of systematic and random measurement error (SRME); (2) used an analytic approach based on G theory to integrate…