The purpose of the present study is to provide a historical account and metasynthesis of which statistical techniques are most frequently used in the fields of education and psychology. Six articles reviewing the "American Educational Research Journal" from 1969 to 1997 and five articles reviewing the psychological literature from 1948 to 2001…

The present article provides a primer on using effect sizes in research. A small heuristic data set is used in order to make the discussion concrete. Additionally, various admonitions for best practice in reporting and interpreting effect sizes are presented. Among these is the admonition to not use Cohen's benchmarks for "small," "medium," and…

Researchers have conducted studies of curriculum for doctoral students in various disciplines and subdisciplines. In this study, the authors examined methods (e.g., quantitative, qualitative, mixed methods) requirements for doctoral programs in education. Such requirements operationalize visions about the meaning of educational research and the…

This article reviews the literature on interventions targeting the academic performance of students with attention-deficit/hyperactivity disorder (ADHD) and does so within the context of the statistical significance testing controversy. Both the arguments for and against null hypothesis statistical significance tests are reviewed. Recent standards…

Arthur Levine's (2007) recent report on the academic preparation of education doctoral students, Educating Researchers, http://www.edschools.org/EducatingResearchers/educating_researchers.pdf is highly critical of both AERA and the scholarly training of education doctoral students. One criticism Levine (2007) offers is his view that educational…

Surveyed AERA members regarding their perceptions of: statistical issues and statistical significance testing; the general linear model; stepwise methods; score reliability; type I and II errors; sample size; statistical probabilities as exclusive measures of effect size; p values as direct measures of result value; and p values evaluating…

Investigated the patterns of statistical usage and reporting in 756 articles published in the "American Educational Research Journal" (AERJ) and the "Journal of Counseling Psychology" (JCP) over 10 years. Evaluated the analytic practices used. Also discusses changes that may be necessary to improve statistics in behavioral…

This paper explains in user-friendly terms why multivariate statistics are so important in educational research. The basic logic of canonical correlation analysis is presented as a simple or bivariate Pearson "r" procedure. It is noted that all statistical tests implicitly involve the calculation of least squares weights, and that all…

After presenting a general linear model as a framework for discussion, this paper reviews five methodology errors that occur in educational research: (1) the use of stepwise methods; (2) the failure to consider in result interpretation the context specificity of analytic weights (e.g., regression beta weights, factor pattern coefficients,…

Stepwise methods are frequently employed in educational and psychological research, both to select useful subsets of variables and to evaluate the order of importance of variables. Three problems with stepwise applications are explored in some detail. First, computer packages use incorrect degrees of freedom in their stepwise computations,…

Given decades of lucid, blunt admonitions that statistical significance tests are often misused, and that the tests are somewhat limited in utility, what is needed is less repeated bashing of statistical tests, and some honest reflection regarding the etiology of researchers' denial and psychological resistance (sometimes unconscious) to improved…

As an extension of B. Thompson's 1998 invited address to the American Educational Research Association, this paper cites two additional common faux pas in research methodology and explores some research issues for the future. These two errors in methodology are the use of univariate analyses in the presence of multiple outcome variables (with the…

The advantages and disadvantages of three analytic methods used to analyze experimental data in educational research are discussed. The same hypothetical data set is used with all methods for a direct comparison. The Analysis of Variance (ANOVA) method and its several analogs are collectively labeled OVA methods and are evaluated. Regression…

Editorial board members of "Educational and Psychological Measurement" were asked to list up to five publications each, written since roughly 1950, that they thought were seminal to shaping the measurement field. This bibliography lists 59 of their selections as a "hit parade" of measurement readings. (SLD)

Comments on discussions of statistical significance testing in this special issue, placing the articles in the context of contemporary research literature. Empirical evidence is cited that suggests that the American Psychological Association's policy of encouraging effect size reporting has had no appreciable effect on the research literature.…