As the 1994 publication manual of the American Psychological Association emphasized, "p" values are affected by sample size. As a result, it can be helpful to interpret the results of statistical significant tests in a sample size context by conducting so-called "what if" analyses. However, these methods can be inaccurate…

The program evaluation standards approved by the American National Standards Institute (ANSI) in 1994 that deal with reliability and validity accurately represent contemporary views of the psychometric community with regard to reliability and validity. As such, these standards move the field forward. The ANSI standards recognize that reliability…

Proposes and illustrates a new method by which "what if" analyses can be conducted using estimated true population effects. Use of these "what if" methods may prevent authors with large sample sizes from overinterpreting their small effects once they see that the small effects would no longer have been statistically significant with only a…

Reviews practices regarding tests of statistical significance and policies of the American Educational Research Association (AERA). Decades of misuse of statistical significance testing are described, and revised editorial policies to improve practice are highlighted. Correct interpretation of statistical tests, interpretation of effect sizes, and…

Various realizations have led to less frequent use of the "OVA" methods (analysis of variance--ANOVA--among others) and to more frequent use of general linear model approaches such as regression. However, too few researchers understand all the various coefficients produced in regression. This paper explains these coefficients and their…

Investigates magnitude of concept-scale interactions when different teachers are evaluated. Suggests such interactions can alter the meaning ascribed to those scales by raters and lessen the comparability of ratings of different teachers. (Author/AH)

In early intervention, researchers often are interested in interpretation aids that can help determine the relative importance of variables when multiple regression models are used, and that facilitate deeper insight into prediction dynamics. Commonality analysis is one approach for helping researchers understand the contributions independent or…

The research literature provides important guidance to counselors working to keep abreast of the latest thinking regarding best practices and recently developed counseling tools. The purpose of this digest is to highlight a few errors that seem to recur within the literature, and to provide some helpful references that further explore these…

The use of tests of statistical significance was explored, first by reviewing some criticisms of contemporary practice in the use of statistical tests as reflected in a series of articles in the "American Psychologist" and in the appointment of a "Task Force on Statistical Inference" by the American Psychological Association…

Too few researchers understand what statistical significance testing does and does not do, and consequently their results are misinterpreted. This Digest explains the concept of statistical significance testing and discusses the meaning of probabilities, the concept of statistical significance, arguments against significance testing,…

Effect sizes have been increasingly emphasized in research as more researchers have recognized that: (1) all parametric analyses (t-tests, analyses of variance, etc.) are correlational; (2) effect sizes have played an important role in meta-analytic work; and (3) statistical significance testing is limited in its capacity to inform scientific…

Latent trait measurement theory was used to investigate the measurement characteristics of both parts of a multiple-choice measure of field-independence, the Finding Embedded Figures Test (FEFT). Analysis was based on data provided by 1,528 students enrolled in one of two middle schools located in the southern United States. Of the subjects, 731…

Dissertations are an important component of the effort to generate knowledge. Thus, dissertation quality may be seen by accreditation and coordinating-board reviewers as a noteworthy reflection on the quality of doctoral programs themselves. The present study reviews methodological errors within Ph.D. dissertations. The illustrative errors are…