Provides a sample of seven published studies in different disciplines that inappropriately generalized reliability coefficients involving several raters to scores generated by a single rater. Score reliability when only one rater is used for scoring is lower than the score reliability for which two raters are used. (SLD)

Studied the reporting preferences of editorial board members of four scholarly journals in education and psychology with regard to analytic practices in the fifth edition of the American Psychological Association "Publication Manual." Responses of 106 board members show the movement toward reform in research reporting practices. (SLD)

Examined the frequency of use of various types of reliability coefficients for a sample of 696 tests in a directory published by the American Psychological Association. Coefficient alpha was the overwhelming favorite, and several measures treated almost universally in textbooks were rarely or never used. Identifies problems encountered in the…

Discusses reliability issues in light of recent studies and debates focused on psychometrics versus datametrics terminology and reliabilities generalization. Discusses the way multiple perspectives on score reliability may affect research practice, editorial policies, and reliability generalization across studies. (SLD)

Identifies deficiencies in the new edition of the American Psychological Association's "Publication Manual," drawing on interviews with some critics and other parties. These include lack of follow-through with explanations and examples of how to report the statistics now recommended. (SLD)

Discusses four reasons for promoting the use of confidence intervals: (1) ease of interpretation; (2) links with familiar statistical tests; (3) promotion of meta-analytic thinking; and (4) increase of information about precision. Discusses calculations of confidence intervals for a basic standardized effect size measure and discusses software for…

This paper makes the point that descriptive and inferential measures of nonnormality and graphic displays of the frequency distribution of important variables should be routine in research reporting. This point is particularly true for research involving measures with nonarbitrary metrics where the distribution shape is unaffected by measurement…

Analyzed 150 articles from the "Journal of Applied Psychology" (JAP) from 1940 to 1999 to determine statistical reporting practices related to null hypothesis significance testing, American Psychological Association guidelines, and reform recommendations. Findings show little evidence that decades of cogent criticisms by reformers have…

Illustrates a number of ways in which confidence intervals for reliability coefficients can be estimated. Suggests that authors who submit articles to "Educational and Psychological Measurement" report confidence intervals for reliability estimates whenever they report score reliabilities and that they note the interval estimation…

Purpose of the investigation was to identify the factorial dimensions of 25 characteristics on a scale for rating research articles. (Authors)

When reporting results, researchers must not change predetermined significance levels. Such attempts to make results more significant are statistically inaccurate, illogical, and unethical. American Psychological Association standards for reporting significance should be more explicit. (CP)

Three dozen observations and recommendations concerning the activities of research consultants in the behavioral sciences are listed and discussed. These guidelines, some stated as aphorisms, are discussed in the categories of planning; implementation; analysis; interpretation; and reporting. (Author/SLD)

This study describes how much and in what ways authors of research studies fail to include adequate information about data collection. Results based on analysis of 220 articles from 22 journals show that the quality of measurement reporting continues to be a problem. (SLD)

Introduces the three subsequent articles from this special section, which extend the discussion of future prospects for progress in the reporting and interpreting of effect sizes by researchers. The authors of these pieces represent diverse views. (SLD)

Makes the case that science is best served when researchers focus on the size of effects and their practical significance. Advocates the use of confidence intervals for deciding whether chance or sampling variability is an unlikely explanation for an observed effect. Calls for more emphasis on effect sizes in the next edition of the American…