Lord's Paradox occurs when a continuous covariate is statistically controlled for and the relationship between a continuous outcome and group status indicator changes in both magnitude and direction. This phenomenon poses a challenge to the notion of evidence-based policy, where data are supposed to be self-evident. We examined 50 effect size…

Structured means analysis is a very useful approach for testing hypotheses about population means on latent constructs. In such models, a z test is most commonly used for testing the statistical significance of the relevant parameter estimates or of the differences between parameter estimates, where a z value is computed based on the asymptotic…

The authors examined 2 approaches for determining the required sample size of Welch's test for detecting equality of means when the greatest difference between any 2 group means is given. It is shown that the actual power obtained with the sample size of the suggested approach is consistently at least as great as the nominal power. However, the…

The Institute of Education Sciences has funded more than 100 experiments to evaluate educational interventions in an effort to generate scientific evidence of program effectiveness on which to base education policy and practice. In general, these studies are designed with the goal of having adequate statistical power to detect the average…

The sample size determination is an important issue for planning research. However, limitations in size have seldom been discussed in the literature. Thus, how to allocate participants into different treatment groups to achieve the desired power is a practical issue that still needs to be addressed when one group size is fixed. The authors focused…

There is general agreement that meta-analysis is an important tool for synthesizing study results in quantitative educational research. Yet, a shared feature of many meta-analyses is a failure to report sufficient information for readers to fully judge the reported findings, such as the populations to which generalizations are to be made,…

A Monte Carlo technique was employed in order to compare the relative power and robustness of the Bartlett, Cochran, Hartley, and Levene tests for homogeniety of variance. (Editor)

One of the most frequently encountered problems in educational research and evaluation is that of evaluating the effect of a treatment in settings over which the researcher or evaluator has little control. A strong plea is made for use of quasi-experimental designs in these situations and this research provides a method of testing the hypothesis…

Critiques some of the strong positions taken by Bruce Thompson on a variety of methodological issues. Provides alternative views, and discusses the role of editorial policy in research reporting. Focuses on: (1) stepwise methods; (2) context specificity; (3) weights and structure coefficients; (4) reliability as a characteristic of scores and not…

Asserts that editors should declare their expectations publicly and expose the rationale for editorial policies to public scrutiny. Supports effect size reporting and the reporting of score reliabilities. Argues against stepwise methods. Also discusses the interpretation of structure coefficients and the use of confidence intervals. (SLD)

Replies to Bruce Thompson's positions on research methodology and editorial policy, addressing each of these issues in the ongoing discussion: (1) structure coefficients; (2) stepwise regression; (3) test reliability; (4) effect sizes; and (5) meta-analysis. (SLD)

Investigated the patterns of statistical usage and reporting in 756 articles published in the "American Educational Research Journal" (AERJ) and the "Journal of Counseling Psychology" (JCP) over 10 years. Evaluated the analytic practices used. Also discusses changes that may be necessary to improve statistics in behavioral…

The purpose of this report is to describe a process for creating unique multiple linear regression problems for each student. Three formulas were utilized to define intercorrelations between variables. (Editor)

Methods for estimating effect sizes when complete data are not reported are presented. When not precise, these methods provide a conservative estimate and, therefore, allow for the inclusion in a meta-analysis of relevant studies whose data might otherwise be discarded. (Author/PN)

This study identified personality differences between college students who had and who did not have emotional problems, and then used these personality differences to predict emotional problems in other students. (Editor)