Reviewed articles published in the "Journal of Applied Psychology" (JAP) to determine how interpretations might have differed if standardized regression coefficients and structure coefficients (or bivariate "r"s of predictors with the criterion) had been interpreted. Summarizes some dramatic misinterpretations or incomplete…

Methods of regression commonality analysis are generalized for use in canonical correlation analysis. An actual data set (involving educators' attitudes toward death and age, locus of control, religion, and occupational role in working with terminally ill children) is employed to illustrate the extension. The method can be applied with respect to…

Stepwise methods are frequently employed in educational and psychological research, both to select useful subsets of variables and to evaluate the order of importance of variables. Three problems with stepwise applications are explored in some detail. First, computer packages use incorrect degrees of freedom in their stepwise computations,…

Virtually all parametric statistical procedures have been shown to be special cases of canonical correlation analysis. This article proposes a logic for conducting stepwise canonical correlation analyses, based upon evaluation of canonical communality coefficients. The procedure is a direct analogue of stepwise multiple regression. (Author/RD)

Various realizations have led to less frequent use of the "OVA" methods (analysis of variance--ANOVA--among others) and to more frequent use of general linear model approaches such as regression. However, too few researchers understand all the various coefficients produced in regression. This paper explains these coefficients and their…

The advantages and disadvantages of three analytic methods used to analyze experimental data in educational research are discussed. The same hypothetical data set is used with all methods for a direct comparison. The Analysis of Variance (ANOVA) method and its several analogs are collectively labeled OVA methods and are evaluated. Regression…

Three criticisms of overreliance on results from statistical significance tests are noted. It is suggested that: (1) statistical significance tests are often tautological; (2) some uses can involve comparisons that are not completely sensible; and (3) using statistical significance tests to evaluate both methodological assumptions (e.g., the…

This paper reports a Monte Carlo study of differences induced by different analysis choices over selected types of aptitude treatment interaction (ATI) data (nine combinations of three sample sizes and three population parameter effect sizes). Generally, ANOVA methods tended to overestimate smaller effect sizes and to underestimate larger effect…

Advantages and disadvantages of three analytic methods are discussed. A hypothetical data set is employed to make the discussion complete. It is suggested that commonality analysis can be usefully employed in research studies in education, particularly when aptitute-treatment interaction studies are involved. (Author/LMO)

Conventional statistical significance tests do not inform the researcher regarding the likelihood that results will replicate. One strategy for evaluating result replication is to use a "bootstrap" resampling of a study's data so that the stability of results across numerous configurations of the subjects can be explored. This paper…

Selected methods for evaluating the stability of research results empirically are described, especially with regard to multivariate analysis. It is critically important to evaluate the influences of sampling error on obtained results; statistical significance testing does not inform judgment regarding the probable replicability or the sampling…

The relationship between analysis of variance (ANOVA) methods and their analogs (analysis of covariance and multiple analyses of variance and covariance--collectively referred to as OVA methods) and the more general analytic case is explored. A small heuristic data set is used, with a hypothetical sample of 20 subjects, randomly assigned to five…

Value-added models, which rate schools for effectiveness while taking into account the poverty and other socioeconomic status of the students, are generating increased interest. This paper describes the use of one such model to evaluate whether school ratings changed when three new programs were introduced: the "Target Teach" curriculum…

Although methodological integrity is not the sole determinant of the value of a program evaluation, decision-makers do have a right, at a minimum, to be able to expect competent work from evaluators. This paper explores five areas where evaluators might improve methodological practices. First, evaluation reports should reflect the limited…