Evaluated were three analysis strategies for evaluating instructional treatments: univariate ANOVAs (a one-factor analysis of variance for each criterion); repeated measures ANOVA (a two-factor analysis in which the criteria become the second factor); and multivariate ANOVA (a one-factor analysis in which the criteria are considered…

Practical methods for implementing true experimental designs in evaluation settings in which such designs are rarely used are presented. Particular attention is paid to educational settings. (Author/JKS)

It is argued that there is a benefit to applying techniques of exploratory data analysis (EDA) to program evaluation. The evaluation of a rehabilitation program for people with rheumatoid arthritis (20 subjects and 21 comparisons) through EDA supports the argument, indicating outcomes more precisely than conventional analysis of variance. (SLD)

For this evaluation of a multicultural education unit, Staged Innovation Design, which allows for experimental group posttest scores to be compared to control-replication group pretest and posttest scores to ascertain instructional effectiveness, was modified by adding a simultaneous pretest for both groups to control for threats to internal…

The purpose of this study was to compare the perceptions of teacher competence within and among three groups of educators as measured by the interview scales of the Instrument for the Observation of Teaching Activities (IOTA). (Auhor)

Although there is a healthy trend toward employment of multivariate statistical methods in the analysis of data from experiments in psychology and education which involve multiple dependent variables, there remains a need to demonstrate the wide variety of data analytic techniques available for post hoc analysis of statistically significant…

Marascuilo and Levin's (1970) notion of Type IV errors is extended, with respect to the interpretation of interactions in analysis of variance (ANOVA) designs. To help clarity what an interaction is and what it is not, in terms of the ANOVA model, the following points are made: (i) interactions should be thought of as linear contrasts involving…

The various costs of Type I and Type II errors of inference from data are discussed. Six methods for minimizing each error type are presented, which may be employed even after data collection for Type I and which minimizes Type II errors by a study design and analytical means combination. (Author/CM)

This paper provides an understanding of the conceptual and operational relationships between evaluation, content analysis, and microcomputers so that evaluation practitioners can better address applied content analysis problems used in their work. The first section presents a general model for conducting an evaluation effort that focuses on…

This document presents papers given at several conferences of Naval Training research personnel, consultants, and investigators from Washington University of St. Louis who were under contract with the Office of Naval Research to develop and test strategies for evaluating training. Part I covers basic issues in training research--learning ability;…

The evaluation design, which prescribes when and from whom to gather data, is described as significant because it can lend credibility to evaluation results. The objectives of this booklet are to aid the evaluator in: choosing a design; putting it into operation; and analyzing and reporting the data. Examples include both formative and summative…

The purpose of this study was to empirically determine the effects of quantified violations of the underlying assumptions of parametric statistical tests commonly used in educational research, namely the correlation coefficient (r) and the t test. The effects of heterogeneity of variance, nonnormality, and nonlinear transformations of scales were…

Three common methodological problems in the analysis of observational data include: (1) failure to base degrees of freedom for the analysis on the appropriate sampling unit; (2) the validity of the design and data are inconsistent with the generalizations reached; (3) the statistic used to calculate coder reliability is often inappropriate and…

Although methodological integrity is not the sole determinant of the value of a program evaluation, decision-makers do have a right, at a minimum, to be able to expect competent work from evaluators. This paper explores five areas where evaluators might improve methodological practices. First, evaluation reports should reflect the limited…

Research is a useful tool for educators because of its ability to uncover causal relationships between variables that are often obscured in field investigations. This textbook presents basic methodological information that has been designed for and proven to assist the educator who seeks to move himself from the laboratory to the field. The book…