An analytical procedure that uses fall kindergarten assessment data to retroactively create equivalent comparison groups for longitudinal research and evaluation studies was designed and tested, and then analytically and empirically compared with random assignment to treatment. Students entering kindergarten in Hawaii are administered the Peabody…

A meta-analysis was conducted on 44 educational studies that used either a (labelled) Hawthorne control group, a manipulation of Hawthorne effects, or a group designed to control for the Hawthorne effect. The sample included published journal articles, ERIC documents or unpublished papers, and dissertations. The studies were coded on 20 variables,…

A solution is proposed for analysis of variance procedures with missing cells, such as may occur when a control group is not assigned to any of the rows or columns of the various experimental groups. Mathematical models for two-way design are presented which define several variables; as well as row effect, column effect, and row and column…

A method is proposed for calculating the sample size needed to achieve acceptable statistical power with a given test. The minimally important difference significant (MIDS) criterion for sample size is explained and supported with recommendations for determining sample size. The MIDS criterion is computationally simple and easy to explain. (SLD)

An analysis of covariance (ANCOVA) is done to correct for chance differences that occur when subjects are assigned randomly to treatment groups. When properly used, this correction results in adjustment of the group means for pre-existing differences caused by sampling error and reduction of the size of the error variance of the analysis. The…

The presence of artifactual bias in analysis of covariance (ANCOVA) and in matching nonequivalent control group (NECG) designs was empirically investigated. The data set was obtained from a study of the effects of a television program on children from three day care centers in Mexico in which the subjects had been randomly selected within centers.…

Suggestions, based on the experience of researchers at Oak Ridge (Tennessee) Associated Universities (ORAU), are offered on how evaluations in which true experimental designs are not possible can be designed for meaningful comparison. The ORAU conducts evaluation and assessment studies of educational programs intended to explore some aspect of the…

Problems with conducting randomized field experiments in education are explored. Focus is on problems encountered while evaluating a group of dropout prevention projects. Project planners were asked to manipulate the subject eligibility criteria until they identified as eligible three to four times as many students as they could serve. They were…

A survey was conducted of expert instructional design practitioners about their use of developmental testing and revision procedures. The degree of correspondence was examined between published recommended practice and actual practice, as reported by 90 respondents to a questionnaire. The surveyed sample were members of the National Society for…

Research efforts have established that the Mantel-Haenszel procedure (MHP) is an effective method for detecting the presence of test items exhibiting differential item functioning (DIF). While the MHP has been advocated for situations where item response theory based methods may not be usable, recent findings have suggested that the performance of…