This document answers the question, "What has been learned about evaluation methodology from the decade of compensatory education"? Ten issues dealing with the evaluation of Title I were identified within a general theoretical framework of evaluation. For each issue it was the aim of this document to do the following: 1)to clarify the…

The effects of using different data analysis methods on estimates of treatment effects of educational programs were investigated. Various regression models, such as those recommended for Title I program evaluations, were studied. The first effect studied was the amount of bias that might be expected to occur in the various settings. Results…

Albuquerque's Title I evaluation staff is in the process of generating a comprehensive local evaluation design because it considers the federally required product evaluation unsatisfactory. The required mean-gain comparisons were extended beyond the dimension of program to the dimensions of school, grade, and Title I instructor. This evaluation…

The need for competent formal evaluation programs, particularly for new federally assisted programs, is expressed. Problems in defining educational evaluation and its requirements, in designing such evaluations, and 3 possible sources of faulty conceptual bases for evaluations, are presented. An attempt is made to define evaluation in general, to…

Twenty-six papers concerning Elementary and Secondary Education Act Title I program evaluation--presented at the 1978 American Educational Research Association/National Council on Measurement in Education meeting--are summarized. Speakers from the federal government are: C.T. Cross (see ED 156 695); R.L. Fishbein; and D. Ozenne. State department…

Meta-analysis of Title I program evaluations shows the norm-referenced model overestimates positive effectiveness; while the regression-discontinuity design underestimates it. Potential biases include residual regression artifacts, attrition and time-of-testing problems in the norm-referenced design, and assignment, measurement, and data…

Educational program evalaution is subject to many difficulties. Problems with evaluation include such conceptual problems as: (1) understanding of decisions to be made; (2) adequate definition; (3) consensus on values and criteria; (4) coordination among administrative levels; and (5) use of an appropriate evaluation model. Technical problems…

Threats to the validity of the Title I Evaluation and Reporting System are covered in two parts: an annotated bibliography of reports concerned with technical issues, and a discussion of threats to validity--from the reporting system in general and from each model in particular. Threats common to all three evaluation models are reported:…

Twelve common errors which can invalidate an otherwise sound evaluation are identified, and ways to avoid them are presented. The hazards are: (1) grade-equivalent scores; (2) inappropriate statistical adjustments with nonequivalent control groups; (3) administering norm-referenced tests at inappropriate times of the school year; (4) inappropriate…

The norm-referenced evaluation model (RMC Model A) for Title I project evaluation, consists of procedures whereby the expected posttest standing of a treatment group under the null condition is generated from their pretest standing. It is assumed that the treatment group is not selected on the basis of their pretest scores and can be considered…

Title I evaluations using the RMC Model C design depend for their interpretation on the assumption that the regression of posttest on pretest is linear across the cut score level when there is no treatment; but there are many instances where nonlinearities may occur. If one applies the analysis of covariance, or model C analysis, large errors may…

The Elementary and Secondary Education Act Title I Evaluation and Reporting System is a method for giving a federally funded project in reading or math an overall score on its cognitive effectiveness. This System introduced the Normal Curve Equivalent (NCE) as an aid in aggregating Title I program scores across states and nationwide regardless of…

The three RMC models endorsed by the U.S. Office of Education for the evaluation of Elementary and Secondary Education Act Title I programs are based on narrowly conceived approaches to evaluation--the measurement of cognitive achievement gains. Each model requires the comparison of observed student performance with an estimate of what level of…

This document examines the following aspects of the federal administration of evaluation, program improvement, and technical assistance under Chapter 1 of the Education Consolidation and Improvement Act: (1) changes in each of the three areas since 1981 when Chapter 1 replaced Title I of the Elementary and Secondary Education Act; (2) activities…