This document answers the question, "What has been learned about evaluation methodology from the decade of compensatory education"? Ten issues dealing with the evaluation of Title I were identified within a general theoretical framework of evaluation. For each issue it was the aim of this document to do the following: 1)to clarify the…

The effects of using different data analysis methods on estimates of treatment effects of educational programs were investigated. Various regression models, such as those recommended for Title I program evaluations, were studied. The first effect studied was the amount of bias that might be expected to occur in the various settings. Results…

The site variability issue for Direct Instruction Follow Through sites is examined from a variety of perspectives--meta-analysis of the Abt Associates' data, norm-referenced comparisons, and a longitudinal look at stability of effects. Much of the seemingly unexplained variability found in both the Abt report and the reports by M. Kennedy and E.…

Clarifies a general approach to understanding teaching effectiveness research, outlines the requirements of the kind of research that is believed likely to result in verified generalizations about the primary dimensions along which effective and ineffective teachers vary, and provides an example of the kind of research it is believed is needed.…

Problems encountered in large-scale educational evaluations are discussed using the evaluation of the Emergency School Aid Act as an example. This extensive article focuses primarily on methodological issues. (JKS)

This paper reviews the following studies on the cost-effectiveness of the Chapter 1 compensatory education program: (1) Sustaining Effects Study; (2) Tallmadge Study; (3) the Kiesling Study for Rand; (4) Instructional Dimensions Study; (5) Response to Educational Needs Project Cost-Effectiveness Study; (6) Educational Testing Service/Ragosta…

Twenty-six papers concerning Elementary and Secondary Education Act Title I program evaluation--presented at the 1978 American Educational Research Association/National Council on Measurement in Education meeting--are summarized. Speakers from the federal government are: C.T. Cross (see ED 156 695); R.L. Fishbein; and D. Ozenne. State department…

Using examples from evaluations of the Emergency School Aid Act (ESAA) Basic Grants Program, the ESAA Pilot Program, and the sustaining effects of compensatory education programs, school and student attrition are discussed. As in the example cases, appreciable attrition can be expected in most longitudinal studies. The possible effects of this…

Meta-analysis of Title I program evaluations shows the norm-referenced model overestimates positive effectiveness; while the regression-discontinuity design underestimates it. Potential biases include residual regression artifacts, attrition and time-of-testing problems in the norm-referenced design, and assignment, measurement, and data…

This conference report identifies research needs in evaluating and documenting large scale school improvement efforts to serve disadvantaged populations. Summaries of the conference presentations are provided, grouped into three sections. The first section examines the basis of conducting evaluations. Several basic assumptions are challenged,…

Possible causes for the discrepancy between the ABT report on the national Project Follow Through and the Saint Louis, Missouri program were discussed: (1) ABT data were positively biased because of non-random, differential attrition from the program and control groups; (2) positive ABT data were due to use of a particular cohort group; or (3)…

A quality control review of Connecticut's Sustained Effects studies was undertaken by the Chapter 1 Evaluation Technical Assistance Center (TAC) operated by RMC Research Corporation. Nineteen Connecticut school districts had selected from among four basic evaluation designs in implementing the Chapter 1 Sustained Effects requirement during 1982 to…

Educational program evalaution is subject to many difficulties. Problems with evaluation include such conceptual problems as: (1) understanding of decisions to be made; (2) adequate definition; (3) consensus on values and criteria; (4) coordination among administrative levels; and (5) use of an appropriate evaluation model. Technical problems…

Threats to the validity of the Title I Evaluation and Reporting System are covered in two parts: an annotated bibliography of reports concerned with technical issues, and a discussion of threats to validity--from the reporting system in general and from each model in particular. Threats common to all three evaluation models are reported:…

Both administratively and in terms of their evaluations, Head Start and Follow Through have been treated as separate programs. Follow Through was initially conceived, however, as an effort to preserve and augment Head Start gains. In this report, achievement test data on a set of children who participated in both Head Start and Follow Through are…