This study attempts to clarify some of the issues involved in defining the effectiveness of compensatory education programs. It discusses the extent to which conclusions about the effectiveness of compensatory education programs are affected by two major components of an evaluation: the period of time on which the evaluation is based and the standard against which the program's effectiveness is judged. It is argued that the philosophy of compensatory education suggests that evaluations should measure program effectiveness over a period of time longer than the school year; in other words, that evaluations should assess the extent to which effects are sustained. Achievement gains for several programs, based on at least two periods of time, are calculated. The primary finding of these analyses is that conclusions about program effectiveness, regardless of what standard is used, are greatly influenced by the period of time over which the program is judged. Specifically, it is shown that the inclusion of the summer months in the evaluation can substantially reduce estimates of achievement and often reverse positive judgements of program effectiveness. The data demonstrate that programs can show evidence of sustained effects. (Author/AM)