A school improvement program that provided support to poor-performing schools on the basis of needs identified in a school improvement plan was implemented in 72 government schools in Jamaica, from 1998 to 2005. In this independent evaluation of the program, we use propensity score matching to create, post hoc, a control group of schools that were similar to program schools in the baseline year. By the final year of the program, we find that program schools had received more inputs to improve literacy and numeracy than control schools, and that some inputs associated with the program were correlated with improvements school average achievement: supplementary reading materials, additional training for reading resource teachers, and functioning computers. At the student level, however, we find no evidence that students enrolled in program schools achieved higher reading or math scores than those in control schools. We suggest three possible reasons for this: (a) the lack of sensitivity of the learning measures to improvements at the lower end of the scales; (b) the availability of program-like inputs in non-program schools, provided by other programs and donors; and (c) the growth in student enrollment in the program schools, which may have diluted the program effect for incoming students in upper grades. Schools with school improvement plans did not outperform comparable schools that did not have these plans. (Contains 12 tables.)