Recent reforms in K-12 education have increasingly emphasized the educational system as the unit of change. The theory underlying systemic or systemwide educational reform is that powerful improvements in teaching and learning can come from developing coherence and alignment across the complex and different elements and components of an educational system. The growth of systemic reform efforts has led to a rise in attempts to systemically evaluate these initiatives. Systemic evaluations involve several distinct challenges for evaluators including capturing the system as the unit of intervention, assessing the alignment and coherence between components within the system, and identifying reasonable external comparison groups. Using the example of a school district's systemic reform initiative, the authors examine the challenges and implications of systemic evaluation.