Purpose: Concerns about variation in the quality of pre-service preparation provided by many university-based principal preparation programs (PPPs) has led to calls to use outcomes of program graduates to hold PPPs accountable, which the recent growth of state administrative data systems has made feasible. Little research, however, has assessed the degree to which different outcomes for PPP graduates in fact vary systematically by program. Research Methods: Using administrative data from Tennessee, we link approximately a decade's worth of PPP graduates to their schools, licensure examination scores, and multiple measures of job performance in their first three years as principals, including supervisors' practice ratings on the state evaluation system, teacher and assistant principal ratings of school leadership on a statewide survey, and measures of student achievement growth. We use which PPP a principal completed to predict these outcomes using a regression approach with different sets of covariates. Findings: Although we are able to associate PPPs with high and low principal performance, we find that programs' rankings vary by outcome measure, and we are unable to identify PPPs that perform consistently well or poorly across outcomes. Moreover, we find that Tennessee PPPs vary substantially in the characteristics of the schools into which their principals are hired, and that taking these characteristics into account is important in ordering PPPs based on outcomes. Implications: Although the use of graduates' outcomes to assess PPPs holds promise, policymakers and researchers may arrive at very different assessments of which PPPs are successful depending on which outcomes are chosen and what modeling approaches are employed. [For a related journal article, see EJ1201546.]