Quality Rating and Improvement System (QRIS) is a federally funded system that addresses quality in early care and education in many states throughout the country. At the heart of every QRIS system is a quality appraisal system that measures child care quality against state or regionally established standards. While QRISs are being implemented to measure program quality, influence subsidy rates, and guide quality improvement strategies, research validating QRIS performance as a predictor of child benefit is only beginning. This paper will present results from an item level psychometric analysis of a state developed QRIS tool using Item Response Theory, specifically Rasch analyses, and will share implications of the results for child care policy and implementation of the tool.