One of the most important goals in a statistics class is to develop students who are statistically literate and can reason with statistical concepts. The REALI instrument was designed to concurrently assess statistical literacy and reasoning in introductory statistics students. This paper reports a measurement analysis of the statistical literacy and reasoning subscores from the REALI assessment and the extent to which they are reliable and distinct. Investigation of these subscores is used clarify the relationship between the constructs of statistical literacy and statistical reasoning and to what extent they overlap. The results of this analysis, under a Multidimensional Item Response Theory framework, show that the statistical literacy and reasoning subscores provide no added value over a single general statistical knowledge score. This indicates the two constructs might be indistinguishable from one another.