This is the second of five papers detailing our national study of general education astronomy students' conceptual and reasoning difficulties with cosmology. This article begins our quantitative investigation of the data. We describe how we scored students' responses to four conceptual cosmology surveys, and we present evidence for the inter-rater reliability of those scores. We devote the bulk of this article to a classical test theory analysis of the data. We calculate difficulties and discriminations for each item, and we compute Cronbach's [alpha] as a measure of the reliability of the surveys. We also discuss the implications this analysis has for the validity of the surveys. (Contains 5 tables.) [For Part I, see EJ956291.]