Methods to generate random correlation matrices have been proposed in the literature, but very few instances exist where these correlation matrices are structured or where the statistical properties of the algorithms are known. By relying on the tetrad relation discovered by Spearman and the properties of the beta distribution, an algorithm is proposed to uniformly generate correlation matrices parameterized by the One Factor model with no error covariances (i.e. the congeneric test model). The characteristics of this algorithm are discussed and evaluated. An application of this algorithm is presented to explore the distribution of Cronbach's coefficient alpha as the scale length of a test increases arbitrarily. Recommendations are presented on how the use of this method (or other conceptually similar ones) can be helpful to create benchmark distributions for reliability coefficients and other psychometric statistics to evaluate empirical findings in the literature.