Purpose: Core lexicon measures have received growing attention in research. They are intended to provide clinicians with a clinician-friendly means to quantify word retrieval ability in discourse based on normal expectations of discourse production for specific discourse elicitation tasks. To date, different criteria have been used to develop core lexicon measures by groups of researchers. The need for statistical guidance in pursuit of the psychologically robust measure has been recognized. Aims: This study was to investigate the best criterion for accurate measurement. Specifically, we focused on two criteria (frequency vs. percentage) that have previously been used for the development of core lexicon measures. Method: Core lexicon measures consisting of five different checklists by word class (verbs, nouns, adjectives, adverbs, and function words) and developed by the two criteria were applied to language samples produced by 470 cognitively healthy adults. Performance in word retrieval ability at the discourse level was modeled as a latent variable based on the observed proportions of the production of core lexicon items in two different sets of core lexicon measures using structural equation modeling. Results: Results indicated that both criterion for core lexicon measures capture word retrieval ability in discourse. Greater residual variances were found in the core lexicon measure established by the percentage criterion compared to the one established by the frequency criterion. This indicates that the measure based on the percentage criterion is more affected by measurement errors. Conclusions: The findings provide evidence that the frequency criterion is better to use for the development of core lexicon measures for core nouns, verbs, adjectives, and adverbs, but not for function words. However, our findings are limited to core lexicon measures based on language samples elicited by wordless picture books. This may not be easily applied to other core lexicon measures that use different discourse elicitation tasks due to the difference in quality and quantity of language samples. Ideally, the same approach should be replicated to evaluate the appropriateness of respective criteria in the development of core lexicon measures.