Background: Fragile X syndrome (FXS) is the most common known inherited cause of intellectual disability, yet very few studies have explored the language comprehension skills of children with FXS. We examined the receptive vocabulary, grammatical morphology and syntax skills of boys with FXS (who were additionally classified as having autism, autism spectrum, or no autism) and compared them to boys with Down syndrome (DS) and typically developing (TD) boys at similar non-verbal developmental levels. Methods: The Vocabulary, Grammatical Morphology, and Elaborated Phrases and Sentences subtests of the Test for Auditory Comprehension of Language--3rd Edition (TACL-3) were administered annually up to three times to assess the language comprehension skills of 35 boys with FXS without autism, 24 boys with FXS with autism spectrum, 19 boys with FXS with autism, 45 boys with DS and 40 TD boys at similar non-verbal cognitive levels. Results: After controlling for non-verbal cognition and maternal education levels, we found that the three groups of boys with FXS did not differ from each other but scored lower than the TD boys in language comprehension. The boys with DS scored lower in language comprehension than boys with FXS without autism and TD boys. For all of the groups, scores for receptive vocabulary, grammatical morphology and syntax did not differ. Conclusions: Boys with FXS and boys with DS differed in receptive language levels, demonstrating unique language profiles for each syndrome. Language comprehension appears to be an important area to target in assessment and intervention for both populations.