Broader phenotypes associated with genetic liability, including mild difficulties with pragmatic language skills, have been documented in mothers of children with autism spectrum disorder (ASD) and mothers of children with fragile X syndrome (FXS). This study investigated the relationship between pragmatic difficulties and indicators of maternal well-being and family functioning. Pragmatic difficulty was associated with loneliness in mothers of children with ASD or FXS, and with depression, decreased life satisfaction, and poorer family relationship quality in mothers of children with FXS only. Results inform subtle maternal pragmatic language difficulties as a risk factor that that may contribute to reduced health and well-being, informing tailored support services to better meet the unique needs of families of children with ASD or FXS.