The current study illustrated a mediation model of different aspects of family environmental factors in predicting short-term memory and social understanding in Chinese children. The study investigated how children's short-term memory and social understanding were predicted by family environment factors, including socioeconomic status, home crowdedness, household chaos, and the number of children's books at home (books) among 525 children in Hong Kong and Mainland China. Data on social understanding, socioeconomic status, crowdedness, household chaos, and books were collected by parent questionnaires. Both verbal and visual short-term memory were measured individually. The results showed that the books mediated the role of crowdedness to both verbal and visual short-term memory in Mainland China, whereas in Hong Kong, crowdedness directly predicted two short-term memory with socioeconomic status controlled. Moreover, the role of crowdedness to social understanding is fully mediated by household chaos in Hong Kong and by both household chaos and books in Mainland China. Our study highlighted that manipulating soft aspects of family environmental factors (maintaining an organized, less chaotic family environment, putting more books at home) could enhance children's fundamental short-term memory capacity and social functioning.