Children born to parents with fewer years of education are more likely to have depression, anxiety, and attention-deficit hyperactivity disorder (ADHD), but it is unclear to what extent these associations are causal. We estimated the effect of parents' educational attainment on children's depressive, anxiety, and ADHD traits at age 8 years, in a sample of 40,879 Norwegian children born in 1998-2009 and their parents. We used within-family Mendelian randomization, which employs genetic variants as instrumental variables, and controlled for direct genetic effects by adjusting for children's polygenic indexes. We found little evidence that mothers' or fathers' educational attainment independently affected children's depressive, anxiety, or ADHD traits. However, children's own polygenic scores for educational attainment were independently and negatively associated with these traits. Results suggest that differences in these traits according to parents' education may reflect direct genetic effects more than genetic nurture. Consequences of social disadvantage for children's mental health may however be more visible in samples with more socioeconomic variation, or contexts with larger socioeconomic disparities than present-day Norway. Further research is required in populations with more educational and economic inequality and in other age groups.