Background: This study examined the impact of the reading-related affective factors "home environment" and school "environment" on predicting the likelihood of students being either high-proficiency or low-proficiency readers. Data from 3,875 Hong Kong SAR Grade 4 students participating in an international comparative assessment were analyzed. Methods: Multilevel regression analysis was used to model the relationship between affective factors (i.e., reading attitude, reading motivation, and reading self-concept) peer bullying, family context (i.e., home socioeconomic status/SES), and school context (i.e., school SES, school bullying, and school safety and order). Results: The likelihood of being a reader with high proficiency was found to be associated with reading attitude, reading motivation, reading self-concept, peer bullying, school bullying, and school SES, whereas the likelihood of being a reader with low proficiency was associated with reading self-concept and peer bullying only. Conclusions: These findings suggest that reading-related affective characteristics and school context may be more likely to promote rather than limit reading success.