Background: ChatGPT, a generative artificial intelligence (GenAI) chatbot, has gained significant traction as a tool for supporting students learning. Despite its growing popularity, there is still no academic consensus on its effectiveness in enhancing students' academic achievement. Objectives: This study aims to explore the effect of ChatGPT on students' academic achievement through a meta-analysis. It seeks to identify the overall effect size and examine variations based on moderators such as educational level, discipline, intervention duration, sample size, knowledge type, instructional model, role-setting, learning approach, and generated content. Methods: A meta-analysis was conducted on 37 studies (comprising 37 effect sizes) published between 2022 and 2025. The studies were analysed to calculate the overall effect size (Hedges' g) and to explore subgroup differences. Results and Conclusion: The findings reveal that ChatGPT has a moderately positive impact on students' academic achievement, with an overall effect size of g = 0.577 (95% CI [0.395, 0.759], p < 0.001). Further analysis of moderating variables indicates that no significant differences are observed across educational levels, role-setting, or learning approaches. A greater effect is observed in the social sciences compared to other disciplines; an intervention duration of 5-10 weeks has a larger impact on academic achievement compared to other durations; sample sizes ranging from 21 to 40 participants exhibit a larger impact on academic achievement than other sample sizes; ChatGPT is more effective in supporting the learning of declarative knowledge compared to procedural knowledge; the combination of traditional classrooms with ChatGPT is more effective than using ChatGPT in a flipped classroom; compared to generating code, using ChatGPT to generate text has better academic achievement.